Link Search Menu Expand Document
calendar_month 28-Jan-25

Overlay Fabric Orchestration

This section describes how to deploy a NetConductor EVPN-VXLAN overlay on a deployed underlay using the Underlay Wizard.

A three-tier network identified as Herndon is used to illustrate the process.

Some steps require manual CLI configuration of switches. Complete these steps using the MultiEdit feature of Central UI groups.

Table of contents

Network Review

Best practice uses the NetConductor Underlay Orchestration workflow to create an OSPF underlay. After all requirements for the Fabric workflow are met, the EVPN-VXLAN overlay can be built.

Fabric deployment procedures assume that a large campus underlay was created using the Underlay Orchestration workflow. If the campus was deployed using the VSG Campus Wired Connectivity procedures, review the brownfield migration considerations.

The following illustration shows a Layer 3 access configuration along with the fabric personas used during the EVPN configuration. Device fabric personas to be assigned are shown in bold. Access aggregation switches perform only underlay functions and do not have fabric personas.

The Herndon site is summarized as 10.10.0.0/20.

The following table lists the IP subnets of the underlay networks.

DescriptionIP Subnet
Routed Interface IP Pool10.10.0.0/24
Loopback IP Pool10.10.1.0/24
Gateway Underlay IP Space10.10.9.0/28

The following table lists the IP subnets used to deploy the distributed overlay. Best practice is to segment wired and wireless traffic into separate networks and segment the gateways and APs to isolate AP broadcast traffic.

DescriptionVRFVLANIP Subnet
Gateway ManagementInfrastructure30110.10.2.0/24
UXI Sensor ManagementInfrastructure30210.10.3.0/24
AP ManagementInfrastructure30310.10.4.0/24
Overlay Fabric WiredCorporate10010.10.5.0/24
Overlay Fabric WirelessCorporate10210.10.6.0/24
Overlay Fabric Wired - GuestGuest20010.10.7.0/24
Overlay Fabric Wireless - GuestGuest20110.10.8.0/24

DHCP Considerations

Ensure that DHCP scopes exist for the above subnets. A scope must be created for VTEP loopbacks and excluded from DHCP allocation. This is required for most DHCP servers to accept the DHCP Discovery. The DHCP must be configured to accept option 82. If the DHCP server is not configured to accept option 82, when it receives requests containing option 82 information, it cannot use the information to set parameters and it cannot echo the information in its response message. Older servers, such as Windows 2008, may not support option 82.

Note: In the fabric wizard, changing the DHCP Server VRF to any VRF other than default results in the creation of additional loopbacks with the same IP address as loopback 1 in a selected VRF.

Device Onboarding Considerations

An out-of-band management network for infrastructure devices is recommended when possible, but not required. In this procedure, the switches are managed out-of-band, but the APs, Gateways, and UXI sensors are managed in-band.

Gateways use an underlay interface for initial communication with Central, and are then migrated to an overlay interface for communication and tunneling with the access points. Access points and UXI sensors are managed in the overlay.

When devices are managed in-band, various levels of infrastructure must be configured so the device is reachable. For the access points and UXI sensors, the fabric must be fully provisioned and the border handoff must be configured to extend reachability for the overlay networks. Because the gateways are initially managed in the underlay, they are reachable as long as the underlay is fully configured and extended via the border handoff.

Configure Role Policy

Note: Customers must contact their Aruba Account Manager for addition to the allow-list for the Global Policy Manager feature.

Roles and role policies are provisioned at a global level and apply to all fabrics. This procedure uses two sample roles:

  • EMPLOYEE
  • CONTRACTOR

The following example role-to-role policy prevents employees and contractors from communicating.

Create Roles

Use this procedure to create the EMPLOYEE and CONTRACTOR roles:

Step 1 In the filter dropdown, select Global, if it is not already selected. On the left menu, select Security.

Step 2 Click the Client Roles tab.

Step 3 Create a new role by clicking the + (plus sign) in the upper right corner of the table.

Step 4 In the Create new role window, assign the following settings, then click Save.

  • Name : EMPLOYEE
  • Description: <insert optional role description>
  • Policy Identifier: <use default value>
  • Allow default role to source role permissions for wired clients: <selected>

Note: The Allow default role to source role permissions for wired clients option creates policy rules that allow clients assigned the role to send and receive ARP packets and traffic from outside the fabric.

Step 5 Repeat steps 3 and 4 to create the CONTRACTOR role and enter an optional description.

Define Role-to-Role Policy

Use this procedure to create a policy to prevent the EMPLOYEE and CONTRACTOR roles from communicating.

Step 1 Mouse-over the CONTRACTOR row and click the edit icon (pencil) on the right.

Step 2 In the PERMISSIONS edit window for the CONTRACTOR role, click the edit icon (pencil) at the top right. The Assign Permissions window appears.

Step 3 In the Assign Permissions window, assign the following settings and click Save.

  • CONTRACTOR (self):
    • Allow Source to Destination: checked
    • Allow Destination to Source: checked
  • EMPLOYEE:
    • Allow Source to Destination: unchecked
    • Allow Destination to Source: unchecked

Step 4 Repeat steps 1 to 3 for the EMPLOYEE role using the following settings.

  • CONTRACTOR
    • Allow Source to Destination: unchecked
    • Allow Destination to Source: unchecked
  • EMPLOYEE (self):
    • Allow Source to Destination: checked
    • Allow Destination to Source: checked

Deploy the Fabric

Use the fabric wizard to deploy an overlay fabric. Follow the procedures below to provision the VXLAN interfaces, EVPN control plane, VRFs, fabric VLANs, and Anycast Gateways.

Note: All Aruba CX switches included in the fabric must be in the same Central UI group and have advanced licenses.

Create The Fabric

Step 1 In the Global dropdown, select the switch group. In this example, the group is HERCP-FAB.

Step 2 On the left menu, select Devices.

Step 3 Select Switches, then select Config.

Step 4 Under Routing, select Fabrics.

Step 5 On the Fabrics table, click the + (plus sign) at the top right.

Step 6 In the Create a New Fabric workflow, click Name Fabric, assign the following settings, and click Next.

  • Fabric Name: Herndon-Fabric

  • BGP AS Number: Use default

  • VLAN Client Presence Detect: Enabled

  • FIB Optimization: Disabled

Enabling VLAN Client Presence Detect is recommended to provide increased scalability by avoiding flooding BUM traffic to VTEPs when a client is not detected for a given VLAN. Disabling FIB Optimization is recommended because it can affect the control plane based on traffic patterns.

Step 7 On the Add Devices page, select each access switch and use the Assign selected devices to window to assign the Edge persona. Click Apply.

Step 8 Repeat steps 6 and 7 for the RR, Border, and Stub device personas, then click Next.

DevicePersona
HERCP-CR1-1RR
HERCP-CR1-2RR
HERCP-CR1-STB1-1Stub
HERCP-CR1-STB1-2Stub
HERCP-CR1-BRDR1-1Border
HERCP-CR1-BRDR1-2Border
HERCP-AG1-AC1Edge
HERCP-AG1-AC2Edge
HERCP-AG2-AC1Edge
HERCP-AG2-AC2Edge

Note: You must click Apply after each persona selection to save the assignment.

Note: Because aggregation switches are underlay devices, they are not assigned personas.

Step 9 On the Add Overlay Network click the + (plus sign) at the top right.

Step 10 Assign the following Overlay Network settings:

  • Name: Corporate
  • VNI: 10000

Step 11 Repeat step 9 and 10 for the Guest and Infrastructure networks.

Note: Rename or delete the default overlay_network.

Step 12 On the Stub Tunnels to Gateway page, click the + (plus sign) at the top right of the table.

Step 13 In the Tunnels table, assign the following settings. Click outside the new row to continue.

  • Switch: HERCP-CR1-STB-1
  • Gateway List IP: 10.10.9.4, 10.10.9.5

Note: Gateway IPs must match the VXLAN Tunnel Source configured on the AOS-10 Gateways, configured later in this procedure.

Step 14 Repeat steps 10 and 11 for additional stub switches. Click Next.

Step 15 Review the Summary page for accuracy. Return to previous pages and make corrections, if needed. Click Save.

Create the Fabric Segments

Follow these steps to create segments within the fabric.

Step 1 Expand the Herndon-Fabric, then click the New Segment icon.

Step 1 On the Overlay Network & VLAN page of the New Segment workflow, assign the following settings using the + (plus sign) to add DHCP servers, and click Next.

  • Overlay Network: Corporate
  • VLAN Name: Overlay Fabric Wired
  • VLAN ID: 100
  • Default Gateway IP: 10.10.5.1
  • IPv4 Version: IPv4
  • Subnet Mask: 24
  • DHCP Server: 10.2.120.98, 10.2.120.99
  • DHCP Server VRF: default

Step 3 Skip the role mapping page by clicking Next.

Note: It is recommended not to map roles to segments at this stage. Instead, use the NAC server to assign both the VLAN and Role, which is specified in the RADIUS response during device authentication.

Step 4 On the Devices page, select the Edge devices, then click Next.

Step 5 Review the Summary page for accuracy, then click Save.

Step 6 Repeat the step above until all segments are created.

VLAN NameOverlay NetworkVLAN IDIP SubnetApply to Devices
Gateway ManagementInfrastructure30110.10.2.0/24Stub VTEPs
UXI Sensor ManagementInfrastructure30210.10.3.0/24Edge VTEPs
AP ManagementInfrastructure30310.10.4.0/24Edge VTEPs
Overlay Fabric WirelessCorporate10210.10.6.0/24Stub VTEPs
Overlay Fabric Wired - GuestGuest20010.10.7.0/24Edge VTEPs
Overlay Fabric Wireless -GuestGuest20110.10.8.0/24Stub VTEPs

Configure Wireless Integration

This procedure assumes that the AOS-10 Gateway has been configured according to the Campus Gateway Deploy Guide. The following VLANs are required:

  • VLAN 10: Underlay VLAN
  • VLAN 301: Gateway Management
  • VLAN 102: Overlay Fabric Wireless
  • VLAN 201: Overlay Fabric Wireless Guest

VLAN 10 serves as both the underlay VLAN and the source for the static VXLAN tunnel. VLAN 301 is dedicated to managing the gateway. VLAN 10 and VLAN 301 are trunked to the stub switches. VLAN 102 and VLAN 201 are SSID VLANs used for client access, and they extend over the static VXLAN tunnel.

Wireless gateways establish static VXLAN tunnels with the stub switch. This enables connectivity within the fabric and maps VLANs to VNIs. This procedure enables jumbo frames on gateway interfaces and configures the static VXLAN tunnels.

Before proceeding, ensure that the following are configured on the stub switch.

  • The IP MTU is set on the VLAN interface.

  • The MTU is configured on the LAG members of the stub switch.

The configuration shown below represents the initial setup on the stub switches at the start of this procedure.

interface vlan 10
    description UNDERLAY BETWEEN GATEWAY AND STUB
    ip mtu 9198
    ip address 10.10.0.66/28
    active-gateway ip mac a2:01:00:a2:a2:a2
    active-gateway ip 10.10.0.65
interface lag 11 multi-chassis
    description Stub-GW-1
    no shutdown
    no routing
    vlan trunk native 10
    vlan trunk allowed 1,10
    lacp mode active
    lacp fallback
    exit
interface lag 12 multi-chassis
    description Stub-GW-2
    no shutdown
    no routing
    vlan trunk native 10
    vlan trunk allowed 1,10
    lacp mode active
    lacp fallback
    exit
interface 1/1/5
    description HERCP-GW
    mtu 9198
    no shutdown
    lag 11
interface 1/1/6
    description HERCP-GW
    mtu 9198
    no shutdown
    lag 12

Add Gateway Overlay VLAN to Stub Switch Trunk

VLAN 301 was created on the stub with the Fabric Wizard. In this step, VLAN 301 is added to the LAG interfaces connecting to the gateways to the border.

Step 1 In the Global dropdown, select the switch group. In this procedure, the group is HERCP-FAB.

Step 2 On the left menu, select Devices.

Step 3 Select Switches, then select Config.

Step 4 Enable the MultiEdit toggle.

Step 5 Select the two stub switches.

Step 6 Click Edit Config.

Step 7 Enter the following configuration and click Save.

interface lag 11 multi-chassis
    description Stub-GW-1
    vlan trunk allowed 1,10,301
    exit
interface lag 12 multi-chassis
    description Stub-GW-2
    vlan trunk allowed 1,10,301
    exit

Note: Ensure that the fabric wireless VLANs (101,201) are not trunked in the underlay. This can cause loops in the network.

Verify Jumbo Frames on the Gateway

This procedure details how to enable jumbo frames on the AOS-10 Gateways.

Refer to Configuring Wireless Group Settings to assist with enabling jumbo frame processing.

Refer to Configure Gateway VLANs to assist with verifying jumbo frames on the port channel.

Configure VLANs on Gateways

This step configures additional required VLANs for the NetConductor deployment.

Step 1 In the Global dropdown, select the switch group. In this example, the group is HERCP-FAB.

Step 2 On the left menu, select Devices.

Step 3 Select Gateways, then select Config.

Step 4 Select the Interface tab and select VLANs. In the lower left, click the + (plus sign).

Step 5 In the New VLAN window, assign the following settings, then click Save Settings.

  • VLAN name: Overlay-MGMT
  • VLAN ID/Range: 301

Step 6 Repeat this procedure for VLAN 102 (Overlay-Fabric-Wireless) and VLAN 201 (Overlay-Fabric-Wireless-Guest).

Modify Port Channel on Gateways

Step 1 On the Gateways page, select the Interface tab, then the Ports tab.

Step 2 Select PC-0.

Step 3 Scroll to the port-channel configuration.

Step 4 From the dropdown, select VLAN 301 to add it.

Step 5 Click Save Settings.

Configure Static Routes

Use the following procedure to configure required routing on the gateway. A default route points to VLAN 301, which is in the fabric. A static route for the loopback IP space points to VLAN 10 and is used to establish the static VXLAN tunnel.

Step 1 On the Gateways tab, select the Routing tab, then the IP Routes tab.

Step 2 Expand the Static Default Gateway section. At the bottom of the table, click the + (plus sign).

Step 3 On the New Default Gateway page, enter the IP address, then click Save Settings.

  • Default Gateway IP: 10.10.2.1

Step 4 Select the original static default gateway and set the Cost to 50, making this a backup route. Then click Save Settings.

Note: Verify there is no local override in any of the gateway device static default configurations. The local overrides may have been set during onboarding. Remove all static default gateway local overrides.

Step 5 Expand the IP Routes section. At the bottom of the table, click the + (plus sign).

Step 6 On the New Default Gateway page, assign the following settings, then click Save Settings.

  • Destination IP Address: 10.10.1.0

  • Destination Network Mask: 255.255.255.0

  • Next hop IP Address: 10.10.9.1

Configure Static VXLAN Tunnel

Use the following procedure to configure static VXLAN tunnels between the gateways and stub switches:

Step 1 In the Global dropdown, select the switch group. In this example, the group is HERCP-FAB.

Step 2 On the left menu, select Devices.

Step 3 Select Gateways, then select Config.

Step 4 Click the Interface tab and click VXLAN Tunnels. Click the + (plus sign) at the lower left.

Step 5 On the Add VXLAN Tunnel page, assign the following settings:

  • IP Version: IPv4
  • VXLAN Tunnel Source: VLAN
  • VLAN Interface: 10
  • Virtual tunnel end point (vtep) peer IP: 10.10.1.14
  • MTU: 9198
  • Enable tunnel admin state: checked
  • Enable global policy identifier (gpid): *checked

Note: The 10.10.10.14 IP address above is the loopback1 IP address shared among the VSX pair devices acting as stub VTEPs. Use NetEdit to obtain the IP address from one of the border switches.

Step 6 Click the + (plus sign) in VLAN/VNI mapping, assign the following settings and click OK.

  • VLAN ID: 102
  • Virtual network identifier: 102

Step 7 Repeat the VLAN-to-VNI mapping for all SSID VLANs.

NameVLANVNI
Overlay Fabric Wireless102102
Overlay Fabric Wireless - Guest201201

Modify Role Policy on the gateway

AOS-10 gateways direct all traffic flows through the AOS-Firewall and apply policies determined by the user’s role. Each role has a role-to-role (r2r) policy that incorporates rules from the Global Policy Manager (GPM). Although the user interface may display this policy as empty, it contains the rules from the GPM.

To enable access to resources not defined in the GPM, such as DNS, DHCP, and the Internet, additional policies must be created for each role. This procedure adds the allowall policy, which serves as a default policy, to each role in order to deny any access explicitly defined in the GPM while allowing all other traffic. If more granular filtering is required, additional custom policies can be applied to specific roles.

Complete the procedure for each Role. The EMPLOYEE role is shown below.

Step 1 Select the HERCP-FAB group.

Step 2 On the left menu, select Devices.

Step 3 Select Gateways, then select Config.

Step 4 Click the Security tab. In the Roles section, select the EMPLOYEE role.

Step 5 Click the + (plus sign).

Step 6 Click the Add an existing policy button.

Step 7 Select Policy name allowall.

Step 8 Click Save.

Note: After adding the allowall policy, additional global and apprf policies appear. These are empty system policies that apply to all roles by default. If using the default policies, consider how they could impact forwarding when applied to roles.

Create Fabric SSID

Refer to Configuring Wireless Access to assist with creating an SSID named SSID-HERCP-01 that authenticates to ClearPass. Configure this SSID to place users into VLAN 301.

Configure External Connectivity

External connectivity can be configured in two distinct ways.

VRF Lite handoff allows each fabric VRF to connect to devices such as firewalls with multiple zones, upstream routing devices with extended VRFs, or the global routing table. Although this guide does not illustrate VRF Lite handoff, an example is available in the Data Center Deployment Guide. This option is suitable when connecting to devices that do not support EVPN-VXLAN.

Configuring an EVPN-VXLAN handoff enables the extension of both VRF and role information. When integrating with an HPE Aruba Networking SD-WAN solution, the role and VRF are maintained throughout the SD-WAN fabric. Detailed instructions for this configuration are available in the EdgeConnect SD-WAN Multi-Site chapter.

ClearPass Integration

RADIUS-based authentication is required on all edge ports participating in the fabric. ClearPass is the recommended solution.

Ensure that edge switches and edge ports are configured to support 802.1x. Refer to the Configure RADIUS and UBT section for guidance.

Modify the ClearPass services as needed to ensure that ClearPass returns a role and VLAN.

The below screenshot shows the RADIUS response returned by ClearPass after successful authentication.

Edge Port Configuration

Edge ports should be configured as colorless ports. Use Port Profiles to configure edge ports. Use Device Profiles to detect APs and UXI sensors dynamically and place them in the correct VLAN.

Verification

The steps below illustrate how to verify functionality for a distributed fabric deployment. Central provides a remote console that enables CLI access on any managed switch. Refer to the Verify OSPF Operation section for a more detailed overview.

Verify Underlay

Step 1 In a Remote Console window, type the command show ip ospf neighbors and press ENTER. Confirm that the state is “FULL” for all appropriate OSPF peers.

Step 2 In a Remote Console window, type the command show ip route and press ENTER. Confirm that all loopback0 and 1 /32 routes are listed.

Verify Overlay

The EVPN verification below is recommended for all fabric switches. VXLAN verification is recommended for the edge, border, and stub devices. Endpoint verification is recommended for edge switches.

Step 1 In a Remote Console window, type the command show bgp all summary and press ENTER. Confirm that BGP peering is active between the route reflectors and all fabric devices.

Step 2 In a Remote Console window, type the commandshow evpn evi and press ENTER. Verify the EVPN configuration and operational state.

Step 3 In a Remote Console window, type the command show bgp l2vpn evpn and press ENTER. Verify EVPN overlay routes.

Step 4 In a Remote Console window, type the command show evpn mac-ip and press ENTER. Verify that overlay MAC/IP address information is learned from EVPN.

Step 5 In a Remote Console window, type the command show interface vxlan 1 and press ENTER. Verify that VXLAN tunnels are established.

Step 6 In a Remote Console window, type the command show port-access clients and press ENTER. Verify the authentication state of an endpoint and confirm proper role assignment.

Step 7 In a Remote Console window, type the command show port-access gbp and press ENTER. Verify that configured GBP policies are applied.

Verify Gateways

VXLAN verification confirms the operational state of the static VXLAN tunnel. It is recommended for all gateways.

Step 1 In the Global dropdown, select the switch group. In this example, the group is CP-HER-FAB.

Step 2 On the left menu, select Devices.

Step 3 Select Gateways, then select Clusters.

Step 4 In the Name column, click the name of the fabric connected cluster.

Step 5 Select the Tunnels page.

Step 6 In the Tunnels table, find the stub switch VTEP address in the Destination Device column, and confirm that the Status column indicates “Up”.

Brownfield Considerations

A NetConductor fabric can be deployed over an existing OSPF underlay using the Fabric workflow. This is supported on underlays with a Layer 2 or a Layer 3 configuration to the access layer. Certain requirements must be met for successful deployment.

Step 1 All Aruba CX switches to be included in the fabric must be in the same Central UI group. Gateways and access points do not need to be in the same group.

Step 2 Migrate underlay configured switches to the fabric group in Central using the Retain CX-switch configuration option to preserve the existing underlay.

Step 3 Configure loopback0 as the interface for OSPF routers. Create loopback1 for use by the VXLAN configuration of the Fabric workflow.

Step 4 When deploying the EVPN fabric over an existing Layer 2 access deployment, create a transit VLAN from aggregation to access switches for running OSPF and enabling Layer 3 access to the loopback interfaces of the access layer switches.

Step 5 Configure all underlay switch interfaces for an MTU of 9198 bytes to ensure unfragmented transport of VXLAN packets through the network.