The Easy button for understanding ACI; VXLAN

When I have discussions regarding Cisco Data Center networking and I bring up ACI, there is always a common pushback that I hear; the learning curve is too steep, and they must relearn networking to use ACI.  The answer to that is, that while ACI like any other Software Defined networking platform requires some learning as you are approaching networking from a new perspective, the underlying networking rules and protocols that we’re all used to is still there.  And better yet, when it comes to learning ACI, there is an easy button and if you know it, you will quickly understand what’s going on in the ACI network.

The easy button is VXLAN.  How does VXLAN make things easy?  Because when you look at what’s going on in an ACI network, you are essentially looking at a VXLAN network with a centralized policy-based management system.  There are a few minor differences in the protocols being run across the network, but it is still a VXLAN network at its core.

A brief overview of VXLAN is that it stands for Virtual eXtensible LAN (Local Area Network) and it takes the logical separation of a layer 2 bridge domain that you would get from VLANs and pushes it further.  The standards for VXLAN were developed by VMware, Cisco, and Arista to be able to extend a layer 2 bridge domain across a layer 3 routing domain and to provide more logical segmentation versatility.  VXLAN uses a 24-bit VNID for tagging traffic which allows for 16 million segments as opposed to the 12-bit 802.1Q VLAN ID which only gives you 4096 segments.  VXLAN utilizes MAC-in-UDP encapsulation and tunneling to extend its layer 2 segments across a routed layer 3 network.  Layer 2 ethernet frame is placed inside of a UDP-IP packet as shown in the VXLAN Frame breakdown and sent across a VXLAN tunnel.

A VXLAN tunnel exists between the two devices at the ends of the VXLAN network that encapsulate the inner Ethernet frame.  These tunnels are stateless UDP tunnels.  The devices at each end of the VXLAN tunnel (whether physical or virtual) are VXLAN Tunnel Endpoints or VTEPs.  The VTEPs encapsulate and de-encapsulate the VLXLAN frame.  Each VTEP requires two interfaces, a local LAN layer 2 interface and an IP-based layer 3 interface.  When traffic from the local layer 2 network needs to traverse the VXLAN tunnel, it is received and encapsulated, then sent out the layer 3 interface.  Conversely, traffic from the VXLAN network destined for that device’s layer 2 LAN is received on the layer 3 interface, it is de-encapsulated and placed in the appropriate VLAN (if necessary) and sent out the layer 2 interface.

As shown in the VXLAN frame image, these are complex frames with layers of encapsulation and in a legacy NXOS or IOS setup, this requires a complex setup process.  A high-level overview of setting up VXLAN looks like this:

1. Configure layer 3 links between the switches in your VXLAN environment.
2. Configure a link-state routing protocol (OSPF or IS-IS) between your switches.
3. Configure BGP between the switches and set up BGP route reflectors on non-VTEP devices.
4. Configuring multicasting across your VXLAN network
5. Enabling VXLAN on the switches
6. Mapping VLANs to VXLAN VNIDs
7. Creating NVE interfaces for the VNIs
8. Configure BGP EVPN
9. Connect devices to the network and set them into the correct VLANs/VNIDs

That is a lot of configuration across multiple devices and can quickly turn into a major challenge even on a small deployment, especially if you mis-configure something and need to go back and find the error.  That’s where ACI comes in to save the day.  When you deploy an ACI network, you are deploying a VXLAN network in your infrastructure, but ACI does all the work for you and is done in a manner of minutes.

But wait, I said that VXLAN was the easy button for ACI, not the other way around.  In truth, they are the easy buttons for each other.  ACI does the work for you in setting up VXLAN, and by understanding how VXLAN works, the ACI learning curve is greatly reduced and when you look at the configuration elements in the ACI GUI, you see VXLAN elements.  ACI Leaf switches are the VXLAN VTEPs, and the spine switches are the route reflectors.  A VXLAN network can be configured for multi-tenancy,  ACI is built with multi-tenancy and when it first comes online, there are already three tenants in your ACI infrastructure. Both VXLAN and ACI utilize Layer 3 routing instances called VRFs.  In ACI you configure a Layer 2, Bridge Domain, which is often referred to as being “like a VLAN”, but it is really a VXLAN segment with a VNID and multicast group.

While setting up all of these networking elements in VXLAN can take hours (or days on large deployments), they take minutes in ACI with just a few mouse clicks and filling out a few fields in a web browser.  Then once all of that is set up in ACI, you can dive into the Application profiles which is where the real power in using Cisco ACI for your data center.