Building a Cisco ACI Lab

This article is a summary of the best solutions to practice Cisco ACI before using it in a production environment.

This is a recurrent question I always get when I teach my clients about ACI: what is the best way to practice when you only have your production network at your disposal? A lot of people are in this situation, the company choose to refresh the Data Center with Cisco ACI, send their network engineers into training, and expect them to be able to operate the fabric immediately, just like that :) It’s almost impossible, and even dangerous because the gap to be fluent with ACI is too high, the mechanisms are much different from the CLI model to the ACI model to master them in a week.

From a simple VLAN to an Endpoint Group coupled to a Bridge domain, from the forwarding table to the data-plane learning, from a few lines of well known CLI commands, to a dozen objects linked together to configure an access port. you have to practice!

Here are the options I always recommend:

• For the enterprises, the best option is to have an ACI Mini Fabric in addition to the real Fabric, your engineers will thank you for that!
• For the network engineer who doesn’t have this chance, the best remaining options are from my point of vue:
  • Cisco DevNet Sandboxes (easy access to shared APIC for unlimited time or reservable APIC in limited time, on limited versions of ACI)
  • Cisco ACI Simulator (easy access, full-time availability on your company servers and some privacy to test your target configurations)

Cisco Mini Fabric

The mini fabric is composed of a physical APIC (M3), two Leafs (93180YC-EX), two Spines (C9332C). 5RU for the network, and prepare one or two more rack units for your preferred server, with VMware ESX installed (for example) and vCenter to be able to add two Virtual APIC, build an APIC cluster of 3 members, and be ready to test a VMM domain integration. It’s not yet enough to test multi-pod and multi-site, but it will allow you to configure a real fabric, test the behavior of physical and virtual endpoints.

If you want to test multi-pod/multi-site, take a look at the lab of Michael here: https://www.mvankleij.nl/post/aci-lab/ 

The product reference is ACI-C9332-VAPIC-B1 for the 2 spines and the APIC. You will have to add the Leafs in addition, two to be able to test the VPC feature, the N9K-C93180YC-FX-24 is a good choice. Public prices are high and this is why many clients are not going this way, but I’m sure your partners will get you a good price!

If you don’t want to invest that much, the minimum number of devices you need to have a working fabric (not ready for production, but good enough for a small lab) is a physical APIC (M3), a Spine (C9332C) a Leaf (93180YC-EX) and a Server with your favorite hypervisor.

Cisco DevNet

Cisco DevNet is the easiest solution to practice/discover quickly and easily Cisco ACI remotely, without owning anything but without the possibility to test real case scenarios of failovers or even VMM integrations. Devnet is also a goldmine of information about ACI, take a look at all the doc, scripts, automation tools listed here: https://developer.cisco.com/docs/aci/.

There are two types of sandboxes:

• The always-on ACI Simulator where you have access to an APIC shared with many other people.
• The reservable ACI Simulator where you can have your fresh APIC for up to 5 days, for yourself. (a VPN access is necessary).

Cisco ACI Simulator

The Cisco ACI Simulator is the solution I recommend the most because it allows you to touch ACI on your laptop if you have enough ressources, or an ESX from your enterprise, once installed it’s easy to test your enterprise’s target configurations of Tenant/Fabric policies as it’s not shared publicly. The main defaults of ACI Simulator are that its 40Gbps, so it might be hard to get, and you can’t test the data-plane, like for DevNet (giving you access to ACI Simulator, so same drawbacks), forget about the real test case scenarios of failovers or VMM integrations.

It is something really important to be able to understand the simple things like the effects of enforcing/unenforcing a VRF or an EPG, and verifying immediately if an endpoint can still ping… or not.

Please find some useful documents I recommend to help you install your virtual machine of Cisco ACI Simulator:

• Download link: https://software.cisco.com/download/home/286283149/type/286283168/release/  
• Install Guide 4.x : https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/4-x/ACI-Simulator-Getting-Started/cisco-aci-simulator-getting-started-guide-4x/b_ACI_Simulator_Guide_Rel_3_x_chapter_010.html 
• Install Guide 5.x : https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/5-x/getting-started/cisco-apic-getting-started-guide-50x/m_apic_preface.html 
• HowTo Blog: http://vadmin-land.azurewebsites.net/simulate-cisco-aci-with-aci-simulator-vm/

In brief, you will need to:

• Download the 4 parts of the .ova
• Reassemble the 4 parts into a usable .ova (cat command (Linux) or the type command (Windows))
• Deploy the .ova on your ESX server
• Start the VM and initialize your APIC from the VM Console (don’t be like me and follow the getting started guides correctly, don’t change the default values from the APIC initialization process or you won’t be able to find your leaf to simulate the provisioning of leaf/spine. The TEP addresses, infrastructure Vlan & co must be left as default.)
• The VM will reboot after this initialization and you will be able to connect to the GUI / REST API / SSH right after and proceed with the virtual leaf/spine registration.

Cisco ACI Simulator download is not free, you need to have the right account to download the images, but I’m sure you know someone at your favorite partner who can help you to get the right version you need. Here is the dashboard after the APIC initialization, just like the real one. Enjoy, and practice!

What about you? How did you practice with ACI?

Leave a comment to explain.

Benoit

Network engineer at CNS Communications. CCIE #47705, focused on R&S, Data Center, SD-WAN & Automation.

More Posts - Website

Follow Me: