Supervisor 2/2E and Admin VDC

In June of 2012 Cisco announced a new addition to the Nexus 7000 family, the Supervisor 2 and Supervisor 2E. These new members of the family bring some really exciting capabilities to the platform. I like Virtual Device Contexts (VDCs) a bit….ok, quite a bit….ok, they are my favorite thing to talk about on this platform and I earned the VDCBadger moniker in 2011 at Cisco Live. If I could sum it up, I’d be like Turtle Boy – “I like VDCs”.  For reference - http://www.youtube.com/watch?v=CMNry4PE93Y

Enough rambling, the new supervisors bring new capabilities that we’ll discuss in more detail. First and foremost, let’s look at what we have. There are two models of the new supervisor: Supervisor 2 and Supervisor 2E. There are some key differences between the modules starting with the CPUs. Both SUP2 and SUP2E use Intel Xeon Quad-Core CPUs which alone brings a lot more control plane power, but SUP2E has two CPUs. Additionally the SUP2 ships with 12GB of RAM and SUP2E has 32GB of RAM. You can see in the images below the differences in hardware.

Supervisor 2

Supervisor 2E

            

 

 

 

 

 

 

These combine to allow some significant increases in scalability across the chassis, primarily with the number of VDCs. Supervisor 2 supports 4 VDCs while Supervisor 2E supports 8! Additionally a new capability called “Admin VDC” comes in NX-OS 6.1(1) on the Supervisor 2/2E so you’ll frequently see the VDC count listed as 4+1 and 8+1 with the +1 being the Admin VDC. More details on Admin VDC in a bit. Know that Admin VDC is a management context that is a direct result of customer feedback. The additional CPU also brings higher scale for Nexus 2000s (FEX), IEEE 1588 PTP clients with more scale increments across the chassis to come.

The new Supervisors also take advantage of a new 64 bit kernel for NX-OS, USB flash and operations that require CPU, like saving the configuration, ISSU, etc are all faster. SUP2 also brings FCoE on the F2 series modules to customers as well. Finally, one last nerd knob is CPU Shares – pretty much QoS for the CPU in a multi-VDC environment. One thing you’ll notice is missing is the Connectivity Management Processor (CMP). This was done intentionally and not without a lot of thought and feedback. Long story short, most customers were not using it. Everyone agreed CMP was a cool idea, but it was rarely plugged in. Removing it means the SUP2 and SUP2E use less power, which is a key concern for a lot of customers. What does SUP2 look like on the CLI? Funny you ask, I happen to have some CLI.

Hardware

  cisco Nexus7000 C7009 (9 Slot) Chassis ("Supervisor module-2")

  Intel(R) Xeon(R) CPU         with 32745276 kB of memory.
  Processor Board ID JAF1608ACEK

 

N7K-1# show mod

Mod  Ports  Module-Type                         Model           Status
---  -----  ----------------------------------- --------------- ------
2    0      Supervisor module-2                 N7K-SUP2E       active *
3    32     1/10 Gbps Ethernet Module           N7K-F132XP-15   ok
4    8      10 Gbps Ethernet XL Module          N7K-M108X2-12L  ok
6    48     1/10 Gbps Ethernet Module           N7K-F248XP-25   ok
7    48     10/100/1000 Mbps Ethernet XL Module N7K-M148GT-11L  ok

Looks a lot like a you’d expect.  J

I mentioned Admin VDC, so let’s dig in to what Admin VDC does. First, let’s talk about VDCs in general – there are two kinds of VDCs – the default VDC and non-default VDCs. The default VDC is the VDC the switch operates in if you are not using VDCs. Read that last sentence again.  It really means that even if you are not using VDCs, you are using VDCs. 

Customers asked for an administrative context to perform system-wide operations and the Admin VDC came to be. See, we *do* read your surveys and hear your feedback!

Admin VDC is a new type of VDC, specialized in that it is 100% administrative only. Admin VDC is designed to allow “run the box” type functions to be performed in a context separate from data plane traffic. The following configuration or tasks can be performed in Admin VDC:

1 – VDC operations – creation, deletion, suspension, all resource allocation including CPU shares

2 – Install operations – ISSU/ISSD of the NX-OS, EPLD upgrades, feature set installation (FCoE/FabricPath/FEX/MPLS) and licensing

3 – Reload – individual VDC or entire chassis

4 – Control Plane Policing – class map, policy map definition and application

5 – Ethanalyzer of control plane traffic

6 -  GOLD diagnostic - start/stop tests, configure tests

7 – Miscellaneous module operations - Out of service, purge config for removed modules

8 – Admin VDC specific debugging – bootvar, copp, diagnostics(GOLD), ethdstats, exceptionlog, giscm, license, oim, plog, and psshelper_gsvc.  Of the list, ethdstats, gicsm, oim, plog and pssheler_gsvc are system level processes and the rest are either admin VDC only tasks like boot configuration, CoPP, GOLD and licensing.

Finally, Admin VDC cannot have Ethernet interfaces inside it other than the Management interface (mgmt0). This also means no routing protocols, L2 protocols or other L2/L3 features are available or configurable in Admin VDC.

In a switch with a default VDC configuration, all of these functions would be done from the default VDC, but by moving these capabilities to the Admin VDC the data place VDCs are “cleaner.” An additional benefit is that it lends itself to multi-tenancy where a network operator could give control over an entire VDC without the tenant seeing system-wide configuration parameters.

So how does one get Admin VDC? First, you need SUP2 or SUP2E – it’s coming for SUP1 – so remain calm and carry on. With your shiny new SUP2/SUP2E, you’ll see Admin VDC is a prompt during the boot cycle.

 

   Enter the password for "admin":

  Confirm the password for "admin":

  Do you want to enable admin vdc (yes/no) [n]:

 You can also convert the default VDC to Admin VDC if you answered no to the prompt. Let’s take a N7K with 4 VDCs and convert the default VDC.  That looks like this:

N7K-1# show vdc

vdc_id  vdc_name state    mac               type        lc
------  -------- -----    ----------        ---------   ------
1       N7K-1    active   00:26:98:0f:d9:c1 Ethernet    m1 f1 m1xl m2xl
2       Agg1     active   00:26:98:0f:d9:c2 Ethernet    m1 f1 m1xl m2xl
3       OTV1     active   00:26:98:0f:d9:c3 Ethernet    m1 f1 m1xl m2xl
4       Access1  active   00:26:98:0f:d9:c4 Ethernet    f2  
Now, let’s convert it.

N7K-1# config

Enter configuration commands, one per line.  End with CNTL/Z.

N7K-1(config)# system admin-vdc

N7K-1(config)#

N7K-1(config)# show vdc

vdc_id  vdc_name  state    mac                 type        lc

------  --------  -----    ----------          ---------   ------
1       N7K-1      active  00:26:98:0f:d9:c1   Admin       None
2       Agg1       active  00:26:98:0f:d9:c2   Ethernet    m1 f1 m1xl m2xl
3       OTV1       active  00:26:98:0f:d9:c3   Ethernet    m1 f1 m1xl m2xl
4       Access1    active  00:26:98:0f:d9:c4   Ethernet    f2

Note that when you do this, some key changes are made to what used to be the default VDC. See how the linecard support (LC) is changed to none. Refer to my earlier comment about routing protocols, L2 technologies, etc. not working in Admin VDC.

In case you are wondering what happens if you try to create too many VDCs see below where we have a SUP2E with 9 VDCs (8+1) and we try to create a new VDC called “TooMany.”

N7K-1(config)# show vdc

vdc_id  vdc_name     state       mac                 type        lc

 ------  --------     -----      ----------          ---------   ------
1       N7K-1        active     00:26:98:0f:d9:c1   Admin       None
2       Agg1         active     00:26:98:0f:d9:c2   Ethernet    f2
3       Core1        active     00:26:98:0f:d9:c3   Ethernet    m1 f1 m1xl m2xl
4       Access1      active     00:26:98:0f:d9:c4   Ethernet    m1 f1 m1xl m2xl
5       FCoE         active     00:26:98:0f:d9:c5   Storage     f1 f2
6       DMZ          active     00:26:98:0f:d9:c6   Ethernet    m1 f1 m1xl m2xl
7       Lab          active     00:26:98:0f:d9:c7   Ethernet    m1xl m2xl
8       FP-Test      active     00:26:98:0f:d9:c8   Ethernet    f2
9       Nuke         active     00:26:98:0f:d9:c9   Ethernet    f1

N7K-1(config)# vdc TooMany

ERROR: You have reached the maximum number of allowed vdcs [8]
N7K-1(config)#

Pretty darn cool, IMHO. As always, your comments and feedback are appreciated!