Sunday, September 16, 2012

FabricPath Authentication in NX-OS

A few weeks ago I was asked by a colleague about some issues they were having with FabricPath authentication. I really hadn’t delved into the details of it yet so took the opportunity to do so. I thought my experience would be good to blog about and share with others. I can see this being a topic some of our more security conscious customers implement and would make a good topic for the CCIE Data Center.

First and foremost, I’m going to assume you have a basic working knowledge of FabricPath, Cisco’s scalable Layer 2 solution that eliminates Spanning Tree Protocol and adds some enhancements that are sorely needed in L2 networks like Time To Live (TTL), Reverse Path Forwarding (RPF) and uses IS-IS as a control plane protocol.  It’s the fact that FabricPath uses IS-IS that makes it very easy and familiar for customers to enable authentication in their fabric. If you have ever configured authentication for a routing protocol in Cisco IOS or NX-OS, this will be similar with all of your favorites like key chains, key strings and hashing algorithms. Hopefully that nugget of information doesn’t send you into a tail spin of despair.  ;)

With FabricPath there are two levels of authentication that can be enabled. The first is at the domain level for the entire switch (or VDC!). Authentication here will prevent routes from being learned. Important to note that ISIS adjacencies can be formed on the interface level even when the domain authentication is mismatched. This domain level authentication is for LSP and NSP exchange not PDUs on the interfaces.  If you are not careful, you can blackhole traffic during the implementation of authentication, just like you would with any other routing protocol.
A quick order of operation to enable domain level authentication would be to define a key-chain with keys which contain key-strings defined underneath. The key strings are the actual password and NX-OS allows you to define multiple key-strings so you can rotate passwords as needed and even includes nerd knobs for setting start and end times. After the key chains are defined, they are applied to the FabricpPath domain. Let’s quit typing and let the CLI do the talking.

We start with a VDC that has FabricPath, is in a fabric with other devices but doesn’t have authentication enabled. We can see we have not learned any routes.
N7K-2-Access2# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id


FabricPath Unicast Route Table for Topology-Default
0/4/0, number of next-hops: 0
        via ---- , [60/0], 24 day/s 00:32:41, local
0/69/1, number of next-hops: 0
1/69/0, number of next-hops: 0
        via ---- , [60/0], 15 day/s 04:18:01, local
2/69/0, number of next-hops: 0
        via ---- , [60/0], 15 day/s 04:18:01, local

We can also see we are adjacent to some other devices, but also note that we *don’t* see their name under system ID, just the MAC address. This is a quick point that something is amiss with the control plane. They are in bold and red below.
N7K-2-Access2# show fabricpath isis adj
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
0026.980f.d9c4  N/A             1      UP     00:00:25   port-channel1
0024.98eb.ff42  N/A             1      UP     00:00:29   Ethernet3/9
0024.98eb.ff42  N/A             1      UP     00:00:27   Ethernet3/10
0026.980f.d9c2  N/A             1      UP     00:00:22   Ethernet3/20
0026.980f.d9c2  N/A             1      UP     00:00:29   Ethernet3/21

Now we’ll add the authentication and start with the key-chain and call it “domain” then define key 0 and the key-string of “domain” (not very creative am I?) and then finally apply it to the fabricpath domain default.
 
N7K-2-Access2# config
Enter configuration commands, one per line.  End with CNTL/Z.
N7K-2-Access2(config)# key chain domain
N7K-2-Access2(config-keychain)# key 0
N7K-2-Access2(config-keychain-key)# key-string domain
N7K-2-Access2(config-keychain-key)# fabricpath domain default
N7K-2-Access2(config-fabricpath-isis)# authentication key domain

Now let’s see what that does for us.  Much happier now aren’t we?
N7K-2-Access2(config-fabricpath-isis)# show  fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id

FabricPath Unicast Route Table for Topology-Default
0/4/0, number of next-hops: 0
        via ---- , [60/0], 24 day/s 00:33:32, local
0/69/1, number of next-hops: 0
1/1/0, number of next-hops: 2
        via Eth3/20, [115/40], 0 day/s 00:00:10, isis_fabricpath-default
        via Eth3/21, [115/40], 0 day/s 00:00:10, isis_fabricpath-default
1/2/0, number of next-hops: 2
        via Eth3/9, [115/40], 0 day/s 00:00:11, isis_fabricpath-default
        via Eth3/10, [115/40], 0 day/s 00:00:11, isis_fabricpath-default
1/69/0, number of next-hops: 0
        via ---- , [60/0], 15 day/s 04:18:52, local
1/100/0, number of next-hops: 4
        via Eth3/9, [115/40], 0 day/s 00:00:11, isis_fabricpath-default
        via Eth3/10, [115/40], 0 day/s 00:00:11, isis_fabricpath-default
        via Eth3/20, [115/40], 0 day/s 00:00:10, isis_fabricpath-default
        via Eth3/21, [115/40], 0 day/s 00:00:10, isis_fabricpath-default
2/69/0, number of next-hops: 0
        via ---- , [60/0], 15 day/s 04:18:52, local
N7K-2-Access2(config-fabricpath-isis)#

The exact same sequence applies to interface-level authentication and looks like the CLI below. We can see that when we have two non-functioning states here – INIT and LOST. INIT is from me removing the key-chain and flapping the interface (shut/no shut) and LOST is from me removing the pre-defined key chain and the adjacency going down to N7K-1-Agg1.
N7K-2-Access2# show fab isis adj
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K-1-Access1   N/A             1      UP     00:00:27   port-channel1
N7K-2-Agg2      N/A             1      INIT   00:00:22   Ethernet3/9
N7K-2-Agg2      N/A             1      UP     00:00:23   Ethernet3/10
N7K-1-Agg1      N/A             1      LOST   00:04:57   Ethernet3/20
N7K-1-Agg1      N/A             1      UP     00:00:30   Ethernet3/21

Now we’ll add our key chain and key string.
N7K-2-Access2# config
Enter configuration commands, one per line.  End with CNTL/Z.
N7K-2-Access2(config)#
N7K-2-Access2(config-keychain-key)# int e3/9
N7K-2-Access2(config-if)# fabricpath isis authentication-type cleartext
N7K-2-Access2(config-if)# fabricpath isis authentication key-chain interface
N7K-2-Access2(config-if)#
N7K-2-Access2(config-if)# key chain interface
N7K-2-Access2(config-keychain)#key 0
N7K-2-Access2(config-keychain-key)# key-string 7 interface
N7K-2-Access2(config-keychain-key)#
N7K-2-Access2(config-keychain-key)# int e3/9
N7K-2-Access2(config-if)# fabricpath isis authentication-type cleartext
N7K-2-Access2(config-if)# fabricpath isis authentication key-chain interface
N7K-2-Access2(config-if)#

A quick check shows us we’re happily adjacent to our swiches.

N7K-2-Access2(config-keychain)# show fab isis adj
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K-1-Access1   N/A             1      UP     00:00:30   port-channel1
N7K-2-Agg2      N/A             1      UP     00:00:29   Ethernet3/9
N7K-2-Agg2      N/A             1      UP     00:00:26   Ethernet3/10
N7K-1-Agg1      N/A             1      UP     00:00:24   Ethernet3/20
N7K-1-Agg1      N/A             1      UP     00:00:31   Ethernet3/21

Finally, a quick command to check the FabricPath authentication status on your device is below:

N7K-2-Access2# show fab isi

Fabricpath IS-IS domain : default
  System ID : 0024.98eb.ff43  IS-Type : L1
  SAP : 432  Queue Handle : 11
  Maximum LSP MTU: 1492
  Graceful Restart enabled. State: Inactive
  Last graceful restart status : none
  Metric-style : advertise(wide), accept(wide)
  Start-Mode: Complete [Start-type configuration]
  Area address(es) :
    00
  Process is up and running
  CIB ID: 3
  Interfaces supported by Fabricpath IS-IS :
    port-channel1
    Ethernet3/9
    Ethernet3/10
    Ethernet3/20
    Ethernet3/21
  Level 1
  Authentication type: MD5
  Authentication keychain: domain  Authentication check specified
  MT-0 Ref-Bw: 400000
  Address family Swid unicast :
    Number of interface : 5
    Distance : 115
  L1 Next SPF: Inactive

N7K-2-Access2#


With this simple exercise you’ve configured FabricPath authentication. Not too bad and very effective. As always when configuring passwords on your device, cut and paste from a common text file is important to avoid empty white spaces at the end of passwords and other nuances that can lead you down the wrong path. In general, I would expect a customer who implements FabricPath authentication will probably configure both domain and interface level authentication.

 

As always, your comments and feedback are appreciated!

 

1 comment:

  1. the domain auth is very confusing. When you enable this on one side only the isis adj look like working but the actual forwarding frame is not working.
    Thanks for this post, was very helpful for me.
    Roie

    ReplyDelete