cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

TrustSec N7k Specific Troubleshooting Information

682
Views
0
Helpful
0
Comments

 

<< BACK TO THE MAIN TRUSTSEC TROUBLESHOOTING GUIDE

 

TrustSec N7k Specific Troubleshooting

InformationCheck to see if SGACL is loaded into N7k HW:

Prompt-N7k# show system internal access-list output entries detail

~snip~

  Tcam 0 resource usage:

  ----------------------

  Label_a = 0x801

   Bank 0

   ------

     IPv4 Class

       Policies: Rbacl()

       Netflow profile: 0

       Netflow deny profile: 0

       Entries:

         [Index] Entry [Stats]

         ---------------------

  [0008:0e64:0006] prec 3 deny ip 0.0.17.17/32 0.0.17.17/32  log  [0]

  [0009:0864:0007] prec 3 deny ip 0.0.17.17/32 0.0.17.17/32 fragment  log  [0]

  [000a:0e04:0008] prec 3 permit ip 0.0.0.0/32 0.0.0.0/32   [648]

  [000b:0804:0009] prec 3 permit ip 0.0.0.0/32 0.0.0.0/32 fragment   [1]

  [000c:08e4:000a] prec 3 deny ip 0.0.17.17/32 0.0.3.233/32  log  [0]

  [000d:0ee4:000b] prec 3 deny ip 0.0.17.17/32 0.0.3.233/32 fragment  log  [0]

  [000e:0824:000c] prec 3 deny ip 0.0.0.17/32 0.0.0.19/32  log  [0]

  [000f:0e24:000d] prec 3 deny ip 0.0.0.17/32 0.0.0.19/32 fragment  log  [0]

  [0014:0a04:0012] prec 3 permit ip 0.0.0.0/0 0.0.0.0/0  log  [3331]

  [0015:0004:0013] prec 3 permit ip 0.0.0.0/0 0.0.0.0/0 fragment  log  [0]

  [0016:0904:0014] prec 3 permit ip 0.0.0.0/0 0.0.0.0/0   [0]

  [0017:0b04:0015] prec 3 permit ip 0.0.0.0/0 0.0.0.0/0 fragment   [0]

~snip~

The IP address type entries above are not IP addresses, they are SGT's.

For example, 0.0.0.17  = SGT 17

Another example, 0.0.17.17 = 256 * 17 + 17 = SGT 4369

N7k cache information to check IP to SGT map allocation

First, caching needs to be enabled on the N7k:

Caching needs to be enabled:

Be careful in a live deployment. Enabling caching will dramatically reduce platform performance.

cts role-based sgt-caching with-enforcement

Prompt-N7k# show logging ip access-list cache detail

SGT     Src IP        Dst IP     S-Port    D-Port    Src Intf         Protocol           Hits

------------------------------------------------------------------------------------------------

1001    10.1.100.2       10.2.50.10         53      57566     Ethernet1/1 (17)UDP                0

1001    10.1.100.2       10.2.50.10         49155   52224     Ethernet1/1 (6)TCP                0

1001    10.1.100.2       10.2.50.10         53      49221     Ethernet1/1 (17)UDP                0

4369    10.4.1.10        10.1.100.2         59887   53        Ethernet2/1 (17)UDP                0

4369    10.4.1.10        10.1.100.2         57604   53        Ethernet2/1 (17)UDP                2

0       10.2.50.10       10.1.100.2         137     137       Ethernet2/1 (17)UDP                12

How to check if the CTS app has sent the IP-SGT mapping down to the FIB?

Lookup the IP in the event-history and check the SGT mapping:

Prompt-N7k# Show cts internal event-history sgtmap | grep 10.4.1.10    <----- Checking for SGT

12:14:26.931259 E_CTS_IP_SGT_UPDATE  SAL opc: 1, IP: 10.4.1.10, VLAN: 0, VRF: 1, if_index: , SGT: 4369

Prompt-N7k# Show cts internal event-history sgtmap | grep 10.1.100.2    <----- Checking for DGT

12:13:41.177868 E_CTS_IP_SGT_UPDATE  SAL opc: 1, IP: 10.1.100.2, VLAN: 0, VRF: 1, if_index: , SGT: 1001

How to check if the SGT and DGT are correctly programmed in the FIB?

Prompt-N7k# Show forwarding security group-tag vrf all | grep 10.4.1.10    <----- Checking for SGT

  1. 10.4.1.10/32        4369 1
  2. 10.4.1.10/32        4369 1
Prompt-N7k# Show forwarding security group-tag vrf all | grep 10.1.100.2    <----- Checking for DGT
  1. 10.1.100.2/32       1001 1
  2. 10.1.100.2/32       1001 1

How to check the DGT will be derived correctly from the FIB lookup?

Prompt-N7k# show sys internal forwarding ipv4 route 10.1.100.2 detail module 1 | grep DGT   <----- Show DGT for destination IP for mod 1

Dev: 1 , Idx: 0x16c8  , RPF Flags: VG    , DGT: 1001, VPN: 1

Prompt-N7k# show sys internal forwarding ipv4 route 10.1.100.2 detail module 2 | grep DGT   <----- Show DGT for destination IP for mod 2

Dev: 1 , Idx: 0x1200d , RPF Flags: VG , DGT: 1001, VPN: 1

How to use the N7k Ethanalyzer function to monitor traffic?

Prompt-N7k# ethanalyzer local interface inband display-filter ip.addr==10.4.1.10 limit-captured-frames 100

Capturing on inband

2016-04-04 12:58:35.298187    10.4.1.10 -> 10.1.100.4   ICMP 74 Echo (ping) request  id=0x0001, seq=128/32768, ttl=126

2016-04-04 12:58:35.298950   10.1.100.4 -> 10.4.1.10    ICMP 74 Echo (ping) reply    id=0x0001, seq=128/32768, ttl=126

(request in 69)

How to check historic TrustSec events on the N7k?

There are a number of options to use after the 'show cts internal event-history' command. Some are shown below:

The following checks events related to downloading the environment data:

Prompt-N7k# show cts internal event-history env-fsm

>>>>FSM: <CTS Global FSM> has 11 logged transitions<<<<<

1) FSM:<CTS Global FSM> Transition at 767672 usecs after Mon Apr 25 10:52:20 2016

    Previous state: [CTS_ENV_DNLD_ST_INIT_STATE]

    Triggered event: [CTS_ENV_E_DOWNLOAD_ENV_FROM_AAA]

    Next state: [CTS_ENV_DNLD_ST_WAIT_FOR_ENVIRONMENT_DATA]

~snip

11) FSM:<CTS Global FSM> Transition at 818502 usecs after Mon Apr 25 11:00:01 2016

    Previous state: [CTS_ENV_DNLD_ST_WAIT_FOR_HW_PROGRAMMING]

    Triggered event: [CTS_ENV_E_HARDWARE_UPDATE_SUCCESSFUL]

    Next state: [CTS_ENV_DNLD_ST_ENV_DOWNLOAD_DONE]

    Curr state: [CTS_ENV_DNLD_ST_ENV_DOWNLOAD_DONE]

Prompt-N7k# show cts internal event-history errors

1) Event:E_DEBUG, length:55, at 788416 usecs after Mon Apr 25 10:56:01 2016

    [105] recv_aaa_env:Error in aaa response: status = 2048

2) Event:E_DEBUG, length:34, at 796056 usecs after Mon Apr 25 10:55:50 2016

    [105] getpeername failed errno:107

Prompt-N7k# show cts internal event-history host-tracking

11:05:44.278881 E_CTS_IF_LEARN       IP: 10.10.2.10, if_index: Ethernet1/2

11:05:44.278878 E_CTS_MAC_LEARN      Source: MAC SDB lookup, VLAN: 20, MAC: 0050.5688.913e, if

_index: Ethernet1/2 (0x1a001000), flags: 0x107

11:05:44.278864 E_CTS_IP_LEARN       Source: AM notification, IP: 10.10.2.10, MAC: 0050.5688.9

13e, VLAN: 20, VRF: 0

Prompt-N7k# show cts internal event-history ifc all

>>>>FSM: <Eth1/2 IFC FSM> has 7 logged transitions<<<<<

1) FSM:<Eth1/2 IFC FSM> Transition at 948165 usecs after Mon Apr 25 10:55:45 2016

    Previous state: [CTS_IFC_ST_INIT_STATE]

    Triggered event: [CTS_IFC_E_PRECONFIG_START]

    Next state: [CTS_IFC_ST_]

~snip

7) FSM:<Eth1/2 IFC FSM> Transition at 75352 usecs after Mon Apr 25 10:55:48 2016

    Previous state: [CTS_IFC_ST_SAP_NEGOTIATING_STATE]

    Triggered event: [CTS_IFC_E_RECV_SAP_BYPASS]

    Next state: [CTS_IFC_ST_CTS_OPEN_STATE]

    Curr state: [CTS_IFC_ST_CTS_OPEN_STATE]

Prompt-N7k# show cts internal event-history rbacl

10:52:20.772472 E_CTS_SESS_OPEN      Session id: 0x00000000571df6eb

10:52:20.772169 E_CTS_TASK_EXEC      Apply SGT: 17, DGT: 19 RBACL: Deny_ip

10:52:20.772162 E_CTS_SUB_TASK_NQ    Apply SGT: 65535, DGT: 65535 RBACL: Permit IP

10:52:20.772150 E_CTS_SUB_TASK_NQ    Apply SGT: 4369, DGT: 4369 RBACL: Deny_ip

10:52:20.772132 E_CTS_TASK_NQ        Apply SGT: 17, DGT: 19 RBACL: Deny_ip

Prompt-N7k# show cts internal event-history sgtmap

12:11:45.582119 E_CTS_IP_SGT_UPDATE  SAL opc: 2, IP: 10.8.1.2, VLAN: 0, VRF: 1, if_index: , SGT: 0

12:05:48.619825 E_CTS_IP_SGT_UPDATE  SAL opc: 1, IP: 10.8.1.2, VLAN: 0, VRF: 1, if_index: , SGT: 2

11:22:54.796754 E_CTS_IP_SGT_UPDATE  SAL opc: 1, IP: 1.1.1.2, VLAN: 0, VRF: 1, if_index: , SGT: 15

11:22:54.678628 E_CTS_IP_SGT_UPDATE  SAL opc: 1, IP: 2.2.2.2, VLAN: 0, VRF: 1, if_index: , SGT: 5

Prompt-N7k# show cts internal event-history sxp peer all

>>>>FSM: <Peer 10.1.101.5 Peer FSM> has 4 logged transitions<<<<<

1) FSM:<Peer 10.1.101.5 Peer FSM> Transition at 372059 usecs after Mon Apr 25 10:56:53 2016

    Previous state: [CTS_SXP_PEER_ST__INIT_STATE]

    Triggered event: [CTS_SXP_PEER_E_SEND_OPEN]

    Next state: [CTS_SXP_PEER_ST__PENDING_ON]

~snip

4) FSM:<Peer 10.1.101.5 Peer FSM> Transition at 375313 usecs after Mon Apr 25 10:56:53 2016

    Previous state: [CTS_SXP_PEER_ST__ON]

    Triggered event: [CTS_SXP_PEER_E_UPDATE]

    Next state: [FSM_ST_NO_CHANGE]

    Curr state: [CTS_SXP_PEER_ST__ON]

How to Show the TrustSec (and MacSec) Capability of the N7k Linecards?

Prompt-N7k# show cts capability interface all

CTS capability information for interface(s)

--------- --- ------ ---------------------------------------

Interface SGT MacSec Comments

--------- --- ------ ---------------------------------------

Eth1/1    Yes Yes    cts dot1x and manual configs allowed

Eth1/2    Yes Yes    cts dot1x and manual configs allowed

Eth1/3    Yes Yes    cts dot1x and manual configs allowed

Eth1/4    Yes Yes    cts dot1x and manual configs allowed

How to Check access-list Resource Utilization?

Prompt-N7k# show hardware access-list resource utilization module 1

INSTANCE 0x0

-------------

         ACL Hardware Resource Utilization (Mod 1)

         --------------------------------------------

                          Used    Free    Percent

                                          Utilization

-----------------------------------------------------

Tcam 0, Bank 0           12      16372   0.07

Tcam 0, Bank 1           3       16381   0.02

Tcam 1, Bank 0           14      16370   0.09

Tcam 1, Bank 1           304     16080   1.86

LOU                       4       100     3.84  

Both LOU Operands         2      

Single LOU Operands       2      

LOU L4 src port:          1

LOU L4 dst port:          1

LOU L3 packet len:        0

LOU IP tos:               0

LOU IP dscp:              0

LOU ip precedence:        0

LOU ip TTL:               0

TCP Flags                 0       16      0.00  

Protocol CAM              3       4       42.85 

Mac Etype/Proto CAM       9       5       64.28 

Non L4op labels, Tcam 0   1       6142    0.01  

Non L4op labels, Tcam 1   2       6141    0.03  

L4 op labels, Tcam 0      0       2047    0.00  

L4 op labels, Tcam 1      1       2046    0.04  

Ingress Dest info table   2       510     0.39  

Egress Dest info table    0       512     0.00  

<< BACK TO THE MAIN TRUSTSEC TROUBLESHOOTING GUIDE