This document provides the steps to perform an ELAM on the Nexus 7000 F1-Module, explains the most relevant outputs, and how to interpret their results. Please refer to the following document for an overview on ELAM:
Topology
In this example a host on Vlan10 (10.1.1.101 with MAC 0050.56a1.1a01) port Eth3/18 sends an ICMP request to a host also on Vlan10 (10.1.1.102 with MAC 0050.56a1.1aef) off port Eth3/26. We will use ELAM to capture this single frame between the hosts. It's important to remember that ELAM allows us to capture a single frame.
To perform an ELAM on the Nexus7000, you need to first attach to the appropriate module. This requires the network-admin privilege.
N7K# attach module 3
Attaching to module 3 ...
To exit type 'exit', to abort type '$.'
module-3#
Determine the ingress FE
We expect the traffic to ingress the switch on port Eth3/18. Checking the modules in the system we can see that module 3 is an F1 module. Remember, the Nexus 7000 is fully distributed and the modules, not the supervisor, are responsible for making the forwarding decision for dataplane traffic.
N7K# show module 3
Mod Ports Module-Type Model Status
--- ----- ----------------------------------- ------------------ ----------
3 32 1/10 Gbps Ethernet Module N7K-F132XP-15 ok
For F1 modules, we want to perform the ELAM on the L2 forwarding engine (FE) with internal codename Orion. The F1 has 16 FEs per module. We need to determine which Orion ASIC is the FE for port Eth3/18. We can use the following command to verify:
module-3# show hardware internal dev-port-map
(some output omitted)
--------------------------------------------------------------
CARD_TYPE: DCE 32 port 10G
>Front Panel ports:32
--------------------------------------------------------------
Device name Dev role Abbr num_inst:
--------------------------------------------------------------
> Orion Fwding Driver DEV_LAYER_2_LOOKUP L2LKP 16
+--------------------------------------------------------------+
+-----------+++FRONT PANEL PORT TO ASIC INSTANCE MAP+++--------+
+--------------------------------------------------------------+
FP port | PHYS | MAC_0 | L2LKP | QUEUE |SWICHF
...
18 8 8 8 8 1
From the output above, we can see that Eth3/18 is on Orion (L2LKP) instance 8.
module-3# elam asic orion instance 8
module-3(orion-elam)#
Configure the Trigger
The Orion ASIC has a very limited set of ELAM triggers compared to the other FEs on Nexus 7000. This is because the F1 is an L2 only module. It will be switching based off MAC information (or SwitchID in FabricPath environments). For NX-OS you can utilize the question mark to help parse out the ELAM trigger
module-3(orion-elam)# trigger di field ?
da Destination mac-address
mim_da Destination mac-in-mac-address
mim_sa Source mac-in-mac-address
sa Source mac-address
vlan
For this example we want to capture the frame based off source and destination MAC address on the ingress decision block. Note that F1 does not require a separate dbus and rbus trigger.
Trigger
module-3(orion-elam)# trigger di field sa 0050.56a1.1a01 da 0050.56a1.1aef
Start the Capture
Different from the other Nexus7000 modules, the F1 ELAM is started as soon as the triggered is configured. We can check the status of the ELAM via the status command.
module-3(orion-elam)# status
Armed
Once the frame matching the trigger has been received by the FE we will see the ELAM as triggered:
module-3(orion-elam)# status
Triggered
Interpret the Results
We can display the results via the show capture command. Below is an excerpt of the ELAM data that is most relevant in this example.
module-3(orion-elam)# show capture
(some output omitted)
dc3v4_si[11:0] : 17
vlanx : a
di : 1e or 1f
res_eth_da : 5056a11aef
res_eth_sa : 5056a11a01
Note that the F1 ELAM combines the data used to make the forwarding decision along with the forwarding result within the same output. From the example above we can see the source LTL (dc3v4_si) , the destination LTL (di), the vlan (vlanx), and MAC addresses. Note that the MAC address format in the ELAM output does not include prepending zeros.
Destination MAC (res_eth_da) 5056a11aef = 0050.56a1.1aef
Source MAC (res_eth_sa) 5056a11a01 = 0050.56a1.1a01
The source LTL (dc3v4_si) represents the port in which the frame was received.
N7K# show system internal pixm info ltl 0x17
Type LTL
---------------------------------
PHY_PORT Eth3/18
The above output shows that source LTL of 0x17 maps to port Eth3/18. This confirms that the frame was received on Eth3/18.
Note that the F1 ELAM displays two results for the destination LTL (1e or 1f). This occurs because the ELAM parser is not able to read the least significant bit of the ELAM data resulting in a slight ambiguity. The rule of thumb is to validate the hardware MAC entry for the destination address and cross-check this with the destination LTL in the ELAM
module-3# show hardware mac address-table fe 8 address 0050.56a1.1aef vlan 10 vdc 1
FE | Valid| PI| BD | MAC | Index|
| | | | | |
---+------+---+------+---------------+-------+ (some output omitted)
8 1 0 34 0050.56a1.1aef 0x0001f
N7K# show system internal pixm info ltl 0x1f
Type LTL
---------------------------------
PHY_PORT Eth3/26
The above output validates that Orion instance 8 (the FE making the forwarding decision for Eth3/18) has a hardware MAC entry for the destination 0050.56a1.1aef of 0x1f. This index is also the destination LTL (di) within the F1 ELAM. Finally, we validated that the LTL 0x1f maps to port Eth3/26. This confirms that the frame was sent out Eth3/26.
Further Information
Another command to remember is show system internal pixm info ltl-region, which will show how the switch has allocated the pool of LTL's. This is useful to understand the purpose of an LTL if it does not match to a physical port. A good example is a drop LTL:
N7K# show system internal pixm info ltl 0x11a0
0x11a0 is not configured
N7K# show system internal pixm info ltl-region
LTL POOL TYPE SIZE RANGE
=====================================================================
DCE/FC Pool 1024 0x0000 to 0x03ff
SUP Inband LTL 32 0x0400 to 0x041f
MD Flood LTL 1 0x0420
Central R/W 1 0x0421
UCAST Pool 1536 0x0422 to 0x0a21
PC Pool 1720 0x0a22 to 0x10d9
LC CPU Pool 32 0x1152 to 0x1171
EARL Pool 72 0x10da to 0x1121
SPAN Pool 48 0x1122 to 0x1151
UCAST VDC Use Pool 16 0x1172 to 0x1181
UCAST Generic Pool 30 0x1182 to 0x119f
LISP Pool 4 0x1198 to 0x119b
Invalid SI 1 0x119c to 0x119c
ESPAN SI 1 0x119d to 0x119d
Recirc SI 1 0x119e to 0x119e
Drop DI 2 0x119f to 0x11a0
UCAST (L3_SVI_SI) Region 31 0x11a1 to 0x11bf
UCAST (Fex/GPC/SVI-ES) 3648 0x11c0 to 0x1fff
UCAST Reserved for Future Use Region 2048 0x2000 to 0x27ff
======================> UCAST MCAST BOUNDARY <======================
VDC OMF Pool 32 0x2800 to 0x281f
