Cisco Embedded Logic Analyzer Module (ELAM)

[vsurresh]

I recently came across a situation where I needed to check if a frame is received and forwarded correctly on a Nexus 9K. There are multiple ways we can achieve this but I found ELAM (Embedded Logic Analyzer Module) capture is very powerful and granular. ELAM can look inside Cisco ASIC and understand how a packet is forwarded. It can capture a packet in real-time without disruptions to performance or control-plane resources.

There are four main steps involved:

1. Identify the Ingress forwarding engine
2. Configure ELAM trigger
3. Start the ELAM capture
4. View the capture report

Let's say you are pinging from 10.1.1.1 to 10.100.100.100 but the ping fails and you wanted to see if the packet is arriving on Eth1/10 interface on the Nexus and leaving on Eth1/11. If everything configured correctly the packet arrives on Eth1/10 will have: 

src.mac =bb.bb src.ip=10.1.1.1

dst.mac=cc.cc dst.ip=10.100.100.100

ELAM capture procedure

Step 1 - Identify port's ASIC, Slice and Srcld

93180YC-EX# show hardware internal tah int eth1/10

#########################################

IfIndex: 436234560
DstIndex: 6079
IfType: 26
Asic: 0
Asic: 0
AsicPort: 20
SrcId: 40
Slice: 0
PortOnSlice: 20

Step 2 - Attach to the module

93180YC-EX# attach module 1

Step 3 - Enter ELAM configuration mode and assign the ASIC

93180YC-EX# debug platform internal tah elam asic 0

Step 4 - Configure the trigger

module-1(TAH-elam)# trigger init asic 0 slice 0 use-src-id 40
param values: start asic 0, start slice 0, lu-a2d 1, in-select 6, out-select 0, src_id 40

Step 5 - Set the ELAM triggers using SRC & DEST IP and start the ELAM

module-1(TAH-elam-insel6)# reset 
module-1(TAH-elam-insel6)# set outer ipv4 dst_ip 10.100.100.100 src_ip 10.1.1.1
module-1(TAH-elam-insel6)# start 

Step 6 - Verify if your ELAM triggered and review the results

module-1(TAH-elam-insel6)# report 

SUGARBOWL ELAM REPORT SUMMARY
slot - 1, asic - 0, slice - 0
============================
Incoming Interface: Eth1/10
Src Idx : 0x23, Src BD : 4158

Outgoing Interface Info: dmod 1, dpid 24

Dst Idx : 0x601, Dst BD : 116
Packet Type: IPv4
Dst MAC address: CC:CC
Src MAC address: BB:BB
.1q Tag0 VLAN: 10, cos = 0x7

Dst IPv4 address: 10.100.100.100
Src IPv4 address: 10.1.1.1
Ver = 4, DSCP = 0, Don't Fragment = 1
Proto = 1, TTL = 255, More Fragments = 0
Hdr len = 20, Pkt len = 8028, Checksum = 0x495d

L4 Protocol : 1
ICMP type : 8
ICMP code : 0

Drop Info:
----------
LUA:
LUB:
LUC:
LUD:

Final Drops:

Outgoing Interface Info shows the egress interface. Here in our case, the correct egress interface is Eth1/11. This can be checked with,

93180YC-EX# show system internal ethpm info interface ethernet 1/11 | grep index

Ethernet1/11 - if_index: 0x1A006A00 IF_STATIC_INFO: port_name=Ethernet1/11,if_index:0x1a006a00,ltl=6075,slot=0, nxos_port=212,dmod=1,dpid=24,unit=0,queue=65535,xbar_unitbmp=0x0,ns_pid=255,slice_num= 0,port_on_slice=24,src_id=48

Looking at the ELAM capture we can conclude that the ICMP packet ingress on Eth1/10 and egress on Eth1/11.

Thanks for the read

Suresh