Topology
I have fex connected to eth104/1/10 to N5k-2
++++Currently as we could see there is no traffic or errors on interface
N5k-2# show interface ethernet 104/1/10
Ethernet104/1/10 is up
Last clearing of "show interface" counters 00:01:04
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 64 bits/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 1.33 Gbps, 52.47 Kpps; output rate 120 bps, 0 pps
RX
0 unicast packets 0 multicast packets 0 broadcast packets
0 input packets 0 bytes
0 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
0 unicast packets 3 multicast packets 0 broadcast packets
3 output packets 930 bytes
0 jumbo packets
0 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
0 interface resets
++++There are no stomp or crc error on the asic currently
N5k-2# show hardware internal carmel asic 0 interrupt match mtu
Slot 0 Carmel 0 interrupt source status:
Interrupt name |Status
----------------------------------------------------------------+------
Done.
++++Now i will send jumbo frame size 1600 from IXIA connected to port 104/1/10 on N5k-2
++++You could see frame as counted as jumbo on the HIF port 104/1/10
N5k-2# show interface ethernet 104/1/10
Ethernet104/1/10 is up
Hardware: 1000/10000 Ethernet, address: f84f.57b9.84cb (bia f84f.57b9.84cb)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 5/255
Encapsulation ARPA
Port mode is access
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is on
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 00:25:14
Last clearing of "show interface" counters 00:04:39
30 seconds input rate 60392440 bits/sec, 4718 packets/sec
30 seconds output rate 160 bits/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 218.38 Mbps, 457 pps; output rate 120 bps, 0 pps
RX
136725 unicast packets 0 multicast packets 0 broadcast packets
136724 input packets 218758400 bytes
136722 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
0 unicast packets 15 multicast packets 0 broadcast packets
15 output packets 4650 bytes
0 jumbo packets
0 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
0 interface resets
++++On fex we could see traffic is received on Hif9 and send to Nif0
++++I could see frames are received on HIF9 ( 104/1/10)
fex-104# show platform software woodside rmon 0 hif9 for counter RX_PKT_1519 ( since i am sending packects size 1600)
++++We could see that frames are counted as TX error on NIF0 port
fex-104# show platform software woodside rmon 0 ni0
++++Fex interface (eth1/2) see these frames as rx jumbo and input error
N5k-2# show interface ethernet 1/2
Ethernet1/2 is up
Hardware: 1000/10000 Ethernet, address: 002a.6a6a.4f09 (bia 002a.6a6a.4f09)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 128/255, txload 35/255, rxload 37/255
Encapsulation ARPA
Port mode is fex-fabric
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 02:41:50
Last clearing of "show interface" counters 00:08:57
30 seconds input rate 1779506656 bits/sec, 139018 packets/sec
30 seconds output rate 144 bits/sec, 15 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 1.46 Gbps, 114.02 Kpps; output rate 1.40 Gbps, 114.02 Kpps
RX
2501 unicast packets 36368655 multicast packets 0 broadcast packets
36371156 input packets 58192101472 bytes
36369824 jumbo packets 0 storm suppression bytes
0 runts 0 giants 36368484 CRC 0 no buffer
36368484 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
1217 unicast packets 36368658 multicast packets 0 broadcast packets
36369875 output packets 55862244410 bytes
83 jumbo packets
83 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
0 interface resets
++++Now lets check the Carmel and Forwarding asic and see if the interrupt are triggered on those in the right directions
N5k-2# show hardware internal carmel all-ports
Carmel Port Info:
name |log|car|mac|flag|adm|opr|m:s:l|ipt|fab|xcar|xpt|if_index|diag|ucVer
-------+---+---+---+----+---+---+-----+---+---+----+---+--------+----+-----
xgb1/1 |0 |0 |0 -|b7 |dis|dn |0:0:f|0 |92 |0 |0 |1a000000|pass| 0.00
xgb1/2 |1 |0 |1 p|b7 |en |up |1:1:f|1 |88 |0 |0 |1a001000|pass| 0.00
xgb1/3 |2 |0 |2 p|b7 |dis|dn |2:2:f|2 |93 |0 |0 |1a002000|pass| 0.00
xgb1/4 |3 |0 |3 p|b7 |en |up |3:3:f|3 |89 |0 |0 |1a003000|pass| 0.00
xgb1/5 |4 |0 |4 p|b7 |dis|dn |4:4:f|4 |90 |0 |0 |1a004000|pass| 0.00
xgb1/6 |5 |0 |5 -|b7 |dis|dn |5:5:f|5 |94 |0 |0 |1a005000|pass| 0.00
++++Based on the above output my input and output port is mapped to asic 0 and fw instance 1 ( for ingress ) and 3 ( for egress)
++++As expected we see mtu violation on ingress
N5k-2# show hardware internal carmel asic 0 interrupt match mtu
Slot 0 Carmel 0 interrupt source status:
Interrupt name |Status
----------------------------------------------------------------+------
car_bm1_INT_err_ig_mtu_vio |Set
and frame is stopmed on ingress and egress asic ( fw 1 and 3)
N5k-2# show hardware internal carmel asic 0 interrupt match stomp
Slot 0 Carmel 0 interrupt source status:
Interrupt name |Status
----------------------------------------------------------------+------
car_fi1_INT_fi_rx_pkt_crc_stomp |Set
car_fi3_INT_fi_rx_pkt_crc_stomp |Set
car_bm1_INT_norm_igrd_crc_stomp |Set
car_bm1_INT_norm_egrd_crc_stomp |Set
car_bm3_INT_norm_egrd_crc_stomp |Set
car_fw1_INT_ig_pkt_err_eth_crc_stomp |Set
car_fw3_INT_eg_pkt_err_eth_crc_stomp |Set
car_mm1_pb_INT_buf_ecc_crc_rx_crc_stomp |Set
car_mm1_pb_INT_buf_ecc_crc_tx_crc_stomp |Set
car_mm1_xe_INT_xg_rx_crc_stomped |Set
car_mm3_pb_INT_buf_ecc_crc_tx_crc_stomp |Set
Done.
and counted as output jumbo and error on exit interface 1/4
N5k-2# show interface ethernet 1/4
Ethernet1/4 is up
Hardware: 1000/10000 Ethernet, address: 002a.6a6a.4f0b (bia 002a.6a6a.4f0b)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 43/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 02:36:42
Last clearing of "show interface" counters 00:12:23
30 seconds input rate 856 bits/sec, 1 packets/sec
30 seconds output rate 1694861136 bits/sec, 138826 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 648 bps, 1 pps; output rate 1.70 Gbps, 138.79 Kpps
RX
0 unicast packets 1156 multicast packets 0 broadcast packets
1156 input packets 87271 bytes
0 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
0 unicast packets 64874742 multicast packets 0 broadcast packets
64874742 output packets 98998841184 bytes
64874730 jumbo packets
64874730 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
0 interface resets
++++When it reach N5k-1 eth1/4 its counted as input error
N5k-1# show interface ethernet 1/4
Ethernet1/4 is up
Hardware: 1000/10000 Ethernet, address: 002a.6a62.b06b (bia 002a.6a62.b06b)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 1/255, txload 1/255, rxload 43/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 02:37:28
Last clearing of "show interface" counters 00:34:57
30 seconds input rate 1695456184 bits/sec, 138874 packets/sec
30 seconds output rate 856 bits/sec, 1 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 1.70 Gbps, 138.88 Kpps; output rate 648 bps, 1 pps
RX
0 unicast packets 162427934 multicast packets 0 broadcast packets
162427934 input packets 247864983219 bytes
162427899 jumbo packets 0 storm suppression bytes
0 runts 0 giants 162427899 CRC 0 no buffer
162427899 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
0 unicast packets 3252 multicast packets 0 broadcast packets
3252 output packets 245671 bytes
0 jumbo packets
0 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
0 interface resets
++++no mtu voilation on N5k-1
N5k-1# show hardware internal carmel asic 0 interrupt match mtu
Slot 0 Carmel 0 interrupt source status:
Interrupt name |Status
----------------------------------------------------------------+------
Done.
++++checking the Carmel and mac asic mapping for incoming ( eth1/4) and outgoing (eth1/1) interface
N5k-1# show hardware internal carmel all-ports
Carmel Port Info:
name |log|car|mac|flag|adm|opr|m:s:l|ipt|fab|xcar|xpt|if_index|diag|ucVer
-------+---+---+---+----+---+---+-----+---+---+----+---+--------+----+-----
xgb1/1 |0 |0 |0 p|b7 |en |up |0:0:f|0 |92 |0 |0 |1a000000|pass| 0.00
xgb1/2 |1 |0 |1 -|b7 |dis|dn |1:1:f|1 |88 |0 |0 |1a001000|pass| 0.00
xgb1/3 |2 |0 |2 p|b7 |dis|dn |2:2:f|2 |93 |0 |0 |1a002000|pass| 0.00
xgb1/4 |3 |0 |3 p|b7 |en |up |3:3:f|3 |89 |0 |0 |1a003000|pass| 0.00
xgb1/5 |4 |0 |4 p|b7 |dis|dn |4:4:f|4 |90 |0 |0 |1a004000|pass| 0.00
xgb1/6 |5 |0 |5 -|b7 |dis|dn |5:5:f|5 |94 |0 |0 |1a005000|pass| 0.00
++++As could we seen that crc and stomp is marked for ingress and egress asic.
N5k-1# show hardware internal carmel asic 0 interrupt match stomp
Slot 0 Carmel 0 interrupt source status:
Interrupt name |Status
----------------------------------------------------------------+------
car_fi0_INT_fi_rx_pkt_crc_stomp |Set
car_bm0_INT_norm_egrd_crc_stomp |Set
car_bm3_INT_norm_igrd_crc_stomp |Set
car_fw0_INT_eg_pkt_err_eth_crc_stomp |Set
car_fw3_INT_ig_pkt_err_eth_crc_stomp |Set
car_mm0_pb_INT_buf_ecc_crc_tx_crc_stomp |Set
car_mm3_pb_INT_buf_ecc_crc_rx_crc_stomp |Set
++++ Packets fron N5k-1 is marked as tx output error
N5k-1# show interface ethernet 1/1
Ethernet1/1 is up
Hardware: 1000/10000 Ethernet, address: 002a.6a62.b068 (bia 002a.6a62.b068)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 43/255, rxload 1/255
Encapsulation ARPA
Port mode is fex-fabric
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 01:32:39
Last clearing of "show interface" counters 00:37:37
30 seconds input rate 54408 bits/sec, 5 packets/sec
30 seconds output rate 1705846216 bits/sec, 138817 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 52.11 Kbps, 1 pps; output rate 1.71 Gbps, 138.82 Kpps
RX
13026 unicast packets 712 multicast packets 0 broadcast packets
13738 input packets 14833014 bytes
8420 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
4762 unicast packets 183981116 multicast packets 1 broadcast packets
183985879 output packets 282594803247 bytes
183980383 jumbo packets
183980383 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
0 interface resets
++++From Fex we could see the input and output rate received on ni1 and send hif9
fex-104# show platform software woodside rate
++++Checking the rx error on ni1 interface
fex-104# show platform software woodside rmon 0 ni1
++++Checking the tx error on hif9
fex-104# show platform software woodside rmon 0 hif9
This finishes the END to END tracking of jumbo frame ,thanks for looking and feedback!!