07-26-2017 11:58 PM - edited 03-08-2019 11:29 AM
Hi all
We have a 5548UP Nexus which has 3 Fex connected, of the type N2K-C2232TM-10GE. We have 4 ports connected to the uplink, with 3-5 meter Cisco Twinax cables. Configured is a portchannel, basically the normal Fex configuration.
Interestingly, we have on all! uplink ports to a Fex input and CRC errors, which are slowly increasing. Always both counters at the same time.
Running software is: 7.0(8)N1(1)
Here one affected interface:
# sh int e1/21
Ethernet1/21 is up
Dedicated Interface
Belongs to Po103
Hardware: 1000/10000 Ethernet, address: 547f.ee64.2b1c (bia 547f.ee64.2b1c)
Description: Fex 103
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is fex-fabric
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 51week(s) 5day(s)
Last clearing of "show interface" counters never
1 interface resets
30 seconds input rate 35627584 bits/sec, 3231 packets/sec
30 seconds output rate 6776296 bits/sec, 1124 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 37.63 Mbps, 3.94 Kpps; output rate 6.95 Mbps, 846 pps
RX
131306830478 unicast packets 157333119 multicast packets 113042313 broadcast packets
131577205910 input packets 148880646074393 bytes
68718113854 jumbo packets 0 storm suppression bytes
0 runts 0 giants 1872 CRC 0 no buffer
1872 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
148488926817 unicast packets 3378359429 multicast packets 1023470474 broadcast packets
152890756720 output packets 167223615842066 bytes
100646583092 jumbo packets
509 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
# sh int e1/21 trans
Ethernet1/21
transceiver is present
type is SFP-H10GB-CU1M
name is CISCO-TYCO
part number is 1-2053783-1
revision is R
serial number is TED1718A2GY
nominal bitrate is 10300 MBit/sec
Link length supported for copper is 1 m
cisco id is --
cisco extended id number is 4
# sh int e1/21 counters errors
--------------------------------------------------------------------------------
Port Align-Err FCS-Err Xmit-Err Rcv-Err UnderSize OutDiscards
--------------------------------------------------------------------------------
Eth1/21 0 1872 509 1872 0 0
--------------------------------------------------------------------------------
Port Single-Col Multi-Col Late-Col Exces-Col Carri-Sen Runts
--------------------------------------------------------------------------------
Eth1/21 0 0 0 0 0 0
--------------------------------------------------------------------------------
Port Giants SQETest-Err Deferred-Tx IntMacTx-Er IntMacRx-Er Symbol-Err
--------------------------------------------------------------------------------
Eth1/21 0 -- 0 0 509 0
# show fex 103 drop
No. of SS present in fex 103 is = 8
No-credit drop count on 'no-drop class' for SS 0 is = 0
No-credit drop count on 'no-drop class' for SS 1 is = 0
No-credit drop count on 'no-drop class' for SS 2 is = 0
No-credit drop count on 'no-drop class' for SS 3 is = 0
No-credit drop count on 'no-drop class' for SS 4 is = 0
No-credit drop count on 'no-drop class' for SS 5 is = 15
No-credit drop count on 'no-drop class' for SS 6 is = 4
No-credit drop count on 'no-drop class' for SS 7 is = 0
# show fex 103 deta
FEX: 103 Description: FEX0103 state: Online
FEX version: 7.0(8)N1(1) [Switch version: 7.0(8)N1(1)]
FEX Interim version: 7.0(8)N1(1)
Switch Interim version: 7.0(8)N1(1)
Extender Serial: SSI171409G6
Extender Model: N2K-C2232TM-E-10GE, Part No: 73-14645-02
Card Id: 212, Mac Addr: c4:0a:cb:e8:3c:c2, Num Macs: 64
Module Sw Gen: 21 [Switch Sw Gen: 21]
post level: complete
Pinning-mode: static Max-links: 1
Fabric port for control traffic: Eth1/21
FCoE Admin: false
FCoE Oper: true
FCoE FEX AA Configured: false
Fabric interface state:
Po103 - Interface Up. State: Active
Eth1/19 - Interface Up. State: Active
Eth1/20 - Interface Up. State: Active
Eth1/21 - Interface Up. State: Active
Eth1/22 - Interface Up. State: Active
Fex Port State Fabric Port
Eth103/1/1 Up Po103
Eth103/1/2 Up Po103
Eth103/1/3 Up Po103
Eth103/1/4 Up Po103
Eth103/1/5 Up Po103
Eth103/1/6 Up Po103
Eth103/1/7 Up Po103
Eth103/1/8 Up Po103
Eth103/1/9 Up Po103
Eth103/1/10 Up Po103
Eth103/1/11 Up Po103
Eth103/1/12 Up Po103
Eth103/1/13 Down Po103
Eth103/1/14 Up Po103
Eth103/1/15 Up Po103
Eth103/1/16 Up Po103
Eth103/1/17 Up Po103
Eth103/1/18 Up Po103
Eth103/1/19 Up Po103
Eth103/1/20 Up Po103
Eth103/1/21 Up Po103
Eth103/1/22 Up Po103
Eth103/1/23 Up Po103
Eth103/1/24 Up Po103
Eth103/1/25 Up Po103
Eth103/1/26 Up Po103
Eth103/1/27 Up Po103
Eth103/1/28 Up Po103
Eth103/1/29 Up Po103
Eth103/1/30 Down Po103
Eth103/1/31 Down Po103
Eth103/1/32 Down Po103
Logs:
07/29/2016 14:51:51.250149: Module register received
07/29/2016 14:51:51.252601: Registration response sent
07/29/2016 14:51:51.309973: create module inserted event.
07/29/2016 14:51:51.310918: Module Online Sequence
07/29/2016 14:51:55.960318: Module Online
There also seem to be output errors. The logfile shows nothing that seems wrong.
I'm open for suggestions! :)
Thanks
08-01-2017 04:40 PM
The amount of output & input errors are insignificant to the total amount of input and output traffic.
08-02-2017 12:38 AM
0 would be better ;)
I found the reason on some of the ports though, Cat5 STP cables on 10G ports :/ Replaced some cables now and it looks much better.
Luckily the old Cat5 cables are of high quality, but still old.
08-30-2017 04:45 AM
also of note is the mechanism the nexus employs to switch packets, we had an issue where we had crc errors being pushed around the switch path due to the 5k switching the packet before it was able to check for crc errors, this a new switching method that the nexus uses.
10-19-2018 01:35 PM
10-19-2018 02:09 PM
Hello,
which release are you running ?
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: