cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3475
Views
0
Helpful
5
Replies

Input + CRC errors on all ports where a FEX is connected to

patoberli
VIP Alumni
VIP Alumni

Hi all

We have a 5548UP Nexus which has 3 Fex connected, of the type N2K-C2232TM-10GE. We have 4 ports connected to the uplink, with 3-5 meter Cisco Twinax cables. Configured is a portchannel, basically the normal Fex configuration.

Interestingly, we have on all! uplink ports to a Fex input and CRC errors, which are slowly increasing. Always both counters at the same time.

Running software is: 7.0(8)N1(1)

Here one affected interface:

# sh int e1/21
Ethernet1/21 is up
 Dedicated Interface
  Belongs to Po103
  Hardware: 1000/10000 Ethernet, address: 547f.ee64.2b1c (bia 547f.ee64.2b1c)
  Description: Fex 103
  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is fex-fabric
  full-duplex, 10 Gb/s, media type is 10G
  Beacon is turned off
  Input flow-control is off, output flow-control is off
  Rate mode is dedicated
  Switchport monitor is off
  EtherType is 0x8100
  Last link flapped 51week(s) 5day(s)
  Last clearing of "show interface" counters never
  1 interface resets
  30 seconds input rate 35627584 bits/sec, 3231 packets/sec
  30 seconds output rate 6776296 bits/sec, 1124 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 37.63 Mbps, 3.94 Kpps; output rate 6.95 Mbps, 846 pps
  RX
    131306830478 unicast packets  157333119 multicast packets  113042313 broadcast packets
    131577205910 input packets  148880646074393 bytes
    68718113854 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  1872 CRC  0 no buffer
    1872 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    148488926817 unicast packets  3378359429 multicast packets  1023470474 broadcast packets
    152890756720 output packets  167223615842066 bytes
    100646583092 jumbo packets
    509 output error  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble 0 output discard
    0 Tx pause

# sh int e1/21 trans
Ethernet1/21
    transceiver is present
    type is SFP-H10GB-CU1M
    name is CISCO-TYCO     
    part number is 1-2053783-1    
    revision is R  
    serial number is TED1718A2GY    
    nominal bitrate is 10300 MBit/sec
    Link length supported for copper is 1 m
    cisco id is --
    cisco extended id number is 4

# sh int e1/21 counters  errors
--------------------------------------------------------------------------------
Port          Align-Err    FCS-Err   Xmit-Err    Rcv-Err  UnderSize OutDiscards
--------------------------------------------------------------------------------
Eth1/21               0       1872        509       1872          0           0
--------------------------------------------------------------------------------
Port         Single-Col  Multi-Col   Late-Col  Exces-Col  Carri-Sen       Runts
--------------------------------------------------------------------------------
Eth1/21               0          0          0          0          0           0
--------------------------------------------------------------------------------
Port          Giants SQETest-Err Deferred-Tx IntMacTx-Er IntMacRx-Er Symbol-Err
--------------------------------------------------------------------------------
Eth1/21            0          --           0           0         509          0

# show fex 103 drop
No. of SS present in fex 103 is = 8
No-credit drop count on 'no-drop class' for SS 0 is = 0
No-credit drop count on 'no-drop class' for SS 1 is = 0
No-credit drop count on 'no-drop class' for SS 2 is = 0
No-credit drop count on 'no-drop class' for SS 3 is = 0
No-credit drop count on 'no-drop class' for SS 4 is = 0
No-credit drop count on 'no-drop class' for SS 5 is = 15
No-credit drop count on 'no-drop class' for SS 6 is = 4
No-credit drop count on 'no-drop class' for SS 7 is = 0

# show fex 103 deta
FEX: 103 Description: FEX0103   state: Online
  FEX version: 7.0(8)N1(1) [Switch version: 7.0(8)N1(1)]
  FEX Interim version: 7.0(8)N1(1)
  Switch Interim version: 7.0(8)N1(1)
  Extender Serial: SSI171409G6
  Extender Model: N2K-C2232TM-E-10GE,  Part No: 73-14645-02
  Card Id: 212, Mac Addr: c4:0a:cb:e8:3c:c2, Num Macs: 64
  Module Sw Gen: 21  [Switch Sw Gen: 21]
  post level: complete
  Pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/21
  FCoE Admin: false
  FCoE Oper: true
  FCoE FEX AA Configured: false
  Fabric interface state:
    Po103 - Interface Up. State: Active
    Eth1/19 - Interface Up. State: Active
    Eth1/20 - Interface Up. State: Active
    Eth1/21 - Interface Up. State: Active
    Eth1/22 - Interface Up. State: Active
  Fex Port        State  Fabric Port
       Eth103/1/1    Up       Po103
       Eth103/1/2    Up       Po103
       Eth103/1/3    Up       Po103
       Eth103/1/4    Up       Po103
       Eth103/1/5    Up       Po103
       Eth103/1/6    Up       Po103
       Eth103/1/7    Up       Po103
       Eth103/1/8    Up       Po103
       Eth103/1/9    Up       Po103
      Eth103/1/10    Up       Po103
      Eth103/1/11    Up       Po103
      Eth103/1/12    Up       Po103
      Eth103/1/13  Down       Po103
      Eth103/1/14    Up       Po103
      Eth103/1/15    Up       Po103
      Eth103/1/16    Up       Po103
      Eth103/1/17    Up       Po103
      Eth103/1/18    Up       Po103
      Eth103/1/19    Up       Po103
      Eth103/1/20    Up       Po103
      Eth103/1/21    Up       Po103
      Eth103/1/22    Up       Po103
      Eth103/1/23    Up       Po103
      Eth103/1/24    Up       Po103
      Eth103/1/25    Up       Po103
      Eth103/1/26    Up       Po103
      Eth103/1/27    Up       Po103
      Eth103/1/28    Up       Po103
      Eth103/1/29    Up       Po103
      Eth103/1/30  Down       Po103
      Eth103/1/31  Down       Po103
      Eth103/1/32  Down       Po103
Logs:
07/29/2016 14:51:51.250149: Module register received
07/29/2016 14:51:51.252601: Registration response sent
07/29/2016 14:51:51.309973: create module inserted event.
07/29/2016 14:51:51.310918: Module Online Sequence
07/29/2016 14:51:55.960318: Module Online

There also seem to be output errors. The logfile shows nothing that seems wrong.

I'm open for suggestions! :)

Thanks

5 Replies 5

Leo Laohoo
Hall of Fame
Hall of Fame

The amount of output & input errors are insignificant to the total amount of input and output traffic. 

0 would be better ;)

I found the reason on some of the ports though, Cat5 STP cables on 10G ports :/ Replaced some cables now and it looks much better.

Luckily the old Cat5 cables are of high quality, but still old.

also of note is the mechanism the nexus employs to switch packets,  we had an issue where we had crc errors being pushed around the switch path due to the 5k switching the packet before it was able to check for crc errors,  this a new switching method that the nexus uses.   

I have this problem as well. Nexus 9332 with N2K-C2348 series FEXs. Most do not have this issue however none of the following has helped - cleaning fiber, replacing QSFPs moving to different FEX uplink port, and replacing fiber.
I'll be moving ports on one of my 9Ks to see if the errors go away on that device.
We know that clearing counters doesn't fix the problem just masks the symptom.
If all or none of my FEX ports had this issue I would not be concerned. But it is about 7% of them that have errors.

Hello,

 

which release are you running ?

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: