Output errors on Nexus and fex

[bvj197222]

I have a new Nexus 5548, running 6.0(2)N1(2), upgraded from ver 5.x. Both versions had same output errors. This is a single 5548, connected to two Nexus 2224 FEX. The port-channel from the 5548 to the 6500 has many errors, this is the statistics;

(counters cleared two hours ago)

noassnx01#  sh int po1

port-channel1 is up

  Hardware: Port-Channel, address: 002a.6a81.fe87 (bia 002a.6a81.fe87)

  Description: noaascore01

  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

  reliability 245/255, txload 1/255, rxload 1/255

  Encapsulation ARPA

  Port mode is trunk

  full-duplex, 10 Gb/s

  Input flow-control is off, output flow-control is off

  Switchport monitor is off

  EtherType is 0x8100

  Members in this channel: Eth1/32

  2 interface resets

  30 seconds input rate 8599576 bits/sec, 1816 packets/sec

  30 seconds output rate 171224 bits/sec, 75 packets/sec

  Load-Interval #2: 5 minute (300 seconds)

    input rate 8.87 Mbps, 1.81 Kpps; output rate 232.41 Kbps, 58 pps

  RX

    2009138 unicast packets  11204001 multicast packets  204057 broadcast packets

    13417196 input packets  8725391658 bytes

    1142454 jumbo packets  0 storm suppression bytes

    0 runts  0 giants  619614 CRC  0 no buffer

    619614 input error  0 short frame  0 overrun   0 underrun  0 ignored

    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop

    0 input with dribble  0 input discard

    0 Rx pause

  TX

    1534174 unicast packets  36760 multicast packets  7993 broadcast packets

    1578927 output packets  684418660 bytes

    258667 jumbo packets

    71 output errors  0 collision  0 deferred  0 late collision

    0 lost carrier  0 no carrier  0 babble 0 output discard

    0 Tx pause

  2 interface resets

noassnx01# sh int po101

port-channel101 is up

  Hardware: Port-Channel, address: 002a.6a81.fe82 (bia 002a.6a81.fe82)

  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

  reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA

  Port mode is fex-fabric

  full-duplex, 10 Gb/s

  Input flow-control is off, output flow-control is off

  Switchport monitor is off

  EtherType is 0x8100

  Members in this channel: Eth1/27

  0 interface resets

  30 seconds input rate 91712 bits/sec, 24 packets/sec

  30 seconds output rate 287552 bits/sec, 114 packets/sec

  Load-Interval #2: 5 minute (300 seconds)

    input rate 99.95 Kbps, 21 pps; output rate 250.94 Kbps, 77 pps

  RX

    770397 unicast packets  4129 multicast packets  4097 broadcast packets

    778623 input packets  330120242 bytes

    133739 jumbo packets  0 storm suppression bytes

    0 runts  0 giants  77 CRC  0 no buffer

    77 input error  0 short frame  0 overrun   0 underrun  0 ignored

    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop

    0 input with dribble  0 input discard

    0 Rx pause

  TX

    1253970 unicast packets  846071 multicast packets  108775 broadcast packets

    2208816 output packets  1592149563 bytes

    846087 jumbo packets

    67699 output errors  0 collision  0 deferred  0 late collision

    0 lost carrier  0 no carrier  0 babble 0 output discard

    0 Tx pause

  0 interface resets

noassnx01# sh int eth102/1/2

Ethernet102/1/2 is up

  Hardware: 100/1000 Ethernet, address: f029.29cd.2283 (bia f029.29cd.2283)

  Description: TBGDC02

  MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec

  reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA

  Port mode is access

  full-duplex, 100 Mb/s

  Beacon is turned off

  Input flow-control is off, output flow-control is on

  Switchport monitor is off

  EtherType is 0x8100

  Last link flapped 00:07:28

  Last clearing of "show interface" counters 02:16:30

  11 interface resets

  30 seconds input rate 0 bits/sec, 0 packets/sec

  30 seconds output rate 81392 bits/sec, 42 packets/sec

  Load-Interval #2: 5 minute (300 seconds)

    input rate 0 bps, 0 pps; output rate 121.16 Kbps, 53 pps

  RX

    421422 unicast packets  7734 multicast packets  317 broadcast packets

    429473 input packets  274142208 bytes

    0 jumbo packets  0 storm suppression bytes

    0 runts  0 giants  0 CRC  0 no buffer

    0 input error  0 short frame  0 overrun   0 underrun  0 ignored

    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop

    0 input with dribble  0 input discard

    0 Rx pause

  TX

    391690 unicast packets  693846 multicast packets  37829 broadcast packets

    1153849 output packets  829770184 bytes

    303 jumbo packets

    30484 output errors  0 collision  0 deferred  0 late collision

    0 lost carrier  0 no carrier  0 babble 0 output discard

    0 Tx pause

noassnx01#  sh int po1

port-channel1 is up

  Hardware: Port-Channel, address: 002a.6a81.fe87 (bia 002a.6a81.fe87)

  Description: noaascore01

  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

  reliability 245/255, txload 1/255, rxload 1/255

  Encapsulation ARPA

  Port mode is trunk

  full-duplex, 10 Gb/s

  Input flow-control is off, output flow-control is off

  Switchport monitor is off

  EtherType is 0x8100

  Members in this channel: Eth1/32

  2 interface resets

  30 seconds input rate 8599576 bits/sec, 1816 packets/sec

  30 seconds output rate 171224 bits/sec, 75 packets/sec

  Load-Interval #2: 5 minute (300 seconds)

    input rate 8.87 Mbps, 1.81 Kpps; output rate 232.41 Kbps, 58 pps

  RX

    2009138 unicast packets  11204001 multicast packets  204057 broadcast packets

    13417196 input packets  8725391658 bytes

    1142454 jumbo packets  0 storm suppression bytes

    0 runts  0 giants  619614 CRC  0 no buffer

    619614 input error  0 short frame  0 overrun   0 underrun  0 ignored

    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop

    0 input with dribble  0 input discard

    0 Rx pause

  TX

    1534174 unicast packets  36760 multicast packets  7993 broadcast packets

    1578927 output packets  684418660 bytes

    258667 jumbo packets

    71 output errors  0 collision  0 deferred  0 late collision

    0 lost carrier  0 no carrier  0 babble 0 output discard

    0 Tx pause

  2 interface resets

The same goes for the two port channels going to the fex'es;

noassnx01# sh int po101
port-channel101 is up
  Hardware: Port-Channel, address: 002a.6a81.fe82 (bia 002a.6a81.fe82)
  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is fex-fabric
  full-duplex, 10 Gb/s
  Input flow-control is off, output flow-control is off
  Switchport monitor is off
  EtherType is 0x8100
  Members in this channel: Eth1/27
  0 interface resets
  30 seconds input rate 91712 bits/sec, 24 packets/sec
  30 seconds output rate 287552 bits/sec, 114 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 99.95 Kbps, 21 pps; output rate 250.94 Kbps, 77 pps
  RX
    770397 unicast packets  4129 multicast packets  4097 broadcast packets
    778623 input packets  330120242 bytes
    133739 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  77 CRC  0 no buffer
    77 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    1253970 unicast packets  846071 multicast packets  108775 broadcast packets
    2208816 output packets  1592149563 bytes
    846087 jumbo packets
    67699 output errors  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble 0 output discard
    0 Tx pause
  0 interface resets

We also have errors on some of the interfaces on the fex'es, but it's mainly output errors as shown here;

noassnx01# sh int eth102/1/2
Ethernet102/1/2 is up
  Hardware: 100/1000 Ethernet, address: f029.29cd.2283 (bia f029.29cd.2283)
  Description: TBGDC02
  MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is access
  full-duplex, 100 Mb/s
  Beacon is turned off
  Input flow-control is off, output flow-control is on
  Switchport monitor is off
  EtherType is 0x8100
  Last link flapped 00:07:28
  Last clearing of "show interface" counters 02:16:30
  11 interface resets
  30 seconds input rate 0 bits/sec, 0 packets/sec
  30 seconds output rate 81392 bits/sec, 42 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 0 bps, 0 pps; output rate 121.16 Kbps, 53 pps
  RX
    421422 unicast packets  7734 multicast packets  317 broadcast packets
    429473 input packets  274142208 bytes
    0 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    391690 unicast packets  693846 multicast packets  37829 broadcast packets
    1153849 output packets  829770184 bytes
    303 jumbo packets
    30484 output errors  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble 0 output discard
    0 Tx pause

Can anyone give me an idea of where to look?