Nexus 5000 - Odd Ethernet interface behavior (link down inactive)

[sgonsalv]

Hi Guys,

This would sound really trivial but it is very odd behavior.

- We have a server connected to a 2, Nexus 5000s (for resiliancy)

- When there is no config on the ethernet interfaces whatsoever, the ethernet interface is UP / UP, there is minimal amount of traffic on the link etc. E.g.

Ethernet1/16 is up
  Hardware: 1000/10000 Ethernet, address: 000d.ece7.85d7 (bia 000d.ece7.85d7)
  Description: shipley-p1.its RK14/A13
  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is access
  full-duplex, 10 Gb/s, media type is 1/10g
  Beacon is turned off
  Input flow-control is off, output flow-control is off
  Rate mode is dedicated
  Switchport monitor is off
  Last link flapped 00:00:07
  Last clearing of "show interface" counters 05:42:32
  30 seconds input rate 0 bits/sec, 0 packets/sec
  30 seconds output rate 96 bits/sec, 0 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 0 bps, 0 pps; output rate 8 bps, 0 pps
  RX
    0 unicast packets  0 multicast packets  0 broadcast packets
    0 input packets  0 bytes
    0 jumbo packets  0 storm suppression packets
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    0 unicast packets  163 multicast packets  0 broadcast packets
    163 output packets  15883 bytes
    0 jumbo packets
    0 output errors  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble
    0 Tx pause
  1 interface resets

- As soon as I configure the link to be an access port, the link goes down, flagging "inactivity" E.g.

sh int e1/16
Ethernet1/16 is down (inactive)
  Hardware: 1000/10000 Ethernet, address: 000d.ece7.85d7 (bia 000d.ece7.85d7)
  Description: shipley-p1.its RK14/A13
  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is access
  auto-duplex, 10 Gb/s, media type is 1/10g
  Beacon is turned off
  Input flow-control is off, output flow-control is off
  Rate mode is dedicated
  Switchport monitor is off
  Last link flapped 05:38:03
  Last clearing of "show interface" counters 05:41:33
  30 seconds input rate 0 bits/sec, 0 packets/sec
  30 seconds output rate 0 bits/sec, 0 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
  RX
    0 unicast packets  0 multicast packets  0 broadcast packets
    0 input packets  0 bytes
    0 jumbo packets  0 storm suppression packets
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    0 unicast packets  146 multicast packets  0 broadcast packets
    146 output packets  13083 bytes
    0 jumbo packets
    0 output errors  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble
    0 Tx pause
  0 interface resets

- This behavior is seen on both 5Ks

- I've tried using a different set of ports, changed SFPs, and fibre cabling to no avail

- I can't seem to understand this behavior?!  In that, why would configuring the port cause the link to go down?

- If anyone has experience this before, or could shed some light on this behavior, it would be appreciated.

sh ver
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2010, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.

Software
  BIOS:      version 1.2.0
  loader:    version N/A
  kickstart: version 4.2(1)N1(1)
  system:    version 4.2(1)N1(1)
  power-seq: version v1.2
  BIOS compile time:       06/19/08
  kickstart image file is: bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin
  kickstart compile time:  4/29/2010 19:00:00 [04/30/2010 02:38:04]
  system image file is:    bootflash:/n5000-uk9.4.2.1.N1.1.bin
  system compile time:     4/29/2010 19:00:00 [04/30/2010 03:51:47]

thanks

Sheldon

[Martin Parry · Accepted solution]

Hi

If you have configured the ports as access ports for VLANs which are either:

a) not configured on the switch / present in the vlan database,

or

b) present within the databse, but is in suspended state.

You will experience this issue.  Please can you check the status of the VLAN?

Hope this helps

Martin

[sgonsalv]

Hi Martin,

Thanks for pointing that out - configuring the vlan on the switch certainly did the trick!  It is indeed a good gottcha in the absence of VTP!

thanks very much.

Cheers

Sheldon

[kapache33]

This helped me debug an issue with an access mode port that VLAN did not exist.

[habadr]

Hi Sheldon,

What configuration did you apply to the interface. Please make sure that you created the vlans that you applied to the interface

Thanks

Hatim Badr

[Tymofii Dmytrenko]

I had identical issue

Two interfaces on two different FEXes were INACTIVE. I have two Nexus 5596 in vPC and A/A FEXes.

I also use config-sync feature.

Very same configuration was applied to other ports on other FEXes and they were working with no problems.

interface Ethernet119/1/1

  inherit port-profile PP-Exchange2003

I checked VLAN status associated with this profile and it was active (of course it was, other ports were ok).

I solved it by removing port profile from this port and re-applied it... voila, port changed state to up!

Very very strange.

[engineering]

I had the same problem when I changed a formally trunk port to an access port. The port state changed to 'inactive'

I was able to solve it by doing a 'switchport access vlan 1'; commit and a ''switchport access vlan xxx' afterwards.