we have Nexus 7k with two N7K-SUP2 and one 1/10 Gbps Ethernet Module (N7K-F248XP-25), version 6.1(2)
The problem is that suddenly a port 1000base-SX on N7K-F248XP-25 changed state to down and we have no idea how to make it work again.
We changed SFP, tried "shut/ no shut" and "no switchport", tried to connect to a different switch. All without any results, the port is still down.
We didn't try resetting the module or upgrading the nxos yet ...
Did anybody face a similar issue ? Maybe there is something else we should perform ?
Thanks and happy New Year!
Is the log showing anything error message before it went down? Please post the show interface ether x/y and show logging from the device. Is it only for that port or multiple ports having this issue?
the issue is with one port.
Ethernet3/12 is down (Link not connected)
admin state is up, Dedicated Interface
Hardware: 1000/10000 Ethernet, address: 8478.ac56.d8c2 (bia 4403.a718.ced3)
Description: wroc1s63 via IPS
Internet Address is 22.214.171.124/30
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
full-duplex, 1000 Mb/s, media type is 1G
Beacon is turned off
Auto-Negotiation is turned off
Input flow-control is off, output flow-control is off
Auto-mdix is turned off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
EEE (efficient-ethernet) : n/a
Last link flapped 1d23h
Last clearing of "show interface" counters never
4 interface resets
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 0 bits/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
119360 unicast packets 0 multicast packets 0 broadcast packets
118760 input packets 168876720 bytes
0 jumbo packets 0 storm suppression packets
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
298535 unicast packets 6873009 multicast packets 2244793 broadcast packets
9415735 output packets 2393355700 bytes
0 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
Dec 30 20:09:34 wroc1n71 : 2013 Dec 30 20:09:34 CET: %ETHPORT-5-IF_DOWN_CFG_CHANGE: Interface Ethernet3/12 is down(Config change) (after configuring port to "no switchport")
Dec 30 20:09:35 wroc1n71 : 2013 Dec 30 20:09:35 CET: %ETHPORT-5-SPEED: Interface Ethernet3/12, operational speed changed to 1 Gbps
Dec 30 20:09:35 wroc1n71 : 2013 Dec 30 20:09:35 CET: %ETHPORT-5-IF_DUPLEX: Interface Ethernet3/12, operational duplex mode changed to Full
Dec 30 20:09:35 wroc1n71 : 2013 Dec 30 20:09:35 CET: %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet3/12, operational Receive Flow Control state changed to off
Dec 30 20:09:35 wroc1n71 : 2013 Dec 30 20:09:35 CET: %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet3/12, operational Transmit Flow Control state changed to off
Dec 30 20:09:35 wroc1n71 : 2013 Dec 30 20:09:35 CET: %ETHPORT-5-IF_UP: Interface Ethernet3/12 is up in Layer3
Dec 30 20:09:35 wroc1n71 : 2013 Dec 30 20:09:35 CET: %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet3/12 is down (Link failure)
I strongly suspect some layer 1 issue here. You have mentioned that you changed the SFP and tried to flap and change the configs as well which is strange. What does " sh inter eth 3/12 capabilities " and "show interface eth3/12 trans details" show for working and not working port? Do you see any difference in power levels between working and not working port? Bettter to open up a TAC case and get more details.
there is no differences for working and not-working port regarding capabilities:
Type (SFP capable): 1000base-SX
Trunk encap. type: 802.1Q
FabricPath capable: yes
Broadcast suppression: percentage(0-100)
Rate mode: dedicated
Port mode: Routed,Switched
QOS scheduling: rx-(8q4t),tx-(3p5q1t)
CoS rewrite: yes
ToS rewrite: yes
TDR capable: no
Link Debounce: yes
Link Debounce Time: yes
FEX Fabric: yes
dot1Q-tunnel mode: yes
Pvlan Trunk capable: yes
Port Group Members: 9-12
EEE (efficient-eth): no
PFC capable: yes
Power levels for no-working and for working port:
Tx Power -5.07 dBm -5.10 dBm
Rx Power -7.37 dBm -6.07 dBm
Maybe this is a module hardware problem after all ...
Could be a hardware issue. But unless we do a fault analysis we might not be able to confirm for sure. Get in touch with TAC for additional assistance.
Ok two things. First tell us the " show module " output and run som hardware disgnostics on the module.
Second thing is what is the device connected to the other end? Please try hard-coding the speed to 1G on the other end device. I have seen this kind of issue where you have to hard-code the speed to 1G on the networking devices to make it work specially in routers.
Try this and let us know.
show module command:
Mod Ports Module-Type Model Status
--- ----- ----------------------------------- ------------------ ----------
3 48 1/10 Gbps Ethernet Module N7K-F248XP-25 ok
Mod Sw Hw
--- -------------- ------
3 6.1(2) 1.0
Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
3 44-03-a7-18-ce-c8 to 44-03-a7-18-ce-fb JAF1649ASSS
Mod Online Diag Status
Chassis Ejector Support: Enabled
Left ejector CLOSE, Right ejector CLOSE, Module HW does support ejector based shutdown.
On the other end normally was connected IPS McAfee IntruShield, but we tried also connect to Catalyst 3550 (GBIC GE port).
What hardware diagnostics do you mean ? We cannot afford to do any disruptive tests.
Please have a look at the below link, it should help you.
Yes, the diagnostics are disruptive but system will warn you for sure. Dont be in hurry to press enter :-)
Try to do the complete diagnostic test for specified port that is failing.
the link went up when I configured "speed nonegotiate" on the cat.3550 1G port.
Should I consider using this option as a good practice when connecting any device via 1G to Nexus ?