GigabitEthernet is down, line protocol is down (notconnect)

[fsm]

Hi all,

I've got a problem with some sfps which status are "GigabitEthernet is down, line protocol is down (notconnect)"

I've got the impression that this is a problem with the hardware (either the sfps or the fibers between them) since configuration wise it looks OK, but I would like your opinion to rule out any configuration problems or something I've missed, before ordering more hardware.

I can provide more info if necessary

The setup:

2 stacks of 3 switches (6 in total, 12 sfps)

Port-channel between these stacks are always on port gx/0/51 and gx/0/52

On all 6 switches, ios is the same

Cisco IOS Software, C2960X Software (C2960X-UNIVERSALK9-M), Version 15.2(6)E1, RELEASE SOFTWARE (fc4)

Port-channel is mode "on" on all switches

The problem:

Out of 12 sfps, only 4 are working

No lights on the non working ones and if I run the command "show int gx/y/z" this is the output

SW-S02#show int g1/0/51

GigabitEthernet1/0/51 is down, line protocol is down (notconnect)

  Hardware is Gigabit Ethernet, address is 003c.1093.f533 (bia 003c.1093.f533)

  Description: SW-02 UPlink to SW-01

  MTU 1500 bytes, BW 10000 Kbit/sec, DLY 1000 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive not set

  Auto-duplex, Auto-speed, link type is auto, media type is 1000BaseLX SFP

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output never, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 0 bits/sec, 0 packets/sec

     0 packets input, 0 bytes, 0 no buffer

     Received 0 broadcasts (0 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     0 input packets with dribble condition detected

     0 packets output, 0 bytes, 0 underruns

     0 output errors, 0 collisions, 4 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

More info:

Ports are connected as such. Example

SW01 1/0/51==>SW02 1/0/51

But since the person who setup the stack did not force priorities and such the stack looks like the following

SW01	SW02
1	3
2	2
3	1

So logically

SW01 1/0/51==>SW02 3/0/51

Under show inventory, all sfps'a are recognized by the switch. Example

SW-S02#show inventory

NAME: "GigabitEthernet1/0/51", DESCR: "1000BaseLX SFP"

PID: GLC-LH-SMD          , VID: V01  , SN: LEG3A91D9036 

This is the output os sh int status

SW-S02#show int g1/0/51 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi1/0/51  SW-02 UPlink t notconnect   1            auto   auto 1000BaseLX SFP

SW-S02#show int g1/0/52 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi1/0/52  SW-S02 UPlink  notconnect   1            auto   auto 1000BaseLX SFP

SW-S02#show int g2/0/51 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi2/0/51  SW-02 UPlink t notconnect   1            auto   auto 1000BaseLX SFP

SW-S02#show int g2/0/52 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi2/0/52  SW-S02 UPlink  connected    trunk      a-full a-1000 1000BaseLX SFP

SW-S02#show int g3/0/51 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi3/0/51  SW-02 UPlink t notconnect   1            auto   auto 1000BaseLX SFP

SW-S02#show int g3/0/52 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi3/0/52  SW-S02 UPlink  connected    trunk      a-full a-1000 1000BaseLX SFP

SW-S02#

SW-S01#show int g1/0/51 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi1/0/51  SW-S01         notconnect   1            auto   auto 1000BaseLX SFP

SW-S01#show int g1/0/52 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi1/0/52  SW-S01         connected    trunk      a-full a-1000 1000BaseLX SFP

SW-S01#show int g2/0/51 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi2/0/51  SW-01          notconnect   1            auto   auto 1000BaseLX SFP

SW-S01#show int g2/0/52 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi2/0/52  SW-S01         connected    trunk      a-full a-1000 1000BaseLX SFP

SW-S01#show int g3/0/51 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi3/0/51  SW-01          notconnect   1            auto   auto 1000BaseLX SFP

SW-S01#show int g3/0/52 status

Port      Name               Status       Vlan       Duplex  Speed Type

Gi3/0/52  SW-S01         notconnect   1            auto   auto 1000BaseLX SFP

SW-S01#

Confiuration of these ports

SW-S02#show run int g1/0/51

Building configuration...

Current configuration : 130 bytes

!

interface GigabitEthernet1/0/51

 description SW-02 UPlink to SW-01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S02#show run int g1/0/52

Building configuration...

Current configuration : 132 bytes

!

interface GigabitEthernet1/0/52

 description SW-S02 UPlink to SW-S01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S02#show run int g2/0/51

Building configuration...

Current configuration : 130 bytes

!

interface GigabitEthernet2/0/51

 description SW-02 UPlink to SW-01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S02#show run int g2/0/52

Building configuration...

Current configuration : 151 bytes

!

interface GigabitEthernet2/0/52

 description SW-S02 UPlink to SW-S01

 switchport mode trunk

 downshift disable

 channel-group 1 mode on

end

SW-S02#show run int g3/0/51

Building configuration...

Current configuration : 130 bytes

!

interface GigabitEthernet3/0/51

 description SW-02 UPlink to SW-01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S02#show run int g3/0/52

Building configuration...

Current configuration : 132 bytes

!

interface GigabitEthernet3/0/52

 description SW-S02 UPlink to SW-S01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S02#show run in port-channel 1

Building configuration...

Current configuration : 99 bytes

!

interface Port-channel1

 description SW-S02 UPlink to SW-S01

 switchport mode trunk

end

SW-S01#show run int g1/0/51

Building configuration...

Current configuration : 111 bytes

!

interface GigabitEthernet1/0/51

 description SW-S01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S01#show run int g1/0/52

Building configuration...

Current configuration : 111 bytes

!

interface GigabitEthernet1/0/52

 description SW-S01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S01#show run int g2/0/51

Building configuration...

Current configuration : 110 bytes

!

interface GigabitEthernet2/0/51

 description SW-01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S01#show run int g2/0/52

Building configuration...

Current configuration : 111 bytes

!

interface GigabitEthernet2/0/52

 description SW-S01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S01#show run int g3/0/51

Building configuration...

Current configuration : 110 bytes

!

interface GigabitEthernet3/0/51

 description SW-01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S01#show run int g3/0/52

Building configuration...

Current configuration : 111 bytes

!

interface GigabitEthernet3/0/52

 description SW-S01

 switchport mode trunk

 channel-group 1 mode on

end

SW-S01#show run int port-channel 1

Building configuration...

Current configuration : 78 bytes

!

interface Port-channel1

 description SW-S01

 switchport mode trunk

end

Show ip int brief (edited)

SW-S02#show ip int brief

Interface              IP-Address      OK? Method Status                Protocol

GigabitEthernet1/0/51  unassigned      YES unset  down                  down   

GigabitEthernet1/0/52  unassigned      YES unset  down                  down   

GigabitEthernet2/0/51  unassigned      YES unset  down                  down   

GigabitEthernet2/0/52  unassigned      YES unset  up                    up     

GigabitEthernet3/0/51  unassigned      YES unset  down                  down   

GigabitEthernet3/0/52  unassigned      YES unset  up                    up     

Port-channel1          unassigned      YES unset  up                    up     

SW-S01#show ip int brief

Interface              IP-Address      OK? Method Status                Protocol

GigabitEthernet1/0/51  unassigned      YES unset  down                  down   

GigabitEthernet1/0/52  unassigned      YES unset  up                    up     

GigabitEthernet2/0/51  unassigned      YES unset  down                  down   

GigabitEthernet2/0/52  unassigned      YES unset  up                    up     

GigabitEthernet3/0/51  unassigned      YES unset  down                  down   

GigabitEthernet3/0/52  unassigned      YES unset  down                  down   

Port-channel1          unassigned      YES unset  up                    up     

[Georg Pauwen]

Hello,

what was the solution to this ?

[fsm]

we issued the service-unsupported transceiver and it did not work so we switched the rx/tx on the non working ones and they started coming up.

Now I still have doubts since 4/12 are Legrand brand and the 4 ones that originally worked are of some other brand (probably Cisco) so perhaps the command was necessary but the problem was indeed the cabling.

[Patrick.Williams]

Your port channel configurations are configured correctly, the fact that your device is detecting each sfp module I would move to look at the media that is connecting the switches together, have you tried switching the fiber cables around by using the pairs that are confirmed to be working to the non working ports?.I would definitely recommend that to see if you have some bad fiber pairs in the mix before looking at the sfp as an issue.

[fsm]

Thanks for the reply Patrick.

Switching the ends was my first though but the connectors have some sort of protection that does not let me switch the ends.

Usually, when I have any problem of the sort, switching the ends would be my first troubleshooting step, as everything seems to be pointing to a hardware problem (either sfp or fibers).

[pieterh]

I did not see in your post that you checked if the light is comming through ?
if you do not have specialized fiber tester, you can use or a simple laser pointer/pen/ beamer remote
NB! do not look in the beam directly, point the connector to a reflective surface to see if light comes out
1) check if both fibers come through
2) check if a-b pair is correct, and there is a "cross" in the total path

[Mustak!]

Hi,

The SFP is recognised by the switch and configuration looks OK. Common problem is fibre patch leads. Have you checked the fibre cables to make sure transmit is going to receive.