cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
14132
Views
0
Helpful
14
Replies
Highlighted
Beginner

MDS 9148 NX-OS 5.0(1a) Link failure Problem

Hi all,

I'm finding the following problem on my 2 MDS9148 switches.

2012 May  7 14:15:54 SAN01 %PORT-4-IF_SFP_WARNING: Interface fc1/7, Low Rx Power Warning  BK01
2012 May  7 14:15:54 SAN01 %PORT-3-IF_SFP_7ALARM: Interface fc1/7, Low Rx Power Alarm  BK01 

And finally the FC interfaces go down. We checked the fiber cable to be sure that it is not a physical issue and the fiber cables are ok.

This happens with fc1/1, 1/3, 1/4 and several more.

Does anyone have an idea why this could be happening and what the error might be?

Your help will be much appreciated.

Thanks in advance,

Kind regards

14 REPLIES 14
Highlighted
Beginner

Are those ISL ports? What is connected to those ports?

Collect following and post it here

sh int fc1/7 counters  details

sh int fc 1/7 transceiver details

Highlighted


Krun_shah,

I am posting the fc1/1 and fc1/5 instead of fc1/7. All of them are experiencing the same
problem.

To those ports is connected one of the drives of a backup robot.

THe interfaces I believe they are not ISL. (You mean ISL as a trunking method, right?)
Here it is the config of each interface.

interface fc1/1

  switchport description LTO3 
  port-license acquire
  no shutdown

interface fc1/5

  switchport description LTO3

  port-license acquire

  no shutdown
Here are the details for fc1/1 and fc1/5 that are experiencing the same problem as fc1/7.


#sh int fc1/1 counters  details  fc1/1        4417757720 frames, 5859624695228 bytes received       0 class-2 frames, 0 bytes received       0 class-2 discards       0 F_BSY frames, 0 F_RJT frames           generated against class-2 frames       0 port reject frames       4417757720 class-3 frames, 5859624695228 bytes received       0 class-f frames, 0 bytes received       0 discards, 0 errors received       0 discards, 117 errors transmitted       100604456388 frames, 207142346384892 bytes transmitted       0 class-2 frames, 0 bytes transmitted       100604456388 class-3 frames, 207142346384892 bytes transmitted       0 class-3 frames discarded       0 class-f frames, 0 bytes transmitted       0 class-f frames discarded       0 multicast packets received, 0 transmitted       0 broadcast packets received, 0 transmitted       4417756190 unicast packets received, 100611090558 transmitted       85 link failures, 82 sync losses,       77 signal losses       0 primitive sequence protocol errors       163601 invalid transmission words       0 invalid CRCs, 0 Delimiter Errors       0 address identifier errors       0 link reset protocol errors received       0 link reset protocol errors transmitted       0 Offline Sequence errors received       173 Offline Sequence errors transmitted       0 frames received that are shorter than           the minimum allowable frame length           regardless of the CRC error       0 frames received that are longer than           the maximum frame length and also have a           CRC error       0 waits due to lack of transmit credits       0 frames received with length greater           than what was agreed to in FLOGI/PLOGI       0 frames received with length less than           the minimum indicated by the frame header       0 link reset responses received       0 link reset responses transmitted       0 non-operational sequences received       88 non-operational sequences transmitted       0 fragmented frames received       0 frames received with EOF aborts       0 unknown class frames received       0 8b10b disparity errors       117 frames discarded       0 Exchange Link Parameters switch fabric           internal link service request failures       110542953 transitions of BB credit out of zero state       0 transitions of BB credit to zero state       0 Enhanced Inter Switch Link (EISL) frames           discarded       0 framing errors       0 F8 type LIP sequence errors received       0 F8 type LIP sequence errors issued       193 Non F8 type LIP sequence errors received       173 Non F8 type LIP sequence errors issued  # sh int fc 1/1 transceiver details  fc1/1 sfp is present      Name is CISCO-AVAGO          Manufacturer's part number is SFBR-5780APZ-CS2     Revision is G2.3     Serial number is AGD143284VM          FC Transmitter type is short wave laser w/o OFC (SN)     FC Transmitter supports short distance link length     Transmission medium is multimode laser with 62.5 um aperture (M6)     Supported speeds are - Min speed: 2000 Mb/s, Max speed: 8000 Mb/s     Nominal bit rate is 8500 Mb/s     Link length supported for 50/125mm fiber is 50 m     Link length supported for 62.5/125mm fiber is 20 m     Cisco extended id is unknown (0x0)      No tx fault, no rx loss, in sync state, diagnostic monitoring type is 0x68     SFP Diagnostics Information: ----------------------------------------------------------------------------                                      Alarms                  Warnings                                 High        Low         High          Low ----------------------------------------------------------------------------   Temperature  34.66 C         75.00 C     -5.00 C     70.00 C        0.00 C   Voltage       3.27 V          3.63 V      2.97 V      3.46 V        3.13 V   Current       7.09 mA         8.50 mA     2.00 mA     8.50 mA       2.00 mA   Tx Power     -2.91 dBm        1.70 dBm  -14.00 dBm   -1.30 dBm    -10.00 dBm   Rx Power     -6.76 dBm        3.00 dBm  -17.30 dBm    0.00 dBm    -13.30 dBm   Transmit Fault Count = 0 ----------------------------------------------------------------------------   Note: ++  high-alarm; +  high-warning; --  low-alarm; -  low-warning   # sh int fc1/5 counters  details  fc1/5        4486944237 frames, 5759105445416 bytes received       0 class-2 frames, 0 bytes received       0 class-2 discards       0 F_BSY frames, 0 F_RJT frames           generated against class-2 frames       0 port reject frames       4486944237 class-3 frames, 5759105445416 bytes received       0 class-f frames, 0 bytes received       0 discards, 0 errors received       0 discards, 2 errors transmitted       108664953452 frames, 223758725747704 bytes transmitted       0 class-2 frames, 0 bytes transmitted       108664953452 class-3 frames, 223758725747704 bytes transmitted       0 class-3 frames discarded       0 class-f frames, 0 bytes transmitted       0 class-f frames discarded       0 multicast packets received, 0 transmitted       0 broadcast packets received, 0 transmitted       4486942834 unicast packets received, 108671587794 transmitted       63 link failures, 62 sync losses,       6 signal losses       0 primitive sequence protocol errors       124 invalid transmission words       0 invalid CRCs, 0 Delimiter Errors       0 address identifier errors       0 link reset protocol errors received       0 link reset protocol errors transmitted       0 Offline Sequence errors received       129 Offline Sequence errors transmitted       0 frames received that are shorter than           the minimum allowable frame length           regardless of the CRC error       0 frames received that are longer than           the maximum frame length and also have a           CRC error       0 waits due to lack of transmit credits       0 frames received with length greater           than what was agreed to in FLOGI/PLOGI       0 frames received with length less than           the minimum indicated by the frame header       0 link reset responses received       0 link reset responses transmitted       0 non-operational sequences received       66 non-operational sequences transmitted       0 fragmented frames received       0 frames received with EOF aborts       0 unknown class frames received       0 8b10b disparity errors       2 frames discarded       0 Exchange Link Parameters switch fabric           internal link service request failures       294071266 transitions of BB credit out of zero state       0 transitions of BB credit to zero state       0 Enhanced Inter Switch Link (EISL) frames           discarded       0 framing errors       0 F8 type LIP sequence errors received       0 F8 type LIP sequence errors issued       134 Non F8 type LIP sequence errors received       129 Non F8 type LIP sequence errors issued  # sh int fc 1/5 transceiver details  fc1/5 sfp is present      Name is CISCO-AVAGO          Manufacturer's part number is SFBR-5780APZ-CS2     Revision is G2.3     Serial number is AGD143284VS          FC Transmitter type is short wave laser w/o OFC (SN)     FC Transmitter supports short distance link length     Transmission medium is multimode laser with 62.5 um aperture (M6)     Supported speeds are - Min speed: 2000 Mb/s, Max speed: 8000 Mb/s     Nominal bit rate is 8500 Mb/s     Link length supported for 50/125mm fiber is 50 m     Link length supported for 62.5/125mm fiber is 20 m     Cisco extended id is unknown (0x0)      No tx fault, no rx loss, in sync state, diagnostic monitoring type is 0x68     SFP Diagnostics Information: ----------------------------------------------------------------------------                                      Alarms                  Warnings                                 High        Low         High          Low ----------------------------------------------------------------------------   Temperature  36.53 C         75.00 C     -5.00 C     70.00 C        0.00 C   Voltage       3.26 V          3.63 V      2.97 V      3.46 V        3.13 V   Current       7.08 mA         8.50 mA     2.00 mA     8.50 mA       2.00 mA   Tx Power     -2.72 dBm        1.70 dBm  -14.00 dBm   -1.30 dBm    -10.00 dBm   Rx Power     -3.94 dBm        3.00 dBm  -17.30 dBm    0.00 dBm    -13.30 dBm   Transmit Fault Count = 0 ----------------------------------------------------------------------------   Note: ++  high-alarm; +  high-warning; --  low-alarm; -  low-warning
Highlighted

Krun_shah

Seems that the other post has not a good layout.

Here it is: THe error:

2012 Sep  4 09:34:43 SAN-01 %PORT-2-IF_DOWN_LINK_FAILURE: %$VSAN 2%$ Interface fc1/1 is down (Link failure)  LTO3

2012 Sep  4 09:34:44 SAN-01 %PORT-2-IF_DOWN_LINK_FAILURE: %$VSAN 2%$ Interface fc1/1 is down (Link failure)  LTO3

2012 Sep  4 09:34:44 SAN-01 %PORT-2-IF_DOWN_LINK_FAILURE: %$VSAN 2%$ Interface fc1/5 is down (Link failure)  LTO3

2012 Sep  4 09:35:31 SAN-01 %PORT-2-IF_DOWN_LINK_FAILURE: %$VSAN 2%$ Interface fc1/5 is down (Link failure)  LTO3

2012 Sep  4 09:35:31 SAN-01 %PORT-5-IF_UP: %$VSAN 2%$ Interface fc1/1 is up in mode FL  LTO3

2012 Sep  4 09:35:33 SAN-01 %PORT-5-IF_UP: %$VSAN 2%$ Interface fc1/5 is up in mode FL  LTO3

2012 Sep  4 09:36:57 SAN-01 %PORT-4-IF_SFP_WARNING: Interface fc1/1, Low Rx Power Warning  LTO3

2012 Sep  4 09:36:57 SAN-01 %PORT-4-IF_SFP_WARNING: Interface fc1/5, Low Rx Power Warning  LTO3

2012 Sep  4 09:36:57 SAN-01 %PORT-3-IF_SFP_ALARM: Interface fc1/5, Low Rx Power Alarm  LTO3

2012 Sep  4 09:46:58 SAN-01 %PORT-4-IF_SFP_WARNING: Interface fc1/1, Low Rx Power Warning cleared  LTO3

2012 Sep  4 09:46:58 SAN-01 %PORT-3-IF_SFP_ALARM: Interface fc1/1, Low Rx Power Alarm cleared  LTO3

2012 Sep  4 09:46:58 SAN-01 %PORT-4-IF_SFP_WARNING: Interface fc1/5, Low Rx Power Warning cleared  LTO3

2012 Sep  4 09:46:58 SAN-01 %PORT-3-IF_SFP_ALARM: Interface fc1/5, Low Rx Power Alarm cleared 

----

The fc1/1 and fc1/5 ports are experiencing the same problem as fc1/7 so I am pasting the show command output for those ports.

The fc1/1 and fc1/5 are not ISL ports I believe. You mean ISL as the trunking protocol, right?

Here it is the config for both ports.

interface fc1/1

  switchport description LTO3

  port-license acquire

  no shutdown

interface fc1/5

  switchport description LTO3

  port-license acquire

  no shutdown

WE have connected to those ports a drive that is used by a backup robot.

Here are the ouput for the show commands you asked.

sh int fc1/1 counters  details

fc1/1

      4417757720 frames, 5859624695228 bytes received

      0 class-2 frames, 0 bytes received

      0 class-2 discards

      0 F_BSY frames, 0 F_RJT frames

          generated against class-2 frames

      0 port reject frames

      4417757720 class-3 frames, 5859624695228 bytes received

      0 class-f frames, 0 bytes received

      0 discards, 0 errors received

      0 discards, 117 errors transmitted

      100604456388 frames, 207142346384892 bytes transmitted

      0 class-2 frames, 0 bytes transmitted

      100604456388 class-3 frames, 207142346384892 bytes transmitted

      0 class-3 frames discarded

      0 class-f frames, 0 bytes transmitted

      0 class-f frames discarded

      0 multicast packets received, 0 transmitted

      0 broadcast packets received, 0 transmitted

      4417756190 unicast packets received, 100611090558 transmitted

      85 link failures, 82 sync losses,       77 signal losses

      0 primitive sequence protocol errors

      163601 invalid transmission words

      0 invalid CRCs, 0 Delimiter Errors

      0 address identifier errors

      0 link reset protocol errors received

      0 link reset protocol errors transmitted

      0 Offline Sequence errors received

      173 Offline Sequence errors transmitted

      0 frames received that are shorter than

          the minimum allowable frame length

          regardless of the CRC error

      0 frames received that are longer than

          the maximum frame length and also have a

          CRC error

      0 waits due to lack of transmit credits

      0 frames received with length greater

          than what was agreed to in FLOGI/PLOGI

      0 frames received with length less than

          the minimum indicated by the frame header

      0 link reset responses received

      0 link reset responses transmitted

      0 non-operational sequences received

      88 non-operational sequences transmitted

      0 fragmented frames received

      0 frames received with EOF aborts

      0 unknown class frames received

      0 8b10b disparity errors

      117 frames discarded

      0 Exchange Link Parameters switch fabric

          internal link service request failures

      110542953 transitions of BB credit out of zero state

      0 transitions of BB credit to zero state

      0 Enhanced Inter Switch Link (EISL) frames

          discarded

      0 framing errors

      0 F8 type LIP sequence errors received

      0 F8 type LIP sequence errors issued

      193 Non F8 type LIP sequence errors received

      173 Non F8 type LIP sequence errors issued

SAN-01# sh int fc 1/1 transceiver details

fc1/1 sfp is present

    Name is CISCO-AVAGO    

    Manufacturer's part number is SFBR-5780APZ-CS2

    Revision is G2.3

    Serial number is AGD143284VM    

    FC Transmitter type is short wave laser w/o OFC (SN)

    FC Transmitter supports short distance link length

    Transmission medium is multimode laser with 62.5 um aperture (M6)

    Supported speeds are - Min speed: 2000 Mb/s, Max speed: 8000 Mb/s

    Nominal bit rate is 8500 Mb/s

    Link length supported for 50/125mm fiber is 50 m

    Link length supported for 62.5/125mm fiber is 20 m

    Cisco extended id is unknown (0x0)

    No tx fault, no rx loss, in sync state, diagnostic monitoring type is 0x68

    SFP Diagnostics Information:

----------------------------------------------------------------------------

                                     Alarms                  Warnings

                                High        Low         High          Low

----------------------------------------------------------------------------

  Temperature  34.66 C         75.00 C     -5.00 C     70.00 C        0.00 C

  Voltage       3.27 V          3.63 V      2.97 V      3.46 V        3.13 V

  Current       7.09 mA         8.50 mA     2.00 mA     8.50 mA       2.00 mA

  Tx Power     -2.91 dBm        1.70 dBm  -14.00 dBm   -1.30 dBm    -10.00 dBm

  Rx Power     -6.76 dBm        3.00 dBm  -17.30 dBm    0.00 dBm    -13.30 dBm

  Transmit Fault Count = 0

----------------------------------------------------------------------------

  Note: ++  high-alarm; +  high-warning; --  low-alarm; -  low-warning

SAN-01# sh int fc1/5 counters  details

fc1/5

      4486944237 frames, 5759105445416 bytes received

      0 class-2 frames, 0 bytes received

      0 class-2 discards

      0 F_BSY frames, 0 F_RJT frames

          generated against class-2 frames

      0 port reject frames

      4486944237 class-3 frames, 5759105445416 bytes received

      0 class-f frames, 0 bytes received

      0 discards, 0 errors received

      0 discards, 2 errors transmitted

      108664953452 frames, 223758725747704 bytes transmitted

      0 class-2 frames, 0 bytes transmitted

      108664953452 class-3 frames, 223758725747704 bytes transmitted

      0 class-3 frames discarded

      0 class-f frames, 0 bytes transmitted

      0 class-f frames discarded

      0 multicast packets received, 0 transmitted

      0 broadcast packets received, 0 transmitted

      4486942834 unicast packets received, 108671587794 transmitted

      63 link failures, 62 sync losses,       6 signal losses

      0 primitive sequence protocol errors

      124 invalid transmission words

      0 invalid CRCs, 0 Delimiter Errors

      0 address identifier errors

      0 link reset protocol errors received

      0 link reset protocol errors transmitted

      0 Offline Sequence errors received

      129 Offline Sequence errors transmitted

      0 frames received that are shorter than

          the minimum allowable frame length

          regardless of the CRC error

      0 frames received that are longer than

          the maximum frame length and also have a

          CRC error

      0 waits due to lack of transmit credits

      0 frames received with length greater

          than what was agreed to in FLOGI/PLOGI

      0 frames received with length less than

          the minimum indicated by the frame header

      0 link reset responses received

      0 link reset responses transmitted

      0 non-operational sequences received

      66 non-operational sequences transmitted

      0 fragmented frames received

      0 frames received with EOF aborts

      0 unknown class frames received

      0 8b10b disparity errors

      2 frames discarded

      0 Exchange Link Parameters switch fabric

          internal link service request failures

      294071266 transitions of BB credit out of zero state

      0 transitions of BB credit to zero state

      0 Enhanced Inter Switch Link (EISL) frames

          discarded

      0 framing errors

      0 F8 type LIP sequence errors received

      0 F8 type LIP sequence errors issued

      134 Non F8 type LIP sequence errors received

      129 Non F8 type LIP sequence errors issued

SAN-01# sh int fc 1/5 transceiver details

fc1/5 sfp is present

    Name is CISCO-AVAGO    

    Manufacturer's part number is SFBR-5780APZ-CS2

    Revision is G2.3

    Serial number is AGD143284VS    

    FC Transmitter type is short wave laser w/o OFC (SN)

    FC Transmitter supports short distance link length

    Transmission medium is multimode laser with 62.5 um aperture (M6)

    Supported speeds are - Min speed: 2000 Mb/s, Max speed: 8000 Mb/s

    Nominal bit rate is 8500 Mb/s

    Link length supported for 50/125mm fiber is 50 m

    Link length supported for 62.5/125mm fiber is 20 m

    Cisco extended id is unknown (0x0)

    No tx fault, no rx loss, in sync state, diagnostic monitoring type is 0x68

    SFP Diagnostics Information:

----------------------------------------------------------------------------

                                     Alarms                  Warnings

                                High        Low         High          Low

----------------------------------------------------------------------------

  Temperature  36.53 C         75.00 C     -5.00 C     70.00 C        0.00 C

  Voltage       3.26 V          3.63 V      2.97 V      3.46 V        3.13 V

  Current       7.08 mA         8.50 mA     2.00 mA     8.50 mA       2.00 mA

  Tx Power     -2.72 dBm        1.70 dBm  -14.00 dBm   -1.30 dBm    -10.00 dBm

  Rx Power     -3.94 dBm        3.00 dBm  -17.30 dBm    0.00 dBm    -13.30 dBm

  Transmit Fault Count = 0

----------------------------------------------------------------------------

  Note: ++  high-alarm; +  high-warning; --  low-alarm; -  low-warning

thanks in advance,

Fernando

Highlighted

do you have other tape drives that are connected to this switch and are working correctly ? If yes, can you move them to these ports and move the other drives to "working" ports and see if the problem follows those specific tape drives.

@dynamoxxx
Highlighted

Hi dynamoxxx,

From port fc1/1 to f1/16 we have connected different things, servers, drives and other devices. And all ports are experiencing the same problem.

Thanks!

Fernando

Highlighted

i would not think that 6 SFPs would go bad but have you tried to swap those from other know "working" ports ? Also are these direct connects or you are going through a patch panel of some sort ?

@dynamoxxx
Highlighted

Hi Dynamoxx,

Sorry for the late response, but I was sure that I have answered the question you asked previously.

Yes, fibers go through a patch panel. Above, I pasted the output of "show fc1/1 transceiver detail" and the output I got was the following:

SAN-01# sh int fc 1/5 transceiver details

fc1/5 sfp is present

    Name is CISCO-AVAGO    

    Manufacturer's part number is SFBR-5780APZ-CS2

    Revision is G2.3

    Serial number is AGD143284VS    

    FC Transmitter type is short wave laser w/o OFC (SN)

    FC Transmitter supports short distance link length

    Transmission medium is multimode laser with 62.5 um aperture (M6)*********************************************

    Supported speeds are - Min speed: 2000 Mb/s, Max speed: 8000 Mb/s

    Nominal bit rate is 8500 Mb/s******************************************************************************************

    Link length supported for 50/125mm fiber is 50 m

    Link length supported for 62.5/125mm fiber is 20 m

    Cisco extended id is unknown (0x0)

As you can see, it says that the transmission medium is multimode laser with 62.5 um and the nominal rate is 8500 Mb/s.

Is it possible that the transceiver is able to detect what type of fiber is it connected? Is that a default value? May I change that value from the command line?

Besides, the nominal rate, what does exaclty mean and is it possible to modify it?

the GBIC model is DS-SFP-FC8G-SW

thanks in advance for your help,

Fernando

Highlighted

Fernando,

I'm having the same problem, did you solved your?

Thanks in advance,

Roberto

Highlighted

Roberto,

No, some tests still need to be done, but unfortunately, the problem still exists. If I find the solution, I will let you know ASAP.

thanks,

Fernando

Highlighted

Fernando,

Thank you for your prompt reply. I have one more question, have you tried to upgrade the firmware to the 5.2 version?

Regards,

Roberto

Highlighted
Beginner

This is a known software problem (no functional impact). The alarm displays a few minutes after the interface comes up, then clears about 10 minutes later. Solved in NX-OS 4.2(7e) or 5.0.7 and later (CSCtk10527)...

--filiph

Highlighted

Hi Filiph,

thanks for your response. I will check if the upgrade is possible and then let you know the results.

Thanks!

Fernando

Highlighted

Just to clarify....

you still need to investigate why ports go offline with LINK FAILURE as the cause (check logs on the LTO device and the physical connection).

The following "Low Rx Power Warning" is a cosmetic issue that happens when link goes online again.

--filiph

Highlighted
Beginner

Hi All,

I'm having the same problem currently using NX-OS Version 5.1(3)N1(1a) why this is updated version but same problem any one help.

2013 Jun  9 14:05:50 NXS01 %PORT-4-IF_SFP_WARNING: Interface fc2/4, Low Rx Power Warning

2013 Jun  9 14:05:50 NXS01 %PORT-3-IF_SFP_ALARM: Interface fc2/4, Low Rx Power Alarm