cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1375
Views
10
Helpful
5
Replies

3650 stack ports not active

Redbeanie
Level 1
Level 1

We have a stack of 7 c3650-48TD.

There are an increasing number of ports that when they have been inactive for a while, refuse to come back up when a client is connected.

On some suspected faulty ports, the indicator light on the front of the switch goes green, but there is no traffic flowing.

On others the light will go amber suggesting they are blocked by STP. However, looking at the state of that specific port, it doesn’t not show it to be in an err-disabled state.

 

I noticed these error messages appear on the console while unplugging, then plugging in, a cable to a suspect port:

 

*Apr 20 14:52:28.543: %SPI_FC-3-HIGH_WMARK_REACHED: Number of messages in the queue for channel 1 has reached maximum.

-Traceback= 1#58bc7daf4fe087af80f5f05af9e15bb7  :545E8000+1FB56C4 :545E8000+12B3A38 :545E8000+12B3C2C :545E8000+B17AD8 :545E8000+B198BC :545E8000+B1BE38 :545E8000+CC8C64 :545E8000+CD9230 :545E8000+9CF3C8 :545E8000+B0F390 :545E8000+B18C58

*Apr 20 14:52:28.554: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_PROTECTED, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:52:28.554: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_UK_MCAST_BLK, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:52:28.555: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_UK_UCAST_BLK, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:52:28.556: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:VP_STP_STATE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:52:29.553: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/10, changed state to down

*Apr 20 14:52:30.553: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/10, changed state to down

*Apr 20 14:52:30.554: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_ARP_SNOOP, Targets: 0x4, spi-fc ec: 3

stack#

stack#

stack#

*Apr 20 14:53:08.315: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_MODE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.316: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_DTP_MODE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.316: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_MODE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.316: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_DTP_MODE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.316: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_MODE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.316: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_PAGP_MODE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.317: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_PAGP_MODE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.317: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_PROTECTED, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.317: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_UK_MCAST_BLK, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.317: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_UK_UCAST_BLK, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.317: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_MODE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.318: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:VP_STP_STATE, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:08.320: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:E_PORT_SET_ALL_VLANS, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:10.320: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/10, changed state to up

*Apr 20 14:53:10.321: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:PORT_ARP_SNOOP, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:11.324: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/10, changed state to up

*Apr 20 14:53:12.320: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:E_PORT_SET_ALL_VLANS, Targets: 0x4, spi-fc ec: 3

*Apr 20 14:53:16.311: %PLATFORM_PM-0-SPI_FC_FAIL: SPI-FC Failure while sending PM SPI msg. If:Gi2/0/10, Msg Type:E_PORT_SET_ALL_VLANS, Targets: 0x4, spi-fc ec: 3

stack#

....

...

 

This stack is still running on software version 3.03.04SE, and from my quick research, this may be related to a bug in this version?

But am hoping there is another solution that we have missed which will avoid us having to reboot the stack

5 Replies 5

balaji.bandi
Hall of Fame
Hall of Fame

Can you post show switch information, please confirm is there any stack cable open ? how Long the switch running (uptime ?)

 

Looks for me MOSFET Bug   - Try to upgrade to 16.X and see if that fixes the issue.

 

 

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

stack uptime is 5 years, 28 weeks
Uptime for this control processor is 5 years, 28 weeks

 

------------------ show switch detail ------------------

  

Switch/Stack Mac Address : xxxx.xxxx.xxxx     - Local Mac Address

Mac persistency wait time: Indefinite

                                             H/W   Current

Switch#   Role    Mac Address     Priority Version  State

------------------------------------------------------------

*1       Active   xxxx.xxxx.xxxx     15     V01     Ready              

 2       Standby  xxxx.xxxx.xxxx     1      V01     Ready              

 3       Member   xxxx.xxxx.xxxx     1      V01     Ready              

 4       Member   xxxx.xxxx.xxxx     1      V01     Ready              

 5       Member   xxxx.xxxx.xxxx     1      V01     Ready              

 6       Member   xxxx.xxxx.xxxx     1      V02     Ready              

 7       Member   xxxx.xxxx.xxxx     1      V02     Ready              

 

          Stack Port Status             Neighbors    

Switch#  Port 1     Port 2           Port 1   Port 2

--------------------------------------------------------

  1         OK         OK               7        2

  2         OK         OK               1        3

  3         OK         OK               2        4

  4         OK         OK               3        5

  5         OK         OK               4        6

  6         OK         OK               5        7

  7         OK         OK               6        1

 

First i would suggest here is - take maintenance and reboot the switch and see if the problem resolves.

 

when did the last IOS Update took place ?

 

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Leo Laohoo
Hall of Fame
Hall of Fame

Upgrade the firmware of the switch or reboot.  It could be CSCuh78943.

Hi

Please can you give more information regarding the impact of this bug.

I am unable to view any information on it.

Thanks

Review Cisco Networking for a $25 gift card