Cisco 9300 won't recognize sfps after reboot

[kevinfowler]

So I installed a 48 port cisco 9300 and all the sfps that we inserted into the slots were recognized.    I had to reboot the switch and now ports 33-48 don't recognize any SFPs we place in the slots.  Any suggestions?

[marce1000]

 - Upgrade to the latest advisory software release for the particular model (if applicable)  , check if that can help , 

 M.

[kevinfowler]

Actually I'm on gilbralter 16.12.8 on that particular device.   

[marce1000]

  - It would be advisable to power cycle the switch with cords unplugged and see if the problem then remains on future ('warm reboots')  , 

 M.

[kevinfowler]

Thats interesting.   When I rebooted the switch is when we starting having the issue, before that it was working just fine. 

[MHM Cisco World]

The config of sfp is remove when you reboot 

As I know you need eem for this case.

[kevinfowler]

So this morning I tried all of the SFP's in a different switch and had no problems with them being recognized.  That switch is on Gilbralter 16.12.8 which is the same IOS on the "bad" switch.  I went back to that switch and tried some of the ideas I read such as defaulting the port back to it's orignial state.....still nothing.   I took the SFP's and placed them in the 10G uplink module and they all worked.   Any other possible suggestions?   I'm going to do a reboot when I get permission but I'm hoping that doesn't break the rest of the switch since after the past reboot I lost all of the ports mentioned above.   Any suggestions?  

[MHM Cisco World]

dont worry I will check and share with you some point to check.

[MHM Cisco World]

show inventory <<- please share output here 

[Leo Laohoo]

This is a known issue/behaviour with IOS-XE in 16.12.X. 

The workaround is to cold reboot the switch member or upgrade the firmware.

[kevinfowler]

Leo,

We used gibralter 16.12.x on all of our ethernet 9300's, I assumed that that IOS-XE would work perfectly fine for the 9300 SFP switches.   Is that not true?   Should I be on a different flavor of IOS-XE all together?  Amsterdam?   

The fact that all the sfp's are working in the same model 9300 with the same IOS-XE is causing my confusion.   I mispoke earlier, the SFP's were not recognizable before I rebooted the switch.   The port hosting the printer went dead before that so would this be a hardware issue?   That same SFP works everywhere else.  

[Leo Laohoo]

IOS-XE version 16.12.X, regardless if it is a 3650/3850 or 9k, is a trainwreck.  It is one of the buggiest train Cisco has published.  

There are several bugs OIR and it stems down to the process that constantly polls the optic ports.  When it crashes, it crashes spectacularly.  Reloading the offending switch member does not work all the time.  The best workaround (NOTE:  It is still a WORKAROUND) is to remove the power supplies of the offending switch for about 30 seconds before energizing it back.   

Aside from OIR bugs (plural), there are numerous bugs relating to PoE, "ghost" ports (nothing is connected to the port but the logs show the port going down/up constantly) and many, many more.  

For the time being, consider upgrading to 17.3.5 or 17.6.4.  We have been using these two versions for about 6 months and they seem tolerable.  

[johnlloyd_13]

yikes! what a headache

[kevinfowler]

OK I have upgraded to 17.3.5 and 17.3.6 and now I eithewr get the SFP's not recognized or the ports err-disable.   And a side note this is on another 9300.   This is a mess.  I need to be able to update our switches IOS's when needed and I can't even get this to work.   Any suggestions?   We have to use the Cisco 9300-48S and 24S because we have fiber to the desktop.  

[MHM Cisco World]

sorry are this  issue solved after upgrade to 17.3.x ??