%ILET-1-AUTHENTICATION_FAIL on 2960X after IOS Update

[Heinz Schwarzfeuer]

Hi,

after an IOS update from version c2960x-universalk9-tar.152-2.E6 to c2960x-universalk9-tar.152-4.E6 we are getting the message

%ILET-1-AUTHENTICATION_FAIL: This Switch may not have been manufactured by Cisco or with Cisco's authorization.  This product may contain software that was copied in violation of Cisco's license terms.  If your use of this product is the cause of a support issue, Cisco may deny operation of the product, support under your warranty or under a Cisco technical support program such as Smartnet.  Please contact Cisco's Technical Assistance Center for more information.

every 5 minutes on the console of the 2960X-48-LPS-L switch.

Switch#show license

Index 1 Feature: lanlite

        Period left: 0  minute  0  second

Index 2 Feature: lanbase

        Period left: 0  minute  0  second

We´ve downgraded and upgraded it already to several different versions (and always rebooted it by disconnecting power for at least 5 minutes, neither stacking module nor SFP modules present), but the error message still remains.

Any suggestions?

Many thanks!

Heinz

[trust1-it]

Thanks for the quick answer. Actually, the log refers to the switch not the stack module:

*Mar 1 00:00:39.153: Read env variable - LICENSE_BOOT_LEVEL =
*Jan 2 00:00:00.076: %IOS_LICENSE_IMAGE_APPLICATION-6-LICENSE_LEVEL: Module name = c2960x_lanbase Next reboot level = lanbase and License = No valid license found
Feb 4 19:46:28.178: Flexstack module is 1
Feb 4 19:47:14.972: %STACKMGR-4-SWITCH_ADDED: Switch 1 has been ADDED to the stack
Feb 4 19:47:16.070: %SMI-5-CLIENT: Smart Install Client feature is enabled. It is recommended to disable the Smart Install feature when it is not actively used. To disable feature execute 'no vstack' in configuration mode
Feb 4 19:47:16.699: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
Feb 4 19:47:16.699: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0, changed state to down
Feb 4 19:47:18.069: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
Feb 4 19:47:25.818: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan140, changed state to down
Feb 4 19:47:25.849: %SW_VLAN-6-VTP_DOMAIN_NAME_CHG: VTP domain name changed to gci04-fsw01.
Feb 4 19:47:25.933: %SYS-5-CONFIG_I: Configured from memory by console
Feb 4 19:47:26.038: %STACKMGR-5-SWITCH_READY: Switch 1 is READY
Feb 4 19:47:26.038: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 1 has changed to state DOWN
Feb 4 19:47:26.038: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state DOWN
Feb 4 19:47:26.465: %USB_CONSOLE-6-MEDIA_RJ45: Console media-type is RJ45.
Feb 4 19:47:26.552: %STACKMGR-5-MASTER_READY: Master Switch 1 is READY
Feb 4 19:47:26.555: %PLATFORM-6-FLEXSTACK_INSERTED: FlexStack module inserted in Switch 1.
Feb 4 19:47:26.839: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C2960X Software (C2960X-UNIVERSALK9-M), Version 15.2(4)E6, RELEASE SOFTWARE (fc4)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2018 by Cisco Systems, Inc.
Compiled Thu 05-Apr-18 02:53 by prod_rel_team
Feb 4 19:47:26.922: %SSH-5-ENABLED: SSH 1.99 has been enabled
Feb 4 19:47:27.873: %ILET-1-AUTHENTICATION_FAIL: This Switch may not have been manufactured by Cisco or with Cisco's authorization. This product may contain software that was copied in violation of Cisco's license terms. If your use of this product is the cause of a support issue, Cisco may deny operation of the product, support under your warranty or under a Cisco technical support program such as Smartnet. Please contact Cisco's Technical Assistance Center for more information.
Feb 4 19:47:28.499: %LINK-3-UPDOWN: Interface FastEthernet0, changed state to down
Feb 4 19:47:32.941: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/1, changed state to up

[swapsakker]

u need version 152-4.E7 and remove the power for 1min, that will solve it

[Steveosh72]

does not solve the issue.  left the power unplugged over night and still getting the message

[Leo Laohoo]

Raise a TAC Case.

[scott.mickelsonAMS]

We had this issue after upgrading to 15.2(5b)E yesterday.
Cisco had the gumption to pull our licensing for a product that had been in service for 3 years. Total system down for 30+ hours at this point. We're sitting here waiting for the BU team to decide if we're criminals or not, while the customer remains with 192 unusable ports on $10,000 worth of paid hardware with support... They literally pull the license for the port to send traffic. I've been waiting for several hours now and our Onsite team had to go home for the night. We have registered these serials and hardware with Cisco for the last 3 years... Remember - We ran these switches without issue on the older firmware for 3 years and decided to bite the bullet and upgrade. I am now convinced that Cisco doesn't give a dang about their clients ability to do business. It is 100% about the profit margin of the sale. I'll remember how I was treated the next client we engage with. I've lost all credibility in Cisco's ability to care for customers. No 2960X switches going forward - PERIOD.

[scott.willett@simfoods.com]

Try rolling back to the original OS like we did. It fixed it but now we are replacing it since its end of life. Sucks to do but it restores the outage.

[scott.mickelsonAMS]

Thanks Scott.  Does it require the exact original OS, or is there a certain level that we can go back to that does not present the issue? We were on 15.2.4(E1) before.

Thanks in advance.

[scott.willett@simfoods.com]

Im not certain which OS will be clear of the ILET issue and which ones will
not. I would rollback to the original OS and confirm the issue's cleared
and go form there. You may find that there is not any other OSs to resolve
your problem. If your box has Cisco support, first and foremost, engage
Cisco-TAC. Good luck man.

[jedavis]

Rolling back to the original version doesn't clear it.  Once it appears the switch is a brick.  It isn't a software issue.

[cichelson1]

I am receiving a similar issue.  Seems that the issues exists on 2960's with a vo5 manufacture ID.

AUTHENTICATION_FAIL: This Switch may not have been manufactured by Cisco or with Cisco's authorization.  This product may contain software that was copied in violation of Cisco's license terms.  If your use of this product is the cause of a support issue, Cisco may deny operation of the product, support under your warranty or under a Cisco technical support program such as Smartnet.  Please contact Cisco's Technical Assistance Center for more information.

Roll back does not solve the issue.  The switch turns to a BRICK.

This was on the version before :  c2960x-universalk9-mz.152-7.E0a.bin

We are going to consider upgrading to here and see if this solves the issue.  Anyone else have any feedback?

[Leo Laohoo]

As I've said in other posts, this is a well-known issue affecting 2960X and 6840. This is not an IOS bug issue. This is a hardware component issue.
Don't even bother organizing TAC for troubleshooting because, at the end of the day, the success rate (workaround) is very low and wastes your time.
The only way is to RMA the appliance.

[usanitary]

Same, this affected 2 of our switches, and on a P1 with 90 users down, they made me fill out a form to prove my company is not a criminal organization.. Bad move cisco

[Leo Laohoo]

Cisco is no longer "interested" in fixing this.  They are "keen" on everyone upgrading to the Catalyst 9k switches.  

I have long suspected the issue is a hardware design fault in one of the component.  After prolong use/uptime the component would "crash" and start spitting out the error.  

This behaviour is exactly like the two (public facing) Cisco MOSFET bugs that is "exclusive" to the 3650/3850 platform (CSCvd46008 & CSCvj76259).  And, like  CSCur56395, the MOSFET bugs kick in only after prolong use.  

The workaround of cold reboot is just it.  A workaround and a game of Russian Roulette.  There is no guarantee a cold reboot will fix it.  And there is definitely no guarantee it won't come back either. 
The only guaranteed way to "fix" this issue is to RMA the appliance.   

[jedavis]

Yes, but do not despair that the 9k switches will make you obsolete!  They come with an included "stack cable authentication failure" Cisco DOS attack.  We experience more downtime caused by Cisco trying to make sure we are using only Cisco manufactured products than for any other reason.  And it is never because we are not using Cisco manufactured products, it is because the code that attempts to detect it is bad, or the hardware is manufactured by Cisco but has some sort of defect that causes an authentication failure.  

[usanitary]

I experienced this problem with 2 out of 10 "WS-C2960X-48TS-L", after upgrading to  15.2(7)E4. Both had SN FOC19XXXXXX, maybe a bad batch? 

Thank you all for this discussion, the cold restart brought me up. I still raised RMAs, but at least my users are able to work for now.