06-07-2013 08:34 AM - edited 03-07-2019 01:46 PM
Hi
I upgraded a Cisco 2960s stack from 15.0(2)SE2 to 15.0(2)SE3 yesterday.
The switch stack is set to use TACACS for authentication.
Since then, I'm no longer able loging to the switch using ssh or http.
I start a SSH session, enter my username and immediatley I get Access Denied (3 times and the switch drops the connection).
I can't see any tacacs packets being sent from the switch to the ACS server.
The release notes for 15.0(2)SE3 do not indicate any issues with Tacacs.
Any ideas?
Thanks
L
Solved! Go to Solution.
06-12-2013 01:18 AM
Leo,
we are currently working on a plan how to address this issue. Removing the image
from CCO is one possibilty we looking at
Thanks
Michel
06-12-2013 02:02 AM
We do have a "low on memory" on console without TACACS (without anything connected to the switch) => back to 12.2(55)SE7
06-12-2013 03:30 PM
Removing the image from CCO is one possibilty we looking at
Thanks Michel.
06-12-2013 07:59 PM
I have another work around my coworkers and I used to get back in our 3750 we were testing this upgrade on using SNMP. If it's configured it's possible to do the following:
Prepare a file on a tftp server, call it what you want. Inside we used something like:
aaa authentication login default local
aaa authorization exec default local
username recover password this
end
Push the above configuration to the device by setting the necessary values and then activating. This command should do all that in one, just edit the IP address and th line below it to match the filename we created above on the tftp server.
snmpset -v 2c -c private Device .1.3.6.1.4.1.9.9.96.1.1.1.1.2.50 i 1 \
.1.3.6.1.4.1.9.9.96.1.1.1.1.3.50 i 4 \
.1.3.6.1.4.1.9.9.96.1.1.1.1.4.50 i 1 \
.1.3.6.1.4.1.9.9.96.1.1.1.1.5.50 a "10.0.0.2" \
.1.3.6.1.4.1.9.9.96.1.1.1.1.6.50 s "Router.cfg" \
.1.3.6.1.4.1.9.9.96.1.1.1.1.14.50 i 4 \
Check on the status using the following command, just make sure to change the host and community string:
snmpwalk -v 2c -c private 10.0.0.1 .1.3.6.1.4.1.9.9.96.1.1.1.1.10.50
Possible integer responses to this above command are waiting(1), running(2), successful(3), failed(4). If it returns 1 or 2, just keep trying until it reaches 3 or 4. If it times out, keep trying.
When done, destroy the row:
snmpset -v 2c -c private 10.0.0.1 .1.3.6.1.4.1.9.9.96.1.1.1.1.14.50 i 6
At this point we were able to log in and downgrade without leaving our seats. Of course, this does require SNMP with RW to be configured and it may time out occasionally due to the CPU utilization being high. Hope this helps some others.
06-27-2014 03:17 PM
This snmp procedure saved me from a last minute reload and IOS change. Thank you !
I was also lucky to have a RW SNMP community on the switch so that saved me a lot of trouble.
I had to do this on a production 3750X-48T-S using Kaly with atftp. Very easy once I knew what each parameter meant. I used the URL below for reference on the parameters.
Now I am just waiting to proceed with upgrading to 15.0(2)SE6 in the hopes that this version doesn't have the TACACS problem. Has anyone heard of this version being stable ? This is the version recommended on the download site.
06-12-2013 06:41 AM
Hello michelpe,
Tried you suggestions.
Neither with the old nor new cli, single-connection gave satisfying results.
Indeed, the TPLUS-Process didn't go up to 100%, but I couldn't login.
The Tacacs-Server had an Log-Entry by the switch, but the Switch went to the fallback-Method and both the local and the tacacs-password got the answer
% Authentication failed
So single-connection is no usable workaround.
06-12-2013 08:55 AM
Christoph,
Our concern team is working on this and this should get resolved soon.
Untill then we request you to downgrade the firmware and we should get back to you once this is ready and have fix for all the issue.
Sorry for inconvience.
HTH
Regards
Inayath
06-13-2013 04:35 AM
Yes, you are correct, unfortuantly the workaround isnt shown to be very reliable. So its not a very good workaround.
We are working to get a fix out for the issue as soon as we can
06-13-2013 06:10 PM
To all, please refer to CSCug62154.
NOTE: I disagree with the information of "Telnet to the router is not possible. On the console any command issued would take lot of time" --- Telnet, SSH and console doesn't work. Period.
06-16-2013 06:23 AM
I also have run into this with 2960, 3560 and 3560-CG which now all are completely unaccessable. It will take plenty of time an effort to recover from this. As far as I know there is no way to do that without booting into rommon and cause service disruption.
What is very astonishing is that these faulty images still are downloadable. I checked release- notes - nothing about this severe bug - and downloaded the relevant images yesterday, scheduled a reboot during late night, and today the switches are unaccessable. Still there are not even a line of caution about this on cisco.com.
06-16-2013 11:25 AM
We are in the process of removing the image from CCO. Unfortunatly this does take a bit of time.
The release notes are updated since last friday .
Unable to login to switch and shows high CPU utilization when TACACS is used for authentication.
The workaround is to downgrade the switch to Cisco IOS Release 15.0(2)SE2 or not to use TACACS.
An updated IOS versio n(15.0(2)SE4) is targetted to be on CCO the 28th of June. Earlier if possible
06-16-2013 02:28 PM
We are in the process of removing the image from CCO.
Michel,
I know process take some time, but could I also recommend that a WARNING PAGE be inserted as a temporary measure?
An updated IOS versio n(15.0(2)SE4) is targetted to be on CCO the 28th of June.
Wow. That's a long time away.
06-29-2013 04:26 PM
An updated IOS versio n(15.0(2)SE4) is targetted to be on CCO the 28th of June. Earlier if possible
Hi Michel,
Is the release date still on schedule? Can you provide sufferers with any update, please?
06-17-2013 06:05 AM
I think I might have a similar issue with 12.2(55)SE7 on Cat296024TT. After upgrading from 12.2(55)SE5 I am no longer able to log in with SSH. After entering username and displaying login-banner the login hangs.
I can access the switch via console and downgraded back to SE5 and it works fine again.
06-17-2013 07:17 AM
I did test it in 12.2(55)SE7 on a 3750 and dont see a problem.
Could you send me your config used. michelpe@cisco.com
3k_1#sh tacacs
Tacacs+ Server - public : 10.48.91.201/49
Socket opens: 4
Socket closes: 4
Socket aborts: 0
Socket errors: 0
Socket Timeouts: 0
Failed Connect Attempts: 0
Total Packets Sent: 12
Total Packets Recv: 12
3k_1#sh ver | inc IOS
Cisco IOS Software, C3750 Software (C3750-IPSERVICESK9-M), Version 12.2(55)SE7, RELEASE SOFTWARE (fc1)
3k_1#telnet 10.1.24.181
Trying 10.1.24.181 ... Open
Test Switch
Username: michelpe
Password:
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide