2960S Stack, 15.0(2)SE3 & TACACS

[lekeosi11]

Hi

I upgraded a Cisco 2960s stack from 15.0(2)SE2 to 15.0(2)SE3 yesterday.

The switch stack is set to use TACACS for authentication.

Since then, I'm no longer able loging to the switch using ssh or http.

I start a SSH session, enter my username and immediatley I get Access Denied (3 times and the switch drops the connection).

I can't see any tacacs packets being sent from the switch to the ACS server.

The release notes for 15.0(2)SE3 do not indicate any issues with Tacacs.

Any ideas?

Thanks

L

[Leo Laohoo]

Hi Simen,

I've got fleets of 3750/G/E/X running 12.2(55)SE7 and I don't see this behaviour, same config that triggers the bug in 15.0(2)SE3.

[Simen Ringstad]

Maybe the behaviour is due to the fact that our ACS isn't fully operational at the moment and the switch has a problem using the local userdatabase.

If I enter the following config via SNMP I can login in to the switches running 12.2(55)SE7 using a local user:

aaa authentication login default local

aaa authorization exec default local

#sh processes cpu | inc TPLUS

225   256049282  10174479      25165 93.92% 93.95% 93.88%   0 TPLUS

Switch Ports Model              SW Version            SW Image

------ ----- -----              ----------            ----------

*    1 26    WS-C2960-24TT-L    12.2(55)SE7           C2960-LANBASEK9-M

[Scott Plank]

Just ran into this bug after downloading 15.0(2)SE3 a couple of days ago. Fortunately, I only had it deployed on a test switch that I was testing the new, non-deprecated TACACS commands on. Just wondering how this even made it into a release? I know it's not possible to catch every bug, but TACACS is such a basic setup on most devices, I'm surprised it was not caught earlier, or at least identified. Were there other bug fixes in 15.0(2)SE3 that were deemed more important than this one, therefore it was released anyway, knowing this bug was there?

[michelpe]

Scott,

We werent aware of this bug before releasing 15.0(2)SE3 or we would have fixed it before releasing it.

The bug was caused by another bug that was commited at a late moment in the release cycl after we

done the full testing on AAA.

We are looking into how this managed to get through without being detected and will improve where

needed to prevent it from re-occuring

Thanks

Michel

[Leo Laohoo]

Scott,

I just checked, 15.0(2)SE3 has been pulled from the download site. 

[Scott Plank]

Michel -

Good to know, thanks for the response.

Leo -

I saw that yesterday when I downloaded 15.0(2)SE2 to downgrade. Good to see it's no longer available. I went ahead and contacted my local Cisco team & Cisco partner about this issue in case they had other customers who managed to get a hold of the code before it was taken down.

[Todd Volz]

We have this same problem on a brand new stack of 3750X. 

Another workaround is to add a local username and password on the device with the correct TACACS configuration to allow local authentication if TACACS servers are unavailable.

-Todd

[Leo Laohoo]

To all those who are watching this thread, 15.0(2)SE4 is now ready for download.

[Brian Schultz]

It appears this bug is still a problem on 15.0(2)SE5.  I have 2960's running this image with just a couple VLAN's and TACACS that are working just fine.  However, the same model switch (WS-C2960-48PST-L) also running 802.1x and I encounter the low on memory condition and can no longer access the switch via SSH or console. 

Has anyone found a workaround to resolve this issue?  I haven't tried the tacacs single-connection command yet, but intend to do so when I have a maintenance window to reboot the switch.

Thanks,

Brian

[Leo Laohoo]

Hi Brian,

Sorry for the delay.  My mind is still in "vacation" mode.

It appears this bug is still a problem on 15.0(2)SE5

Believe it or not, I refuse to run 15.0(2)SE5 after this code failed in my first stage of testing. 

All my 2960/G/S are running happily on 15.0(2)SE4.  No tracebacks, no crashes.  Nothing. 

All my 3750/G/E/X are running 12.2(55)SE8.

I won't touch 15.0(2)SE5, 15.2(1)E or 15.2(1)E1.

[Steven Coutts]

Hi all

We are seeing the same behaviour on 2960S, 12.2(55)SE7.

Also "C3560 Software (C3560-IPBASEK9-M), Version 12.2(55)SE8" may also be affected.

[Leo Laohoo]

Hmmmm ... I've used 12.2(55)SE7 for a number of months.  I have never seen a behaviour like this before.

[Steven Coutts]

I have a couple here now with dead SSH, pushing some config to remove TACACS via SNMP, then a reboot will bring them back.

[Christoph Faber]

I'd rather suggest you configured something wrong. I'd guess your tacacs not reachable and you don't have a fallback configured.

I have a lot of 2960 / 3560 running with  12.2(55)SE7 and SE8 - no one ever had a problem with tacacs.

[Steven Coutts]

We have not configured anything wrong, the same config with a downgraded IOS and we do not have this problem anymore.

They are sitting with 100% CPU currently, this is a bug, not incorrect config.

Also Simen seems to have seen the issue on 2960 as well.