01-25-2014 12:35 AM - last edited on 03-25-2019 05:31 PM by ciscomoderator
I ran into a weird issue tonight when upgrading some 4948E switches. I upgraded 4 to 15.1(2)SG - cat4500e-ipbasek9-mz.151-2.SG.bin.
2 of them directly connect to firewalls so management stayed on SVIs but the other two were going to be migrated to use VRFs. I made the VRF change including AAA on the one of them and couldn't login to TACACS. Then noticed I couldn't login to the other one either, even though it was still using the SVI.
Checking the debug logs on my TACACS server I noticed that my username "john.smith" was being sent as "john_smith" so it failed auth. We have an LDAP user account for "rancid" and that was fine. So for some reason these two switches were changing the underscore to a period in the username, causing the LDAP authentication to fail.
I checked the configs for these two 4948E's against other switches, same model, same software version. Made the AAA/VRF configs exactly the same - still had this issue. The four I was upgrading tonight do point to a different TACACS server, but only minor things changed in the config, like the key and accounting file.
Rembering how NX-OS will complain about invalid operator strings, and not knowing what else to do, I changed this line in my tac_plus.conf file:
optional shell:roles = "\"operations network network-admin\""
to this:
optional shell:roles = "\"network-admin\""
And viola! It fixed the issue.
Anyone know how that would've caused the issue? That same line is in my other TACACS server. Same switch model, some software, same config. Only difference is IP addresses used and network layout. In the issue tonight, it was a private IP environment and on same subnet as the TACACS server but I don't see how that would have any effect.
Also I did debugs on the switch, and the switch itself was sending the underscore in the username.
01-26-2014 11:45 AM
Can you please collect the console output after enabling "debug tacacs" and "debug aaa authentication" and "debug aaa authorization" and post the results here?
Javier Henderson
Cisco Systems
01-26-2014 12:34 PM
I will see about reverting that tacacs config statement and getting that debug output. Also, I made a typo in my initial post. The *period* in the username was being replaced with an underscore. Here are some tacacs debug logs I have right now showing the issue happening.
Fri Jan 24 23:06:51 2014 [19139]: login query for 'john.smith' tty1 from 172.16.1.200 accepted
Fri Jan 24 23:06:51 2014 [19150]: authorization query for 'john.smith' tty1 from 172.16.1.200 accepted
Fri Jan 24 23:13:45 2014 [20053]: login query for 'john.smith' tty0 from 172.16.1.200 accepted
Fri Jan 24 23:13:54 2014 [20122]: enable query for 'john.smith' tty0 from 172.16.1.200 accepted
Fri Jan 24 23:21:12 2014 [21117]: login query for 'john_smith' tty1 from 172.16.1.200 rejected
Fri Jan 24 23:21:17 2014 [21130]: login query for 'john_smith' tty1 from 172.16.1.200 rejected
Fri Jan 24 23:34:10 2014 [22948]: login query for 'john.smith' tty0 from 172.16.1.210 accepted
Fri Jan 24 23:34:13 2014 [22976]: enable query for 'john.smith' tty0 from 172.16.1.210 accepted
Fri Jan 24 23:34:47 2014 [23049]: login query for 'john_smith' tty2 from 172.16.1.200 rejected
Fri Jan 24 23:35:40 2014 [23070]: login query for 'john_smith' tty2 from 172.16.1.200 rejected
Fri Jan 24 23:36:21 2014 [23271]: login query for 'john_smith' tty1 from 172.16.1.210 rejected
Fri Jan 24 23:36:39 2014 [23325]: login query for 'john.smith' tty2 from 172.16.1.210 accepted
Fri Jan 24 23:36:39 2014 [23336]: authorization query for 'john.smith' tty2 from 172.16.1.210 accepted
Fri Jan 24 23:38:16 2014 [23540]: login query for 'john_smith' tty1 from 172.16.1.200 rejected
Fri Jan 24 23:39:44 2014 [23749]: login query for 'john_smith' tty1 from 172.16.1.200 rejected
Sat Jan 25 00:06:44 2014 [27449]: login query for 'john.smith' tty0 from 172.16.1.210 accepted
Sat Jan 25 00:06:53 2014 [27504]: enable query for 'john.smith' tty0 from 172.16.1.210 accepted
Sat Jan 25 00:34:57 2014 [31581]: login query for 'john_smith' tty1 from 172.16.1.210 rejected
Sat Jan 25 00:52:18 2014 [1510]: login query for 'john_smith' tty1 from 172.16.1.200 rejected
Sat Jan 25 00:52:26 2014 [1525]: login query for 'john_smith' tty1 from 172.16.1.200 rejected
Sat Jan 25 00:59:42 2014 [2617]: login query for 'john_smith' tty2 from 172.16.1.200 rejected
Sat Jan 25 01:05:25 2014 [3499]: login query for 'john.smith' 0 from 172.16.1.220 accepted
Sat Jan 25 01:05:26 2014 [3502]: authorization query for 'john.smith' 0 from 172.16.1.220 accepted
Sat Jan 25 01:07:24 2014 [3790]: login query for 'john.smith' tty2 from 172.16.1.200 accepted
Sat Jan 25 01:07:24 2014 [3801]: authorization query for 'john.smith' tty2 from 172.16.1.200 accepted
Sat Jan 25 01:09:22 2014 [4071]: login query for 'john.smith' 0 from 172.16.1.220 accepted
Sat Jan 25 01:09:23 2014 [4078]: authorization query for 'john.smith' 0 from 172.16.1.220 accepted
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide