Authorization failed on 3560 after IOS upgrade

[v.c.bodenstab]

Hi all,

I've just upgraded a CAT3560-48TS from IOS 12.2(37)SE1(ipservicesk9) to 12.2(46)SE (ipservicesk9). All seems fine untill I tried logging with my TACACS account: I get a authorization failed. Logging in with a local priv15 account works just fine. After removing the following statements:

aaa authorization exec default group auth-server local

aaa authorization commands 0 default group auth-server none

aaa authorization commands 1 default group auth-server none

aaa authorization commands 15 default group auth-server none

everything works fine again.

Also, I've upgraded other 3560 switches to 12.2(46)SE with an ipbase image. Those switches work fine with exactly the same AAA IOS configuration. Any thoughts on this one?

Cheers,

Vincent

[Jagdeep Gambhir]

Vincent,

Please try this command,

"aaa authorization exec default group auth-server if-authenticated none"

Let me know if that fix it.

Regards,

~JG

Do rate helpful posts

[Premdeep Banga]

If everything was working before upgrade and not after, then I would suggest to check/get following debugs,

debug aaa authorization

debug tacacs

term mon

Regards,

Prem

[v.c.bodenstab]

Hey guys,

Thanks for the replies. This is what I see running a few debug sessions while trying to log in:

Oct 28 15:47:13: AAA/BIND(00000036): Bind i/f

Oct 28 15:47:13: TPLUS: Queuing AAA Authentication request 54 for processing

Oct 28 15:47:13: TPLUS: processing authentication start request id 54

Oct 28 15:47:13: TPLUS: Authentication start packet created for 54()

Oct 28 15:47:13: TPLUS: Using server 145.24.1.113

Oct 28 15:47:13: TPLUS(00000036)/0/NB_WAIT/38A911C: Started 5 sec timeout

Oct 28 15:47:13: TPLUS(00000036)/0/NB_WAIT: wrote entire 37 bytes request

Oct 28 15:47:13: TPLUS: Would block while reading pak header

Oct 28 15:47:13: TPLUS(00000036)/0/READ: read entire 12 header bytes (expect 43 bytes)

Oct 28 15:47:13: TPLUS(00000036)/0/READ: read entire 55 bytes response

Oct 28 15:47:13: TPLUS(00000036)/0/38A911C: Processing the reply packet

Oct 28 15:47:13: TPLUS: Received authen response status GET_USER (7)

Oct 28 15:47:15: TPLUS: Queuing AAA Authentication request 54 for processing

Oct 28 15:47:15: TPLUS: processing authentication continue request id 54

Oct 28 15:47:15: TPLUS: Authentication continue packet generated for 54

Oct 28 15:47:15: TPLUS(00000036)/0/WRITE/3838988: Started 5 sec timeout

Oct 28 15:47:15: TPLUS(00000036)/0/WRITE: wrote entire 24 bytes request

Oct 28 15:47:15: TPLUS(00000036)/0/READ: read entire 12 header bytes (expect 16 bytes)

Oct 28 15:47:15: TPLUS(00000036)/0/READ: read entire 28 bytes response

Oct 28 15:47:15: TPLUS(00000036)/0/3838988: Processing the reply packet

Oct 28 15:47:15: TPLUS: Received authen response status GET_PASSWORD (8)

Oct 28 15:47:18: TPLUS: Queuing AAA Authentication request 54 for processing

Oct 28 15:47:18: TPLUS: processing authentication continue request id 54

Oct 28 15:47:18: TPLUS: Authentication continue packet generated for 54

Oct 28 15:47:18: TPLUS(00000036)/0/WRITE/3838988: Started 5 sec timeout

Oct 28 15:47:18: TPLUS(00000036)/0/WRITE: wrote entire 25 bytes request

Oct 28 15:47:18: TPLUS(00000036)/0/READ: read entire 12 header bytes (expect 6 bytes)

Oct 28 15:47:18: TPLUS(00000036)/0/READ: read entire 18 bytes response

Oct 28 15:47:18: TPLUS(00000036)/0/3838988: Processing the reply packet

Oct 28 15:47:18: TPLUS: Received authen response status PASS (2)

Oct 28 15:47:18: AAA/AUTHOR (0x36): Pick method list 'default'

Oct 28 15:47:18: TPLUS: Queuing AAA Authorization request 54 for processing

Oct 28 15:47:18: TPLUS: processing authorization request id 54

Oct 28 15:47:18: TPLUS: Protocol set to None .....Skipping

Oct 28 15:47:18: TPLUS: Sending AV service=shell

Oct 28 15:47:18: TPLUS: Sending AV cmd*

Oct 28 15:47:18: TPLUS: Authorization request created for 54(vincent)

Oct 28 15:47:18: TPLUS: using previously set server 145.24.1.113 from group auth-server

Oct 28 15:47:18: TPLUS(00000036)/0/IDLE/38CDE10: got immediate connect on new 0

Oct 28 15:47:18: TPLUS(00000036)/0/WRITE/38CDE10: Started 5 sec timeout

Oct 28 15:47:18: TPLUS(00000036)/0/WRITE: wrote entire 63 bytes request

Oct 28 15:47:18: TPLUS(00000036)/0/38CDE10: Processing the reply packet - FAIL

Oct 28 15:47:18: AAA/AUTHOR/EXEC(00000036): Authorization FAILED

If I run an wireshark on the Tacacs server, I see the Tacacs server sending a RST packet on the connection... It forcibly closes the connection upon receiving the TACACS Q: AUTHORIZATION request. I've already restarted the TACACS software to no avail.

Any other ideas?? Once again, al other devices run perfectly.

[salmodov]

Try this

aaa new-model

!

aaa authentication login acs_tacacs group tacacs+ local

aaa authentication enable default group tacacs+ enable

aaa authorization exec acs_tacacs group tacacs+ local

aaa accounting exec default start-stop group tacacs+

aaa accounting commands 1 default start-stop group tacacs+

aaa accounting commands 15 default start-stop group tacacs+

aaa accounting connection default start-stop group tacacs+

aaa accounting system default start-stop group tacacs+

!

tacacs-server host 10.x.x.x

tacacs-server host 10.x.x.x

tacacs-server key verify that this correct

[Richard Burts]

Vincent

I see a couple of things in the debug that puzzle me. These lines:

Oct 28 15:47:18: AAA/AUTHOR (0x36): Pick method list 'default'

Oct 28 15:47:18: TPLUS: Queuing AAA Authorization request 54 for processing

Oct 28 15:47:18: TPLUS: processing authorization request id 54

Oct 28 15:47:18: TPLUS: Protocol set to None .....Skipping

especially the line about "Protocol set to None" do not seem to match up with the configuration that you posted. Perhaps you can post the entire AAA configuration and the configuration of the console and of the vty lines. And can you tell us if you were attempting to login via console or vty?

HTH

Rick

[v.c.bodenstab]

Hi Rick,

Following your reply, this is my config. I altered it a little to blank the passwords/secrets.

username root privilege 15 secret 5 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

aaa new-model

!

!

aaa group server tacacs+ auth-server

server 1.1.1.1

!

aaa authentication login default group auth-server local

aaa authorization exec default group auth-server local

aaa authorization commands 0 default group auth-server none

aaa authorization commands 1 default group auth-server none

aaa authorization commands 15 default group auth-server none

aaa accounting send stop-record authentication failure

aaa accounting exec default start-stop group auth-server

aaa accounting commands 0 default start-stop group auth-server

aaa accounting commands 1 default start-stop group auth-server

aaa accounting commands 15 default start-stop group auth-server

aaa accounting system default start-stop group auth-server

!

!

!

aaa session-id common

!

ip tacacs source-interface VlanXXX

!

tacacs-server host 1.1.1.1 single-connection

no tacacs-server directed-request

tacacs-server key XXXXXXXXXXXX

!

line vty 0 4

access-class 75 in

exec-timeout 60 0

transport preferred none

escape-character 3

line vty 5 15

access-class 75 in

!

See anything unusual? I was already thinking that maybe the IPservices version of 12.2(46) handles AAA differently?

Cheers,

Vincent

[Richard Burts]

Vincent

I do not see anything in the config that would explain the symptoms that you describe.

I am still very puzzled about the lines that indicate that the method is none. Since essentially the same config runs ok on other devices I wonder if there might be something in the configuration of the TACACS server that is different for this router?

I am also wondering if there is something different between the ipbase and the ipservices versions of code. Are you in a position to open a case with TAC about this?

I wonder if making this change would do any good. Replace:

aaa authorization commands 0 default group auth-server none

aaa authorization commands 1 default group auth-server none

aaa authorization commands 15 default group auth-server none

with

aaa authorization commands 0 default group auth-server if-authenticated

aaa authorization commands 1 default group auth-server if-authenticated

aaa authorization commands 15 default group auth-server if-authenticated

[edit] in re-reading the original post I see that this was an upgrade of the switch. Assuming that it was working before the upgrade that would say that it is not an issue of how the server is configured for this device.

HTH

Rick

[Premdeep Banga · Accepted solution]

Execute these commands,

no tacacs-server host single-connection

tacacs-server host

Regards,

Prem

Please rate if it helps!

[v.c.bodenstab]

Hi Prem,

It actually *does* work. However, I don't really see this as a fix. Looks more like a work-around. I actually do want one single TCP connection to my TACACS box...

Look like a bug to me. What do you guys think?

cheers,Vincent

[Premdeep Banga · Accepted solution]

I know its not a solution :)

Can you please check, CSCsf25057 and check with the ACS version that you are running ?

Regards,

Prem

Please rate if it helps!

[v.c.bodenstab]

Hi Prem,

A little belated reply but I finally managed to upgrade my ACS installation. I'm running 4.1.4(13) latest/greatest now and the problem has disappeared. I *was* an ACS problem. Thanks very much for your help!

Cheers,

Vincent

[dsiziba_1]

Hi Guys,

I seem to have a simmilar issuse , but with an ASA 5510. I have upgraded the Image using 724-k8.bin with asdm 524.bin. I get the same error whe i try to authenticate with radius (microsoftIAS)

Below is my auth debug output when I test from miscrosoft IAS:

AWI-FW2# radius mkreq: 0x4a

alloc_rip 0x426bbf0

new request 0x4a --> 32 (0x426bbf0)

got user ''

got password

add_req 0x426bbf0 session 0x4a id 32

RADIUS_REQUEST

radius.c: rad_mkpkt

RADIUS packet decode (authentication request)

--------------------------------------

Raw packet data (length = 68).....

01 20 00 44 e6 27 d4 7d 72 c3 40 79 be 1f 6c 35 | . .D.'.}r.@y..l5

ca 3b 58 b1 01 0c 64 6f 75 67 73 69 7a 69 62 61 | .;X...*********

02 12 94 41 eb c9 5e b8 28 4f 24 0e d5 14 71 5e | ...A..^.(O$...q^

1c 64 04 06 0a 0a 01 02 05 06 00 00 00 20 3d 06 | .d........... =.

00 00 00 05 | ....

Parsed packet data.....

Radius: Code = 1 (0x01)

Radius: Identifier = 32 (0x20)

Radius: Length = 68 (0x0044)

Radius: Vector: E627D47D72C34079BE1F6C35CA3B58B1

Radius: Type = 1 (0x01) User-Name

Radius: Length = 12 (0x0C)

Radius: Value (String) =

64 6f 75 67 73 69 7a 69 62 61 | *********

Radius: Type = 2 (0x02) User-Password

Radius: Length = 18 (0x12)

Radius: Value (String) =

94 41 eb c9 5e b8 28 4f 24 0e d5 14 71 5e 1c 64 | .A..^.(O$...q^.d

Radius: Type = 4 (0x04) NAS-IP-Address

Radius: Length = 6 (0x06)

Radius: Value (IP Address) = *.*.*.* (0x0A0A0102)

Radius: Type = 5 (0x05) NAS-Port

Radius: Length = 6 (0x06)

Radius: Value (Hex) = 0x20

Radius: Type = 61 (0x3D) NAS-Port-Type

Radius: Length = 6 (0x06)

Radius: Value (Hex) = 0x5

send pkt 10.10.2.41/1645

fail request 0x4a (10.10.2.41 failed)

RADIUS_DELETE

remove_req 0x426bbf0 session 0x4a id 32

free_rip 0x426bbf0

radius: send queue empty

radius mkreq: 0x4c

alloc_rip 0x426bbf0

new request 0x4c --> 33 (0x426bbf0)

got user ''

got password

add_req 0x426bbf0 session 0x4c id 33

RADIUS_REQUEST

radius.c: rad_mkpkt

RADIUS packet decode (authentication request)

--------------------------------------

Raw packet data (length = 71).....

01 21 00 47 ed 22 b3 70 e9 6e 0f 9c a5 7a 2b 88 | .!.G.".p.n...z+.

21 46 07 34 01 0f 61 64 6d 69 6e 69 73 74 72 61 | !F.4..administra

74 6f 72 02 12 0e 94 15 35 95 91 13 b8 e5 0e a1 | tor.....5.......

9f f4 89 c0 89 04 06 0a 0a 01 02 05 06 00 00 00 | ................

21 3d 06 00 00 00 05 | !=.....

Parsed packet data.....

Radius: Code = 1 (0x01)

Radius: Identifier = 33 (0x21)

Radius: Length = 71 (0x0047)

Radius: Vector: ED22B370E96E0F9CA57A2B8821460734

Radius: Type = 1 (0x01) User-Name

Radius: Length = 15 (0x0F)

Radius: Value (String) =

61 64 6d 69 6e 69 73 74 72 61 74 6f 72 | administrator

Radius: Type = 2 (0x02) User-Password

Radius: Length = 18 (0x12)

Radius: Value (String) =

0e 94 15 35 95 91 13 b8 e5 0e a1 9f f4 89 c0 89 | ...5............

Radius: Type = 4 (0x04) NAS-IP-Address

Radius: Length = 6 (0x06)

Radius: Value (IP Address) = *.*.*.* (0x0A0A0102)

Radius: Type = 5 (0x05) NAS-Port

Radius: Length = 6 (0x06)

Radius: Value (Hex) = 0x21

Radius: Type = 61 (0x3D) NAS-Port-Type

Radius: Length = 6 (0x06)

Radius: Value (Hex) = 0x5

send pkt 10.10.2.40/1645

RADIUS_SENT:server response timeout

RADIUS_DELETE

remove_req 0x426bbf0 session 0x4c id 33

free_rip 0x426bbf0

radius: send queue empty

[Premdeep Banga]

RADIUS_SENT:server response timeout

probably there is no response being sent from the radius server for the request, resulting in the failure.

Regards,

Prem

Please rate if it helps!

[dsiziba_1]

Hi Prem

I can ping the server ok and I have confirmed microsoft IAS is running OK.

What else can I be missing?

Error msges showing on the ASA :

2 Oct 29 2008 11:01:32 113022 AAA Marking RADIUS server x.x.x.x in aaa-server group MicrosoftIAS as FAILED