UCS TACACS authentication after cluster failover

[joeharb]

     We have TACACS running on our UCS enviroment.  When FIA is primary TACACS workings without issue.  When we do a cluster failover and FIB takes over the role of primary TACACS fails.  We can ping the TACACS server from both of the FI's.  When we attemp to test authentication from the nxos FIA resolves fine but FIB gives a "error authenticating to server" response.  The configs look identical.

Has anyone else had this issue or have any suggestions?

Thanks,

Joe                   

[padramas]

Hello Joe,

Please share additional information about the setup

UCSM version

Is FI configured with TACACS server with IP address or hostname ?

Can you please run " test aaa server tacacs+ ...... " command ( nx-os context )  from both FIs and share the output.

Padma

[joeharb]

UCS version is 2.1(1a).

Both FI's are configured with IP address of TACACS server.

Output from FIA (which works)

ValUCS-A(nxos)# test aaa server tacacs+ IPofSERVER ucsm_user XXXXXXXX

user has been authenticated

Attributes downloaded from remote server:

Roles:

        network-admin vdc-admin admin aaa system-admin

User profile attribute:

        shell:roles*"network-admin vdc-admin admin aaa system-admin"

Ouput from FIB (which doesn't)

ValUCS-B(nxos)# test aaa server tacacs+ IPofSERVER ucsm_user XXXXXXXX

error authenticating to server

EDIT:

I have verified that I can ping the IPofSERVER from both FI's.

Thanks,

Joe

Message was edited by: Joseph Harbison

[padramas]

Hello Joe,

Please turn on the debugs on FI-B(nxos) and share the SSH session output

debug tacacs+ aaa-request

debug tacacs+ aaa-request-lowlevel

debug tacacs+ all

Run the test aaa command

Turn off the debugs by " un all "

Padma

[joeharb]

Here is the output

ValUCS-B(nxos)# test aaa server tacacs+ IPofTACACS ucsm_user XXXXXXX
2013 May 24 10:52:57.661229 tacacs: event_loop(): calling process_rd_fd_set
2013 May 24 10:52:57.661623 tacacs: process_rd_fd_set: calling callback for fd 6
2013 May 24 10:52:57.662019 tacacs: fsrv_sdb_process_msg: vdc-id[1] mts_opc[8422][MTS_OPC_TACACS_SERVER_CONFIG] 0xbfffe1f0 0xb5a2dc64 881
2013 May 24 10:52:57.662357 tacacs: Sending it to SDB-Dispatch
2013 May 24 10:52:57.662690 tacacs: Sdb-dispatch did not process
2013 May 24 10:52:57.663053 tacacs: No msg handler in FSRV for mts_opc[8422][MTS_OPC_TACACS_SERVER_CONFIG]
2013 May 24 10:52:57.663387 tacacs: fsrv didnt consume 8422 opcode
2013 May 24 10:52:57.663731 tacacs: process_implicit_cfs_session_start: entering...
2013 May 24 10:52:57.664064 tacacs: process_implicit_cfs_session_start: exiting; we are in distribution disabled state
2013 May 24 10:52:57.664424 tacacs: mts_tplus_server_config: entering....
2013 May 24 10:52:57.664761 tacacs: proto_cfs_distr_session_in_progress : 0
2013 May 24 10:52:57.665144 tacacs: get_pss_url : returning NULL
2013 May 24 10:52:57.692358 tacacs: mts_tplus_server_config: on (null)
2013 May 24 10:52:57.692756 tacacs: tacacs_server_config: entering for server IPofTACACS, index 0
2013 May 24 10:52:57.693094 tacacs: tacacs_server_config: GET request for Protocol server index:0 addr:IPofTACACS
2013 May 24 10:52:57.693427 tacacs: find_tacacs_server: entering for server IPofTACACS
2013 May 24 10:52:57.693809 tacacs: find_tacacs_server: exiting for server IPofTACACS index is 1
2013 May 24 10:52:57.694153 tacacs: tacacs_server_config: Got for Protocol server index:1 addr:IPofTACACS
2013 May 24 10:52:57.694491 tacacs: tacacs_server_config: got back the return value of Protocol server IPofTACACS operation: SUCCESS
2013 May 24 10:52:57.694834 tacacs: tacacs_server_config: returning auth-port 49, acct-port 49 for Protocol server:IPofTACACS
2013 May 24 10:52:57.695168 tacacs: tacacs_server_config: returning retval 0 for Protocol server:IPofTACACS
2013 May 24 10:52:57.700865 tacacs: process_rd_fd_set: callback returned for fd 6
2013 May 24 10:52:57.702596 tacacs: event_loop(): calling process_rd_fd_set
2013 May 24 10:52:57.702971 tacacs: process_rd_fd_set: calling callback for fd 6
2013 May 24 10:52:57.703362 tacacs: fsrv_sdb_process_msg: vdc-id[1] mts_opc[8421][MTS_OPC_TACACS_AAA_REQ] 0xbfffe1f0 0xb59d6264 380
2013 May 24 10:52:57.703700 tacacs: Sending it to SDB-Dispatch
2013 May 24 10:52:57.704035 tacacs: Sdb-dispatch did not process
2013 May 24 10:52:57.704399 tacacs: No msg handler in FSRV for mts_opc[8421][MTS_OPC_TACACS_AAA_REQ]
2013 May 24 10:52:57.704733 tacacs: fsrv didnt consume 8421 opcode
2013 May 24 10:52:57.705077 tacacs: process_implicit_cfs_session_start: entering...
2013 May 24 10:52:57.735438 tacacs: process_implicit_cfs_session_start: exiting; we are in distribution disabled state
2013 May 24 10:52:57.735808 tacacs: process_aaa_tplus_request: entering for aaa session id 0
2013 May 24 10:52:57.736194 tacacs: process_aaa_tplus_request:Checking for state of mgmt0 port with servergroup
2013 May 24 10:52:57.736539 tacacs: tacacs_servergroup_config: entering for server group, index 0
2013 May 24 10:52:57.736873 tacacs: tacacs_servergroup_config: GETNEXT_REQ for Protocol server group index:0 name:
2013 May 24 10:52:57.737234 tacacs: tacacs_servergroup_config: GETNEXT_REQ got Protocol server group index:3 name:tacacs
2013 May 24 10:52:57.737568 tacacs: tacacs_servergroup_config: got back the return value of Protocol group operation:SUCCESS
error authenticating to server
ValUCS-B(nxos)# 2013 May 24 10:52:57.737908 tacacs: tacacs_servergroup_config: returning retval 0 for Protocol server group:tacacs
2013 May 24 10:52:57.738249 tacacs: tacacs_servergroup_config: entering for server grouptacacs, index 3
2013 May 24 10:52:57.738589 tacacs: tacacs_servergroup_config: GETNEXT_REQ for Protocol server group index:3 name:tacacs
2013 May 24 10:52:57.738943 tacacs: tacacs_servergroup_config: GETNEXT_REQ got Protocol server group index:4 name:IPofTACACS
2013 May 24 10:52:57.739275 tacacs: tacacs_servergroup_config: got back the return value of Protocol group operation:SUCCESS
2013 May 24 10:52:57.739616 tacacs: tacacs_servergroup_config: returning retval 0 for Protocol server group:IPofTACACS
2013 May 24 10:52:57.739948 tacacs: tacacs_servergroup_config: entering for server groupIPofTACACS, index 4
2013 May 24 10:52:57.740290 tacacs: tacacs_servergroup_config: GETNEXT_REQ for Protocol server group index:4 name:IPofTACACS
2013 May 24 10:52:57.740632 tacacs: tacacs_servergroup_config: GETNEXT_REQ: PROTO_END_OF_GROUP_TABLE for Protocol server group IPofTACACS
2013 May 24 10:52:57.740975 tacacs: tacacs_servergroup_config: got back the return value of Protocol group operation:reached the end of TACACS+ group table
2013 May 24 10:52:57.741307 tacacs: tacacs_servergroup_config: returning retval 1083965445 for Protocol server group:IPofTACACS
2013 May 24 10:52:57.741652 tacacs: process_aaa_tplus_request: checking for mgmt0 vrf:management against vrf: of requested group
2013 May 24 10:52:57.741992 tacacs: process_aaa_tplus_request(520):aaa_req->server_ip is TRUE shouldn't??
2013 May 24 10:52:57.742335 tacacs: process_aaa_tplus_request: No vrf specified for the server
2013 May 24 10:52:57.742676 tacacs: process_aaa_tplus_request(559): Not retrieving tplus_ctx.if_index No src_intf specified for the server
2013 May 24 10:52:57.743006 tacacs: process_aaa_tplus_request:Moheed TODO aaa_req has server addr IPofTACACS vrf
2013 May 24 10:52:57.743350 tacacs: create_tplus_req_state_machine: entering for aaa session id 0
2013 May 24 10:52:57.743684 tacacs: state machine count 0
2013 May 24 10:52:57.744026 tacacs: init_tplus_req_state_machine: entering for aaa session id 0
2013 May 24 10:52:57.744360 tacacs: init_tplus_req_state_machine: context hostname IPofTACACS, src_intf  and vrf_name  for aaa session id 0
2013 May 24 10:52:57.744704 tacacs: tacacs_server_config: entering for server IPofTACACS, index 0
2013 May 24 10:52:57.745043 tacacs: tacacs_server_config: GET request for Protocol server index:0 addr:IPofTACACS
2013 May 24 10:52:57.746549 tacacs: find_tacacs_server: entering for server IPofTACACS
2013 May 24 10:52:57.746905 tacacs: find_tacacs_server: exiting for server IPofTACACS index is 1
2013 May 24 10:52:57.747245 tacacs: tacacs_server_config: Got for Protocol server index:1 addr:IPofTACACS
2013 May 24 10:52:57.747579 tacacs: tacacs_server_config: got back the return value of Protocol server IPofTACACS operation: SUCCESS
2013 May 24 10:52:57.747910 tacacs: tacacs_server_config: returning auth-port 49, acct-port 49 for Protocol server:IPofTACACS
2013 May 24 10:52:57.748242 tacacs: tacacs_server_config: returning retval 0 for Protocol server:IPofTACACS
2013 May 24 10:52:57.748599 tacacs: tacacs_global_config: entering ...
2013 May 24 10:52:57.748932 tacacs: tacacs_global_config: GET_REQ...
2013 May 24 10:52:57.749272 tacacs: tacacs_global_config: got back the return value of global Protocol configuration operation:SUCCESS
2013 May 24 10:52:57.749612 tacacs: tacacs_global_config: REQ:num server 1
2013 May 24 10:52:57.749942 tacacs: tacacs_global_config: REQ:num group 2
2013 May 24 10:52:57.750274 tacacs: tacacs_global_config: REQ:num timeout 5
2013 May 24 10:52:57.750606 tacacs: tacacs_global_config: REQ:num deadtime 0
2013 May 24 10:52:57.750938 tacacs: tacacs_global_config: REQ:num encryption_type 0
2013 May 24 10:52:57.751268 tacacs: tacacs_global_config: REQ:num if_index 0x0, intf
2013 May 24 10:52:57.751604 tacacs: tacacs_global_config: returning retval 0
2013 May 24 10:52:57.751944 tacacs: init_tplus_req_state_machine(1417):tplus_context don't know about src-intf
2013 May 24 10:52:57.752281 tacacs: init_tplus_req_state_machine(1419):asking to aaa regarding group info.
2013 May 24 10:52:57.752647 tacacs: aaa_server_get_next_group: 1
2013 May 24 10:52:57.752989 tacacs: tacacs_servergroup_config: entering for server group, index 0
2013 May 24 10:52:57.753321 tacacs: tacacs_servergroup_config: GETNEXT_REQ for Protocol server group index:0 name:
2013 May 24 10:52:57.753662 tacacs: tacacs_servergroup_config: GETNEXT_REQ got Protocol server group index:3 name:tacacs
2013 May 24 10:52:57.753995 tacacs: tacacs_servergroup_config: got back the return value of Protocol group operation:SUCCESS
2013 May 24 10:52:57.754327 tacacs: tacacs_servergroup_config: returning retval 0 for Protocol server group:tacacs
2013 May 24 10:52:57.754659 tacacs: aaa_server_get_next_group: found group tacacs
2013 May 24 10:52:57.754991 tacacs: aaa_server_get_next_group: 1
2013 May 24 10:52:57.755324 tacacs: tacacs_servergroup_config: entering for server grouptacacs, index 3
2013 May 24 10:52:57.755849 tacacs: tacacs_servergroup_config: GETNEXT_REQ for Protocol server group index:3 name:tacacs
2013 May 24 10:52:57.756203 tacacs: tacacs_servergroup_config: GETNEXT_REQ got Protocol server group index:4 name:IPofTACACS
2013 May 24 10:52:57.756548 tacacs: tacacs_servergroup_config: got back the return value of Protocol group operation:SUCCESS
2013 May 24 10:52:57.756879 tacacs: tacacs_servergroup_config: returning retval 0 for Protocol server group:IPofTACACS
2013 May 24 10:52:57.757214 tacacs: aaa_server_get_next_group: found group IPofTACACS
2013 May 24 10:52:57.757546 tacacs: aaa_server_get_next_group: 1
2013 May 24 10:52:57.757878 tacacs: tacacs_servergroup_config: entering for server groupIPofTACACS, index 4
2013 May 24 10:52:57.758210 tacacs: tacacs_servergroup_config: GETNEXT_REQ for Protocol server group index:4 name:IPofTACACS
2013 May 24 10:52:57.758554 tacacs: tacacs_servergroup_config: GETNEXT_REQ: PROTO_END_OF_GROUP_TABLE for Protocol server group IPofTACACS
2013 May 24 10:52:57.758888 tacacs: tacacs_servergroup_config: got back the return value of Protocol group operation:reached the end of TACACS+ group table
2013 May 24 10:52:57.759219 tacacs: tacacs_servergroup_config: returning retval 1083965445 for Protocol server group:IPofTACACS
2013 May 24 10:52:57.759551 tacacs: aaa_server_get_next_group: AAA_GROUP_END_OF_TABLE
2013 May 24 10:52:57.759896 tacacs: init_tplus_req_state_machine(1492):No interface/ip configured for group IPofTACACS
2013 May 24 10:52:57.760231 tacacs: init_tplus_req_state_machine(1493):Falling for global
2013 May 24 10:52:57.760565 tacacs: Entering function: get_if_index_from_global_conf
2013 May 24 10:52:57.760898 tacacs: tacacs_global_config: entering ...
2013 May 24 10:52:57.761229 tacacs: tacacs_global_config: GET_REQ...
2013 May 24 10:52:57.761568 tacacs: tacacs_global_config: got back the return value of global Protocol configuration operation:SUCCESS
2013 May 24 10:52:57.761901 tacacs: tacacs_global_config: REQ:num server 1
2013 May 24 10:52:57.762231 tacacs: tacacs_global_config: REQ:num group 2
2013 May 24 10:52:57.762561 tacacs: tacacs_global_config: REQ:num timeout 5
2013 May 24 10:52:57.762892 tacacs: tacacs_global_config: REQ:num deadtime 0
2013 May 24 10:52:57.763231 tacacs: tacacs_global_config: REQ:num encryption_type 0
2013 May 24 10:52:57.763564 tacacs: tacacs_global_config: REQ:num if_index 0x0, intf
2013 May 24 10:52:57.763902 tacacs: tacacs_global_config: returning retval 0
2013 May 24 10:52:57.764238 tacacs: Function get_if_index_from_global_conf: found interface
2013 May 24 10:52:57.764580 tacacs: Exiting function: get_if_index_from_global_conf
2013 May 24 10:52:57.764914 tacacs: init_tplus_req_state_machine(1505):Global source-interface not configured/ or intf isn't up
2013 May 24 10:52:57.765257 tacacs: init_tplus_req_state_machine(1507):Random source-ip will be chosen
2013 May 24 10:52:57.765950 tacacs: init_tplus_req_state_machine: SAM 6XXX series vrf management
2013 May 24 10:52:57.766301 tacacs: init_tplus_req_state_machine: returning TRUE for aaa session id 0
2013 May 24 10:52:57.766645 tacacs: update_tplus_state: entering for aaa session id: 0, current state: 1, next state 1
2013 May 24 10:52:57.767001 tacacs: construct_tac_req: entering for aaa session id: 0
2013 May 24 10:52:57.767343 tacacs: construct_tac_req(1582):aaa session id: 0, aaa type: 1, user_len=9, av_count_net=1, av_count_host=1
2013 May 24 10:52:57.767677 tacacs: tplus_make_authen_start: entering for aaa session: 0
2013 May 24 10:52:57.768017 tacacs: tplus_make_authen_start: aaa session: 0, user_len: 9, port_len: 1, rem_addr_len: 0, user_data_len: 12
2013 May 24 10:52:57.768362 tacacs: tplus_fill_header: entering: tacacs session: 1380080250, seq nu: 1
2013 May 24 10:52:57.768706 tacacs: tplus_make_authen_start: exiting for aaa session: 0 authen action 1 1
2013 May 24 10:52:57.769038 tacacs: update_tplus_state: entering for aaa session id: 0, current state: 1, next state 2
2013 May 24 10:52:57.769386 tacacs: connect_tac_server: entering for aaa session id: 0
2013 May 24 10:52:57.769728 tacacs: vrf is management
2013 May 24 10:52:57.770103 tacacs: l3vm_get_context_id returns 2
2013 May 24 10:52:57.770453 tacacs: intf is not present
2013 May 24 10:52:57.770800 tacacs: Entering : check_local_cache : Line : 978
2013 May 24 10:52:57.771142 tacacs: serv_id : IPofTACACS:49
2013 May 24 10:52:57.771472 tacacs: Entry Found
2013 May 24 10:52:57.771810 tacacs: Entering : get_res_back : Line : 487
2013 May 24 10:52:57.772141 tacacs: No of entries in server_procjob_data_t : 1 
2013 May 24 10:52:57.772486 tacacs: Exiting: get_res_back , Line : 564
2013 May 24 10:52:57.772818 tacacs: Found in Local Cache
2013 May 24 10:52:57.773157 tacacs: non_blocking_connect: server IPofTACACS port 49, ipaddress type = ipv4
2013 May 24 10:52:57.773489 tacacs: non_blocking_connect(268): server IPofTACACS port 49, vrf 2, src_intf not configured
2013 May 24 10:52:57.773873 tacacs: non_blocking_connect(421):Not using src-intf and bind() since ifindex=0
2013 May 24 10:52:57.774295 tacacs: non_blocking_connect(466): connect() successfull
2013 May 24 10:52:57.774639 tacacs: non_blocking_connect: socket 24, num_of_client_sockets 1
2013 May 24 10:52:57.774982 tacacs: connect_tac_server: add_to_wr_fd_table socket 24
2013 May 24 10:52:57.775331 tacacs: update_tplus_state: entering for aaa session id: 0, current state: 2, next state 3
2013 May 24 10:52:57.777356 tacacs: update_tplus_state: exiting for aaa session id: 0, state: 3
2013 May 24 10:52:57.777696 tacacs: connect_tac_server: exiting for aaa session id: 0
2013 May 24 10:52:57.778028 tacacs: update_tplus_state: exiting for aaa session id: 0, state: 2
2013 May 24 10:52:57.778360 tacacs: construct_tac_req: exiting for aaa session id: 0
2013 May 24 10:52:57.778690 tacacs: update_tplus_state: exiting for aaa session id: 0, state: 1
2013 May 24 10:52:57.779036 tacacs: create_tplus_req_state_machine: exiting for aaa session id 0
2013 May 24 10:52:57.779368 tacacs: process_aaa_tplus_request: using server_ip returning TRUE...
2013 May 24 10:52:57.779703 tacacs: process_rd_fd_set: callback returned for fd 6
2013 May 24 10:52:57.796304 tacacs: event_loop(): calling process_rd_fd_set
2013 May 24 10:52:57.796675 tacacs: process_wr_fd_set: calling callback for fd 24
2013 May 24 10:52:57.797020 tacacs: sock_connect_callback: state machine is for server IPofTACACS for socket 24
2013 May 24 10:52:57.797365 tacacs: sock_connect_callback: connect succedded for socket 24
2013 May 24 10:52:57.797700 tacacs: aaa_mark_server_alive:  1
2013 May 24 10:52:57.798115 tacacs: sock_connect_callback: tac payload before encrypt:
2013 May 24 10:52:57.798447 tacacs: out tac payload len:42
2013 May 24 10:52:57.798798 tacacs: tac payload(hex): c1 1 1 1 52 42 5a 7a 0 0 0 1e 1 1 2 1 9 1 0 c 75 63 73 6d 5f 75 73 65 72 30 32 37 38 34 43 75 63 73 51 23 31 32
2013 May 24 10:52:57.799129 tacacs: tac payload(ascii): Á ^A ^A ^A R B Z z    ^^ ^A ^A ^B ^A ^I ^A  ^L u c s m _ u s e r 0 PasswordREmove 2013 May 24 10:52:57.799491 tacacs: sock_connect_callback: going to send on socket 24
2013 May 24 10:52:57.799822 tacacs: out tac payload len:42
2013 May 24 10:52:57.800162 tacacs: tac payload encrypted(hex): c1 1 1 0 52 42 5a 7a 0 0 0 1e 5 1 b5 90 bc 51 f5 f7 2c 74 d6 a8 52 22 cd f7 b 2e 74 26 62 e1 97 e4 4b dd 1e 1f b4 b8
2013 May 24 10:52:57.800510 tacacs: update_tplus_state: entering for aaa session id: 0, current state: 3, next state 4
2013 May 24 10:52:57.800870 tacacs: update_tplus_state: exiting for aaa session id: 0, state: 4
2013 May 24 10:52:57.801203 tacacs: sock_connect_callback: exiting for socket 24
2013 May 24 10:52:57.801560 tacacs: event_loop(): calling process_rd_fd_set
2013 May 24 10:52:57.801903 tacacs: process_wr_fd_set: calling callback for fd 24
2013 May 24 10:52:57.802242 tacacs: sock_write_callback: entering for socket 24
2013 May 24 10:52:57.802584 tacacs: continue_send_tac_req: entering for aaa session id: 0 ,bytes_sent: 0, bytes_left: 42
2013 May 24 10:52:57.802916 tacacs: non_blocking_write: entering for socket 24
2013 May 24 10:52:57.803285 tacacs: non_blocking_write: exiting for socket 24
2013 May 24 10:52:57.803624 tacacs: update_tplus_state: entering for aaa session id: 0, current state: 4, next state 5
2013 May 24 10:52:57.803973 tacacs: update_tplus_state: exiting for aaa session id: 0, state: 5
2013 May 24 10:52:57.804306 tacacs: continue_send_tac_req: exiting for aaa session id: 0
2013 May 24 10:52:57.804654 tacacs: sock_write_callback: exiting for socket 24
2013 May 24 10:52:57.873223 tacacs: event_loop(): calling process_rd_fd_set
2013 May 24 10:52:57.873576 tacacs: process_rd_fd_set: calling callback for fd 24
2013 May 24 10:52:57.873923 tacacs: sock_read_callback: entering for socket 24
2013 May 24 10:52:57.874273 tacacs: continue_rcv_tac_req: entering for aaa session id: 0 ,bytes_recvd: 0, recv_bytes_left: 12
2013 May 24 10:52:57.874608 tacacs: non_blocking_read: entering for socket 24
2013 May 24 10:52:57.874952 tacacs: non_blocking_read: read() errno: 104 for socket 24
2013 May 24 10:52:57.875287 tacacs: continue_rcv_tac_req: non_blocking_read failed for aaa session 0
2013 May 24 10:52:57.875689 tacacs: update_tplus_state: entering for aaa session id: 0, current state: 5, next state 8
2013 May 24 10:52:57.876048 tacacs: switch_tac_server: entering for aaa session 0
2013 May 24 10:52:57.876386 tacacs: switch_tac_server:  server rollover not possible, no servergroup info present for aaa session 0
2013 May 24 10:52:57.876718 tacacs: send_aaa_tplus_resp_error_mts: entering for aaa session 0
2013 May 24 10:52:57.877051 tacacs: construct_aaa_tplus_error_resp: entering for aaa session: 0
2013 May 24 10:52:57.877644 tacacs: construct_aaa_tplus_error_resp: exiting for aaa session: 0
2013 May 24 10:52:57.877993 tacacs: send_aaa_tplus_resp_mts: entering for aaa session 0
2013 May 24 10:52:57.883103 tacacs: send_aaa_tplus_resp_mts: exiting for aaa session 0
2013 May 24 10:52:57.883453 tacacs: send_aaa_tplus_resp_error_mts: exiting for aaa session 0
2013 May 24 10:52:57.883787 tacacs: update_tplus_state: entering for aaa session id: 0, current state: 8, next state 9
2013 May 24 10:52:57.884148 tacacs: close_tac_req_state_machine: entering for aaa session id: 0
2013 May 24 10:52:57.884483 tacacs: tplus_free_req_state_machine: entering for aaa session 0
2013 May 24 10:52:57.884838 tacacs: tplus_free_req_state_machine: exiting for aaa session 0
2013 May 24 10:52:57.885181 tacacs: close_tac_req_state_machine: exiting for aaa session id: 0
2013 May 24 10:52:57.915573 tacacs: update_tplus_state: exiting for aaa session id: 0, state: 9
2013 May 24 10:52:57.915927 tacacs: update_tplus_state: exiting for aaa session id: 0, state: 8
2013 May 24 10:52:57.916261 tacacs: sock_read_callback: exiting for socket 24
2013 May 24 10:52:57.916594 tacacs: process_rd_fd_set: callback returned for fd 24

ValUCS-B(nxos)# un all
2013 May 24 10:53:09.709053 tacacs: event_loop(): calling process_rd_fd_set
2013 May 24 10:53:09.727437 tacacs: process_rd_fd_set: calling callback for fd 6
2013 May 24 10:53:09.727854 tacacs: fsrv_sdb_process_msg: vdc-id[1] mts_opc[922][MTS_OPC_VSH_UNDEBUG_ALL] 0xbfffe1f0 0xb59d6264 202
2013 May 24 10:53:09.728193 tacacs: Sending it to SDB-Dispatch
2013 May 24 10:53:09.728529 tacacs: Sdb-dispatch did not process
2013 May 24 10:53:09.728905 tacacs: No msg handler in FSRV for mts_opc[922][MTS_OPC_VSH_UNDEBUG_ALL]
2013 May 24 10:53:09.729240 tacacs: fsrv didnt consume 922 opcode

Thanks,

[gopis]

It looks like the TACACS config was not synced to FI-B. Could you please check the FSM status.

From CLI

FI-B# sc security

FI-B /security # sc tacacs

FI-B /security/tacacs # show fsm status

Also check the deployed TACACS configuration on NX-OS

FI-B(nxos)# show tacacs-server

[joeharb]

ValUCS-B /security/tacacs # show fsm status

    FSM 1:
        Status: Nop
        Previous Status: Update Ep Success
        Timestamp: 2013-05-21T09:22:34.312
        Try: 0
        Progress (%): 100
        Current Task:

ValUCS-B(nxos)# show tacacs-server
timeout value:5
deadtime value:0
source interface:any available
total number of servers:1

following TACACS+ servers are configured:
        10.0.3.26:
                available on port:49
                TACACS+ shared secret:********
                timeout:5
ValUCS-B(nxos)#

[padramas]

Hello,

We need additional logs to further investigate this issue.

Please open a TAC service request with UCSM tech support bundle.

Padma