5417 Dynamic Authorization failed ErrorCause Session Context Not Found

[ifabrizio]

Dear All,

I have setup a Trustsec Laboratory, composed by Cisco ISE 3.2, and 3 switches enrolled in the Trusec domain.

I have attached a Test PC SGT 16 on a 9300 switch that is connected with an uplink to a 9500 switch, and the 9500 has a portchannell connected to a 4500E switch. On the 4500E there is the second test PC that belongs to a different SGT group SGT 17.

When I try to deploy policy from ISE Trustsec Matrix all works as expected, but I got errors about the CoA, on the ISE, these CoA errors are related to the 9500 and 4500E. The policy that I have enforced from ISE go to block the traffic from  source SGT 17 to destination SGT 16.

Follow the CoA error on ISE

Result

RadiusPacketType	CoANAK
Reply-Message	No valid Session
Error-Cause	Session Context Not Found
cisco-av-pair	security-group-tag=10-0056
cisco-command-code	6

On the 9500/4500E switch I got the same error:

AAA request is from proxycoa proxy create aaa protocol :radius coa proxy relay coa resp(iosd)coa proxy create aaa protocol ,size required to flatten attr uf size : 221coa proxy create aaa protocol :CoA Response Detailscoa proxy create aaa protocol :Attr list : << **bleep**e 0 "#CTSREQUEST#">><< nas-ip-addre(0x664CB281)>><< security-group-tag 0 "10-0058">><< ssg-command-code 0 36 >><< reply-message 0 "No valid Session">><< error-cause 0 1xxx.xxx.xxx.xxx cfg_saddr:xxx.xxx.xx.xxx udpport:60189 sport:0, tableid:0iden:82 rad_code:43 msg_auth_rcvd:TRUE coa_resp:NACK91995C5C5515F7A991A23A6966DDAD2Ecoa proxy crea

On the 9300 where is conneted the Test PC that belongs to the destination SGT 16, and where the policy enforcement is done, I do not have any error:

AAA request is from proxycoa proxy create aaa protocol :radius coa proxy relay coa resp(iosd)coa proxy create aaa protocol ,size required to flatten attr lissize : 153coa proxy create aaa protocol :CoA Response Detailscoa proxy create aaa protocol :Attr list : << **bleep**e 0 "#CTSREQUEST#">><< nas-ip-**bleep** 664CB281)>><< security-group-tag 0 "10-0058">><< ssg-command-code 0 36 >>coa proxy create aaa protocol :server:xxx.xxx.xxx.xxx cfg_saddr:xxx.xxx.xxx.xxx udpporresp:ACKEF76291808BD19172855DE76F5E0FC21coa proxy create aaa protocol,msg send to end point 'SMD' succeeded

Are these nomal way to works? I suppose that the 9500/4500E swiches that do not have the destination SGT conneted, do not find any active session to enforce. Cause the 4500E have conneted the source SGT, but by default the switch where is done the policy enforcement, is the one that has the destination SGT/TEST PC connected. 

Could help pls to understand if there are somenthing wrong?

Best regards,

JF.

[Arne Bier]

Hello @ifabrizio 

What is the role of the 4500 and 9500 in your scenario?  Are they Control or Border Nodes?

I am not an expert in this area, but I also would have expected ISE to send a CoA only to the Edge Nodes. 

Unless of course, the SGT Matrix is downloaded to the Fabric Border Nodes too (but I am uncertain about that).

Were the 4500 and 9500 devices provisioned through DNAC? If so, then I suppose they would have been configured with the RADIUS settings as per the DNAC Network Design intent. In which case, the IOS-XE radius client-author list should contain all the PSN IP addresses.

[ifabrizio]

Hi Arne,

9500 are the core Switches the 9300 is where the servers are connected. The 4500 is an distribution/access switch.

I have made same others tests, and I see that the 9500 and 9300 use SISF with Device Tracking.

The 4500 IO-XE use IPDT and rely on the snooping DHCP and Arp Inspection.

Maybe the error on the CoA "No valid Session" on the 4500 is caused by IPDT feature? I should migrate the 4500 to SISF?

Bye,

JF.

[Arne Bier]

Device-Tracking is a useful feature on the access layer - not sure I would enable that on the distribution layer though. On access switches with Device-Tracking enabled, you should create and then enable a separate DT profile for trunks. Because if you don't, then the MAC addresses of neighbor switches get probed by the access switch too. Waste of time. If you take a look at a switch with a default DT profile applied, you will see many MAC addresses that are foreign to this switch: 

show device-tracking database all

You create a new profile

conf t
device-tracking policy DT-TRUNK
 trusted-port
 device-role switch
 no protocol udp

And apply this profile only to trunks and port channels

interface Port-channel1
 device-tracking attach-policy DT-TRUNK

Now the database is much smaller and cleaner. You will also notice a reduction in CPU usage.

[ifabrizio]

Hi Arne and All,

Sorry for may delay but I have some urgents projects to do, before that I can go back on this issue.

I do not think anymore that this issue can be causated by SISF.

I have made same additional troubleshooting, I have tryed to enforce a new policy from ISE, to block the data traffic between the souce SGT 16 to the destination SGT 17, so I try to do a similar action as I made in my previous test, but reversing the source SGT with the destination SGT. And this time the enforcent do not work, so the traffic was not blocked.

Then I have checked the cts policy on the 4500, where the destination sgt test Pc is connected, and I see that there is no policy about these two SGTs.

The PC authenticate itself, using dot1x (TEAP Inner method EAP-TLS) the authentication is ok:

Result
User-Name PORT002.xxxxxx.com
Class CACS:01010102000004637CC785C4:ise/507490389/12623
EAP-Key-Name 37:43:b3:16:a3:4a:20:1a:06:e1:3e:0e:c2
cisco-av-pair cts:security-group-tag=0011-85

The ISE assign in the Authorization the right cts tag 17(11 hex)

But If I go on the 4500 the ip SGT mapping is not present:

4500#sh cts role-based sgt-map all
Active IPv4-SGT Bindings Information

IP Address SGT Source
============================================

The switch port where the test pc is connected is configured in this way:

interface GigabitEthernet10/32
description LABTEST
switchport access vlan xxx
switchport mode access
switchport nonegotiate
authentication event server dead action authorize
authentication event server dead action authorize voice
authentication event server alive action reinitialize
authentication host-mode multi-auth
authentication order dot1x mab
authentication priority dot1x
authentication port-control auto
authentication periodic
authentication timer reauthenticate server
authentication timer inactivity server
authentication violation restrict
device-tracking attach-policy ACCESS_PORTS
mab
dot1x pae authenticator
dot1x timeout tx-period 10
spanning-tree portfast edge
spanning-tree guard root
end

Why the 4500 (version 03.11.04.E.152-7.E4) do not learn the IP/SGT bindings?

Bye,

JF

[andrewswanson]

Does the output of "sh cts role-based sgt-map all" on the 4500E show no IP-SGT Bindings at all? Not even an INTERNAL binding for switch owned IP Addresses?

Also, does the device connected to gi1/0/32 on the 4500e appear in the device tracking database?

hth
Andy

[ifabrizio]

Hi Andre,

The "sh cts role-based sgt-map all" shows the

sh cts role-based sgt-map all
Active IPv4-SGT Bindings Information

IP Address SGT Source
============================================
1.1.1.2 2 INTERNAL
192.168.28.0/24 100 CLI
192.168.110.0/24 100 CLI
192.168.124.0/24 100 CLI
192.168.232.0/24 100 CLI
192.168.239.0/24 2 CLI
192.168.239.4 2 INTERNAL
192.168.254.0/24 100 CLI

Folows the Device tracking database:

sh device-tracking database
Binding Table has 1 entries, 1 dynamic (limit 100000)
Codes: L - Local, S - Static, ND - Neighbor Discovery, ARP - Address Resolution Protocol, DH4 - IPv4 DHCP, DH6 - IPv6 DHCP, PKT - Other Packet, API - API created
Preflevel flags (prlvl):
0001:MAC and LLA match 0002:Orig trunk 0004:Orig access
0008:Orig trusted trunk 0010:Orig trusted access 0020:DHCP assigned
0040:Cga authenticated 0080:Cert authenticated 0100:Statically assigned

Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
ARP 192.168.248.166 xxxxxxx.04f8 Gi10/32 278 0005 4s REACHABLE 301 s try 0

The SVI about the 192.168.248.160/27 is on a Cisco 9500, follows Its device tracking database:

Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
DH4 192.168.166 xxxxxxxx.04f8 Po5 278 0024 2876mn STALE try 0 518640 s
L 172.26.248.161 xxxxxxxx.e47a Vl278 278 0100 38747mn REACHABLE

IP-SGT Active Bindings Summary
============================================
Total number of CLI bindings = 6
Total number of INTERNAL bindings = 2
Total number of active bindings = 8

On the 9300 the device-tracking database is:

Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
ARP 192.168.248.11 xxxxxxxx0ca5 Gi2/0/48 275 0005 39s REACHABLE 262 s
ARP 192.168.248.1 xxxxxxxxx.e467 Fo1/1/1 275 0005 39s REACHABLE 261 s

On the 9300 the "sh cts role-based sgt-map all" shows the SGT 16 that is asociated to the orher test PC.

But there is a cts manual policy sgt association on the switch port:

interface GigabitEthernet2/0/48
description TRUSTSEC_LAB_DEV1
switchport access vlan 275
switchport mode access
switchport nonegotiate
mtu 9198
device-tracking attach-policy ISE_Track
no cdp enable
cts manual
policy static sgt 16
no propagate sgt
spanning-tree bpduguard enable
spanning-tree guard root

[ifabrizio]

I am sorry I made a mistake:

Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
DH4 192.168.166 xxxxxxxx.04f8 Po5 278 0024 2876mn STALE try 0 518640 s
L 192.168.248.161 xxxxxxxx.e47a Vl278 278 0100 38747mn REACHABLE <----I am wrong the network on this line

[andrewswanson · Accepted solution]

The document below states:

Catalyst 4500 Series Release 3.9 and later, with the introduction of VRF, an SVI is needed for L3 lookup to
derive SGT for switched traffic, and an SVI is also needed on the VLAN for the derivation of source group for L2
traffic.

https://www.cisco.com/c/dam/en/us/solutions/collateral/enterprise-networks/trustsec/policy-platform-capability-matrix.pdf

It could be worth adding an SVI for the vlan used for the trustsec client on the 4500E (think its vlan 278 from your output) - try the SVI with and without an ip address assigned.

hth

Andy

[ifabrizio]

Hi Andre,

You are right. A SVI also without ip address must be created to permit IP-SGT mapping on the 4500.

interface Vlan278
description SVI with no ip address to permit SGT-IP bindings
no ip address
end

sh cts role-based sgt-map all
Active IPv4-SGT Bindings Information

IP Address SGT Source
============================================
1.1.1.2 2 INTERNAL
192.168.28.0/24 100 CLI
192.168.110.0/24 100 CLI
192.168.124.0/24 100 CLI
192.168.232.0/24 100 CLI
192.168.239.0/24 2 CLI
192.168.239.4 2 INTERNAL
192.168.248.167 17 LOCAL <---Now the SGT/IP binding is present!
192.168.254.0/24 100 CLI

And now also the cts policy about source sgt 16 destination sgt 17 is present on the 4500:

sh cts role-based permissions from 16 to 17
IPv4 Role-based permissions from group 16:TRUSTSEC_LAB_DEV1 to group 17:TRUSTSEC_LAB_DEV2:
Permit IP-00

Finally all works now the enforcement works on the 4500:

Jun 25 14:57:47 ITAS: %EPM-6-POLICY_REQ: IP 192.168.248.167| MAC xxxxxxx.04f8| AuditSessionID 010101020000047688B6563C| EVENT APPLY
Jun 25 14:57:48 ITAS: %EPM-6-POLICY_REQ: STANDBY:IP 192.168.248.167| MAC xxxxxxxxx.04f8| AuditSessionID | EVENT APPLY
Jun 25 14:59:49.953 ITAS: CTS-coa-ev:cts_coa_ch_policy_process_msg_event, msg_id(7F19)(update-sgt)
Jun 25 14:59:49.953 ITAS: CTS-coa-ev:AAA attr name (username) type (450)
Jun 25 14:59:49.953 ITAS: CTS-coa-ev:Ignoring unsupported attr(username)
Jun 25 14:59:49.953 ITAS: CTS-coa-ev:Continue to next attr
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:AAA attr name (nas-ip-address) type (600)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:Ignoring unsupported attr(nas-ip-address)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:Continue to next attr
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:AAA attr name (Event-Timestamp) type (445)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:Ignoring unsupported attr(Event-Timestamp)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:Continue to next attr
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:AAA attr name (security-group-tag) type (893)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:AAA attr security-group-tag = 11-0063, strlen(aaa_str) = 7, attr_len = 7
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:cts_coa_ch_update_sgt : update-sgt 17-0063:TRUSTSEC_LAB_DEV2
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:update-sgt new sync_id(243) sgt_policy(628542D4) context_sync_id(0)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:Take the CoA SGT containing new gen-ID
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:
coa upate rbacl peer policy flag (0)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:
coa upate rbacl sgt_policy flag (0)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:req(1001)rcv(20880)prev(0)wait(0)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:cts_coa_sync_context_tracking: t_nodep(62C41130)sync_id(243), data_context(628542D4), type(1), ha_ok(1)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:AAA attr name (ssg-command-code) type (490)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:Ignoring unsupported attr(ssg-command-code)
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:Continue to next attr
Jun 25 14:59:49.954 ITAS: CTS-coa-ev:rec_policy(659ABC34)flag(0)sgt_policy(0)flag(0)install_policy(659ABB40)flag(C0400101)req(1)rcv(20000)prev(0)wait(0)
Jun 25 14:59:49.954 ITAS: CTS_CORE_SYNC_TAG_COA_START sync sent:
Jun 25 14:59:49.954 ITAS: CoA sync_id(243)
Jun 25 14:59:49.955 ITAS: CoA msg_id(7F19)(update-sgt)
Jun 25 14:59:49.955 ITAS: total_avpairs=1
Jun 25 14:59:49.955 ITAS: num_sent_avparirs=1
Jun 25 14:59:49.955 ITAS: avp=security-group-tag=11-0063
Jun 25 14:59:49.955 ITAS: CTS-coa-ev:rsp(ACK) err(CTS_COA_CH_SUCCESS)
Jun 25 15:00:14.229 ITAS: CTS-coa-ev:cts_coa_ch_policy_process_msg_event, msg_id(7F19)(update-sgt)
Jun 25 15:00:14.229 ITAS: CTS-coa-ev:AAA attr name (username) type (450)
Jun 25 15:00:14.229 ITAS: CTS-coa-ev:Ignoring unsupported attr(username)
Jun 25 15:00:14.229 ITAS: CTS-coa-ev:Continue to next attr
Jun 25 15:00:14.229 ITAS: CTS-coa-ev:AAA attr name (nas-ip-address) type (600)
Jun 25 15:00:14.229 ITAS: CTS-coa-ev:Ignoring unsupported attr(nas-ip-address)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:Continue to next attr
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:AAA attr name (Event-Timestamp) type (445)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:Ignoring unsupported attr(Event-Timestamp)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:Continue to next attr
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:AAA attr name (security-group-tag) type (893)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:AAA attr security-group-tag = 11-0064, strlen(aaa_str) = 7, attr_len = 7
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:cts_coa_ch_update_sgt : update-sgt 17-0064:TRUSTSEC_LAB_DEV2
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:update-sgt new sync_id(244) sgt_policy(628544F4) context_sync_id(0)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:Take the CoA SGT containing new gen-ID
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:
coa upate rbacl peer policy flag (0)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:
coa upate rbacl sgt_policy flag (0)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:req(1001)rcv(20880)prev(0)wait(0)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:cts_coa_sync_context_tracking: t_nodep(66731338)sync_id(244), data_context(628544F4), type(1), ha_ok(1)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:AAA attr name (ssg-command-code) type (490)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:Ignoring unsupported attr(ssg-command-code)
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:Continue to next attr
Jun 25 15:00:14.230 ITAS: CTS-coa-ev:rec_policy(659ABB40)flag(0)sgt_policy(0)flag(0)install_policy(659ABC34)flag(C0400101)req(1)rcv(20000)prev(0)wait(0)
Jun 25 15:00:14.230 ITAS: CTS_CORE_SYNC_TAG_COA_START sync sent:
Jun 25 15:00:14.230 ITAS: CoA sync_id(244)
Jun 25 15:00:14.230 ITAS: CoA msg_id(7F19)(update-sgt)
Jun 25 15:00:14.230 ITAS: total_avpairs=1
Jun 25 15:00:14.230 ITAS: num_sent_avparirs=1
Jun 25 15:00:14.231 ITAS: avp=security-group-tag=11-0064
Jun 25 15:00:14.231 ITAS: CTS-coa-ev:rsp(ACK) err(CTS_COA_CH_SUCCESS) <---Now the "No Valid session" error is not more present. It works wekk now. Thank you!

[andrewswanson]

Good to hear that its now working.

I ran into something similar with Cat 3ks a few years ago: Cat 3ks were layer 2 and SGT assignment was by vlan membership. As I had IP Source guard enabled, I had to create placeholder SVIs and enable IP Routing on the CAT 3Ks to get IP-SGT Binding by VLAN working. No such issues with the newer cat 9ks.

Cheers
Andy