Help with Cat9300 Device Sensor

[Arne Bier]

Hello 

Trying to do some device profiling for a wired 802.1X deployment.

No matter what I try, I cannot see  Device Sensor data in the RADIUS accounting.  Is there a magic command that I have forgotten?  It works so easily on Cisco WLC's - but trying to do the same on a Cat 9300 switch requires some special voodoo magic. 

I have tried the following IOS-XE releases: 16.9.2 / 16.9.3 / 16.10.1 - rebooted a few times. No joy.

I can see the Device Sensor Data in the cache, but not in the ISE tcpdump RADIUS decode.  The RADIUS Accounting Requests are sent by the switch - but they don't contain Device Sensor data.  How does one know whether the Device Sensor is correctly configured? 

#show device-sensor details

        Device-Sensor Details
--------------------------------------

Status = Enabled

Protocols:
-----------
CDP            registered  Proto Tlv Limit = 128
LLDP           registered  Proto Tlv Limit = 128
DHCP           registered  Proto Tlv Limit = 128

Protocol Filter Configuration:
---------------------------------
CDP             Include List - cdp-list
LLDP            Include List - lldp-list
DHCP            Include List - dhcp-list

I am aware that DHCP data is not in the Device Sensor, because DHCP Snooping has not yet been enabled.  But I am specifically looking for LLDP and CDP data at the moment.

e.g. a Cisco 4800 AP reports LLDP and CDP - I have truncated the output - it's quite long ...

#show device-sensor cache int fi 2/0/46
Device: 70b3.1747.3c04 on port FiveGigabitEthernet2/0/46
----------------------------------------------------------------------------
Proto Type:Name                       Len Value                       Text
LLDP     6:system-description         199 0C C5 43 69 73 63 6F 20 41  ..Cisco A
                                          50 20 53 6F 66 74 77 61 72  P Softwar
                                          65 2C 20 61 70 33 67 33 2D  e, ap3g3-
                                          6B 39 77 38 20 56 65 72 73  k9w8 Vers

#show access-session int fi 2/0/46 de
            Interface:  FiveGigabitEthernet2/0/46
               IIF-ID:  0x18B6D438
          MAC Address:  70b3.1747.3c04
         IPv6 Address:  Unknown
         IPv4 Address:  10.55.70.40
            User-Name:  70-B3-17-47-3C-04
               Status:  Authorized
               Domain:  DATA
       Oper host mode:  multi-auth
     Oper control dir:  both
      Session timeout:  N/A
    Common Session ID:  1108370A0000002C01144EAC
      Acct Session ID:  0x0000000b
               Handle:  0x3e000022
       Current Policy:  PORT-AUTH-POLICY


Local Policies:

Server Policies:


Method status list:
       Method           State
        dot1x           Stopped
          mab           Authc Success

I wonder whether I am running into a bug? The config below is correct as far as I can tell.  I can see RADIUS Interim-Updates for the Cisco AP above, but none of the Device Sensor is in there.  

Configuring Device Sensor is not trivial - we are told to configure it a certain way and we all follow blindly - nobody wants to over complicate things - we just want to CDP/LLDP/HTTP in the RADIUS please !!!

On Cisco WLC's it's two check-boxes and boom! Done.  That's how it should be.  Remember "QoS done the hard way in 1999" ? Cisco gave us Auto QoS.  We need Auto-Device-Sensor :)

aaa new-model
!
aaa group server radius ISE-GROUP
 server name xxxx
 server name yyyy

aaa authentication dot1x default group ISE-GROUP
aaa authorization network default group ISE-GROUP 
aaa accounting update newinfo periodic 2880
aaa accounting identity default start-stop group ISE-GROUP
!
aaa server radius dynamic-author
 client xxxx
 client yyyy
 
aaa session-id common
!
device-sensor filter-list cdp list cdp-list
 tlv name device-name
 tlv name address-type
 tlv name capabilities-type
 tlv name version-type
 tlv name platform-type
!
device-sensor filter-list lldp list lldp-list
 tlv name system-name
 tlv name system-description
 tlv name system-capabilities
!
device-sensor filter-list dhcp list dhcp-list
 option name host-name
 option name requested-address
 option name parameter-request-list
 option name class-identifier
 option name client-identifier
device-sensor filter-spec dhcp include list dhcp-list
device-sensor filter-spec lldp include list lldp-list
device-sensor filter-spec cdp include list cdp-list
device-sensor notify all-changes

device-tracking tracking auto-source override
device-tracking tracking retry-interval 60
!
device-tracking policy IPDT_POLICY
 security-level glean
 no protocol ndp
 no protocol udp
 tracking enable reachable-lifetime 10
!
authentication critical recovery delay 1000
access-session attributes filter-list list DS_LIST
 vlan-id
 cdp
 lldp
 dhcp
 http
access-session authentication attributes filter-spec include list DS_LIST
access-session accounting attributes filter-spec include list DS_LIST
access-session monitor
access-session acl default passthrough

dot1x system-auth-control
dot1x critical eapol

lldp run


template PORT-AUTH-TEMPLATE
 spanning-tree portfast
 dot1x pae authenticator
 dot1x timeout tx-period 7
 dot1x max-reauth-req 3
 switchport access vlan 123
 switchport mode access
 switchport voice vlan 124
 mab
 access-session closed
 access-session port-control auto
 authentication periodic
 authentication timer reauthenticate server
 service-policy type control subscriber PORT-AUTH-POLICY
 description ** DOT1X Enabled Port **

interface FiveGigabitEthernet2/0/46
 device-tracking attach-policy IPDT_POLICY
 source template PORT-AUTH-TEMPLATE
 spanning-tree portfast


ip access-list extended CRITICAL_AUTH_ACL
 remark ISE down permit all access
 permit ip any any
ip access-list extended IPV4_CRITICAL_ACL
 permit ip any any
!
!
!
radius-server attribute 6 on-for-login-auth
radius-server attribute 8 include-in-access-req
radius-server attribute 25 access-request include
radius-server attribute 31 mac format ietf upper-case
radius-server attribute 31 send nas-port-detail mac-only
radius-server dead-criteria time 10 tries 3
radius-server deadtime 15
!
radius server xxxx
 address ipv4 xxxx auth-port 1812 acct-port 1813
 automate-tester username NAD-Tester ignore-acct-port probe-on
 key xxxx
!
radius server yyyy
 address ipv4 yyyy auth-port 1812 acct-port 1813
 automate-tester username NAD-Tester ignore-acct-port probe-on
 key yyyy
!

[Damien Miller]

I was able to get a 9300-48UXM in my lab and I was hoping to have different results than you. I am also seeing that device sensor data is not being sent to ISE with IBNS 2.0. I am currently testing on 16.9.3 because it is a recommend release per the last route switch TAC time call.

I believe I mentioned to you before that I was also having similar issues with this on the 3850 platform and 16.6.6, in my case it would work and break with reloads. The TAC engineer on my case was was not able to get device sensor to work on the 3850 and 16.6.6 on their end. This was good and bad news, good being that there was indeed a problem then, bad because it wasn't working for TAC either. They just swapped TAC engineers on me before I could get any closure on the issue and I'm now waiting again. This just means that I do not have a specific bug logged.


My config is very close to Hari's prescriptive ISE deployment guide, and I have poured over the white pages and bug tracker looking for something I may have missed. I am concluding that this is an undocumented bug that we are stumbling on. There must be quite a few deployments impacted by this and it's probably being masked by SNMP query polling.

I'm going to continue testing various combinations and I noticed 16.9.4 released a couple weeks ago, so I will also give that a try. I am certainly pretty disappointed that this isn't working since it is basic core functionality for IBN. It's as if it wasn't tested or is some sort of regression that wasn't caught. I'm hoping there is a quick resolution somewhere. It shouldn't be this hard to get something working that Cisco suggests we all use anyways.

[Arne Bier]

Thanks @Damien Miller - I also started off with Hari's config and then took a few things from your config too.  I am astounded that getting device sensor to work is so complicated. On Cisco WLC it's one check box - on/off.  Simple.  And to add to that, when we drink the cool aid and sell this idea of profiling to our customers who have invested heavily in Cat9K line, we end up with egg on our face.  I am building MAB Groups now and shoving all these devices into MAB Groups just to get devices online.  I wonder how others are managing to profile their devices?  DHCP sent to PSN directly? SNMP polling? It still confuses me a bit because we have many different ways of getting profiling data - I don't like the active probing method but it would appear to be a way out of this dilemma.

1st prize - use Device Sensor to gather DHCP/LLDP/CDP/HTTP for "free" - scalable and least amount of hassle for ISE

2nd prize - no Device Sensor - tell ISE to poll SNMP MIB of switch on linkup trap - and configure ip helper to send DHCP to a pair of PSN's - does that effectively achieve the same as Device Sensor?  I am not clear on the feature parity of these two methods - and the relative drawbacks of the second method.

[Damien Miller · Accepted solution]

My testing has demonstrated SNMP query works via two methods, interval polling and after successful authentication. I do not have to send SNMP traps or logs to ISE for ISE to SNMP query the switch.

What I observe is that when an endpoint passes authentication, ISE then SNMP polls the switch gathering device sensor cache/attributes, if a new profile hits once ISE has collected device sensor cache data via SNMP then I have it set to send a COA reauth.

I am collecting DHCP info via helper addresses since snooping is not enabled.

Device sensor via radius is still broken in this flow, but at least it seems like snmp query is a viable temporary workaround until I can get it working.  

[Damien Miller]

Since you and I spoke about this offline, I am going to update for others too.

16.9.4 fixed the device sensor issue for me that was very broken in 16.6.6 and 16.9.3.

[Arne Bier]

Hi @Damien Miller 

Correct - 16.9.4 allowed the Device Sensor feature to work. But it has broken 802.1X - I am getting tracebacks in the console terminal. There is a TAC case to investigate why. It all worked for a few hours and then broke all on its own. I no longer see any RADIUS requests when I bounce a port.

I just want a version of IOS that works. 

[edwardwaithaka]

Hi @Arne Bier 

Did you manage to sort the 802.1x issues after the upgrade.

I have a customer with 16.8.1a facing the device-sensor issue and would like to upgrade to something that works.

[Arne Bier]

Hi @edwardwaithaka 

16.9.4 finally worked. :-)

[edwardwaithaka]

What was the solution to the 802.1x issue?

[Arne Bier]

Apologies. I didn’t see the other part of the question. 802.1X broke because of a config on the port. I was trying to do port profiles and when I pushed a trunk config to the port it killed the switch. At that time we needed trunk ports for flex APs.  Short answer: we stopped using trunk mode on all 802.1X ports and using regular access ports. All working perfectly. fingers crossed it works into the foreseeable future. 

[edwardwaithaka]

Great, thanks

[Damien Miller]

Agreed, 16.9.4+ and 16.12.1.

I had opened a TAC case on 16.6.6, the bug will not be fixed in the 16.6 train.