PAC failing to download due to TLS

[Josh Morris]

ISE 3.1P3, C9300 6.12.3

I am trying to get SXP with enforcement configured, but the AND will not download the PAC. Error messages in the ISE live logs show a failure due to EAP-TLS handshake failure. I do not have Radius DTLS checked. 

lab-c9300-2#show cts cred
CTS password is defined in keystore, device-id = lab-c9300-2

lab-c9300-2#show run | sec aaa
aaa new-model
aaa group server tacacs+ vtygroup
 server-private 10.200.10.1 key xxx
 ip vrf forwarding Mgmt-vrf
aaa group server radius ISE_RADIUS
 server name ISE_PSN_VS
 ip vrf forwarding Mgmt-vrf
 ip radius source-interface GigabitEthernet0/0
aaa authentication login vtylogin group vtygroup local enable
aaa authentication login consolelogin local
aaa authentication login locallogin local
aaa authentication login cts-list group ISE_RADIUS local
aaa authentication dot1x default group ISE_RADIUS
aaa authorization exec default group vtygroup group tacacs+ if-authenticated 
aaa authorization network default group ISE_RADIUS 
aaa authorization network cts-list group ISE_RADIUS 
aaa authorization auth-proxy default group ISE_RADIUS 
aaa accounting update newinfo periodic 420
aaa accounting dot1x default start-stop group ISE_RADIUS
aaa accounting exec default start-stop group vtygroup
aaa accounting commands 0 default start-stop group vtygroup
aaa accounting commands 1 default start-stop group vtygroup
aaa accounting commands 15 default start-stop group vtygroup
aaa accounting commands local
aaa accounting network default start-stop group ISE_RADIUS
aaa accounting system default start-stop group ISE_RADIUS
aaa server radius dynamic-author
 client 10.200.10.1 vrf Mgmt-vrf server-key xxx
 client 10.200.9.5 vrf Mgmt-vrf server-key xxx
 client 10.200.9.6 vrf Mgmt-vrf server-key xxx
aaa session-id common
lab-c9300-2#
lab-c9300-2#
lab-c9300-2#show run | sec cts
aaa authentication login cts-list group ISE_RADIUS local
aaa authorization network cts-list group ISE_RADIUS 
cts authorization list cts-list
cts role-based enforcement
cts role-based enforcement vlan-list 1-4094
lab-c9300-2#
lab-c9300-2#
lab-c9300-2#
lab-c9300-2#show run | sec radius
aaa group server radius ISE_RADIUS
 server name ISE_PSN_VS
 ip vrf forwarding Mgmt-vrf
 ip radius source-interface GigabitEthernet0/0
aaa server radius dynamic-author
 client 10.200.10.1 vrf Mgmt-vrf server-key xxx
 client 10.200.9.5 vrf Mgmt-vrf server-key xxx
 client 10.200.9.6 vrf Mgmt-vrf server-key xxx
ip radius source-interface GigabitEthernet0/0 vrf Mgmt-vrf
radius-server attribute 6 on-for-login-auth
radius-server attribute 6 support-multiple
radius-server attribute 8 include-in-access-req
radius-server attribute 25 access-request include
radius-server dead-criteria time 30 tries 3
radius server ISE_PSN_VS
 address ipv4 10.200.10.1 auth-port 1812 acct-port 1813
 automate-tester username test probe-on
 pac key xxx
lab-c9300-2#

[andrewswanson]

Hi
Have you configured the switch with cts credentials?

cts credentials id <device-id> password <password>

hth
Andy

[Josh Morris]

Yes, I entered the creds. Thanks.

[Damien Miller]

Do you have TLS 1.0 disabled via the "Allow TLS 1.0" checkbox on this page? 

https://<ise  admin node>/admin/#administration/administration_system/administration_system_settings/SecuritySettings

[Josh Morris]

Yes, I initially disabled 1.0, 1.1. SHA1, 3DES ciphers. 

[Greg Gibbs · Accepted solution]

@Josh Morris, I suspect @Damien Miller was asking about TLS 1.0 being disabled due to it's requirement in PAC-based CTS functions as discussed in this post. Unless I'm mistaken, this is still the case.

I would suggest deleting your CTS credentials, re-enabling those ciphers, rebooting the node(s), and trying again.

[Josh Morris]

Thanks, can I just verify that when you say reboot the node, you're only referring to the node that I'm using for SXP?

[Damien Miller · Accepted solution]

When you click the checkbox to enable TLS 1.0 and save, it will force all nodes in the deployment to restart. You need to plan for this to cause an outage as the nodes reload and do it at a planned time. 

As Greg mentioned, I asked about TLS 1.0 because that's the PAC provisioning method most commonly used. There is a new alternative for an API based provisioning in 2.7+. You can read about it here but support is dependent on the network device and keeping TLS 1.0 disabled would require all NADs support this method. 
https://www.cisco.com/c/en/us/td/docs/security/ise/2-7/admin_guide/b_ise_27_admin_guide/b_ISE_admin_27_segmentation.html#id_125321

[Josh Morris]

@Damien Miller @Greg Gibbs  Good stuff, as usual, thanks. When you say the nodes are forced to reboot, are we talking all at once or in sequence? Can you think of a way that I could combine this reboot with a patch and only have a single reboot per node?

[Josh Morris]

An update on this, I tried to disable TLS 1.0/1.1 in my lab (which is running 2.7), and you get the following message before rebooting. So it appears this is just an application server reboot. Does that mean that AAA will still continue to happen in the background and that I just won't be able to manage things in the UI? I never lost a ping during this process, so I don't think the node actually rebooted.

[Greg Gibbs]

No. The application server runs various processes including RADIUS and TACACS+