Re: DNA and ISE re-integration problem

JP10 · ‎07-03-2020

Hello community,

After rebuilding and redeployment of the ISE server, we've encountered troubles with ISE re-integration in DNA-Center.

Both ISE and DNA-C are using self-signed certificates. In trusted certificates of ISE, I can see the DNA-C certificate - I've already tried to delete this cert and certificate appears back every time the integration is triggered from DNA-Center, but integration ends in a FAILED or INACTIVE state. Description of DNA-C unavailable state simply says that there was no HTTP response - before ISE rebuild/redeployment was integration working and there's no change in the network after that.

PxGrid persona is enabled on both ISE nodes. Other pxGrid connections with other services are working just fine with subscribers (FMC, SMC...). I've also tried to delete an old DNA-C client from pxGrid but the new one didn't show in pxGrid Services -> Clients at all. Feature "Automatically approve new certificate-based accounts" is enabled.

Running DNA-C 1.3.1.5 and ISE 2.4 patch 11

Do you have any advice/suggestions before opening TAC for this case? Every help will be appreciated.

willwetherman · ‎07-05-2020

Hi,

I experienced a similar issue that was due to incorrect ISE credentials. You need to make sure that the ISE CLI and GUI admin username and passwords are the same. This caught me out on a recent install as the ISE cluster was rebuilt with a different admin CLI and GUI password. When attempting the integration, the DNAC certificate was added to the ISE truststore, however a connection with pxgrid was never established and DNAC reported an error of INACTIVE.

If this isn't the issue then there are a few other things to try.

1) Regenerate the ISE CA root certificate. See the following post for details. I had the same issue for another install. Here DNAC reported the integration as ACTIVE but a subscriber was never added to pxgrid. Regenerating the ISE CA root cert fixed the issue

https://community.cisco.com/t5/cisco-digital-network/dna-1-3-1-x/td-p/4008059

2) Try restarting the DNAC pxgrid service

magctl service restart -d identity-manager-pxgrid-service

Hope this helps

JP10 · ‎07-06-2020

Hi,

Thank you for your response. Unfortunately, none of these helped. We're using the same password for GUI and CLI on ISE and these credentials are unchanged from the last running deployment / pxGrid. I've also tried to regenerate CA root certificate, but still with INACTIVE state in DNA-Center and with no subscriber in ISE pxGrid service. The restart of service didn't help.

willwetherman · ‎07-06-2020

Ok, The only other thing that I can think of that would generate a HTTP error is if ERS is not enabled on ISE. ERS should be enabled as below and DNAC should be able to reach the ISE nodes on TCP port 9060. Have you checked and verified this as well?

ISE -> Administration -> Settings -> ERS Settings

Enable ERS for Read/Write

Disable CSRF for ERS Request

Edit:

Can you also check that DNAC can reach the ISE nodes on TCP 8910 for PxGrid?

Can you also generate and post the DNAC pxgrid logs when you attempt the integration

magctl service logs -rf pxgrid

JP10 · ‎07-06-2020

ERS setting has been previously set as described. I tried to telnet from DNA-C to ISE on 9060 and that also went well. There's one routing firewall between DNA-C and ISE, but connection events didn't show any block from DNA-C to ISE server, only allow actions.

See log in attachment. Seems that there's exception error during SSL handshake - certificate mismatch

2020-07-06 09:33:18,187 | ERROR | coIseServiceImpl-Worker-1 | | c.c.e.i.u.PxgridConnectionManagerV2 | PxGridManagerV2 : Exception connecting to ISE 10.10.10.83, javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: signature check failed.

Mike.Cifelli · ‎07-06-2020

Here are a few things to be aware of:
-Try referencing this link for assistance: https://community.cisco.com/t5/networking-documents/how-to-cisco-dna-center-ise-integration/ta-p/3896410
-It is possible you could be hitting this bug: https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvp02542/?rfs=iqvred . I have had similar integration issues because of that bug.
-A couple of other logs to check for errors and to help support the TAC case:
magctl service logs -r pki-broker | grep ERROR
magctl service logs -r pxgrid | grep ERROR
Good luck & HTH!

willwetherman · ‎07-06-2020

Ok. Looking through the logs, I can see that DNAC is using a pxgrid endpoint certificate that was issued by the below subordinate CA. If you look at the valid from date of the Sub CA, you will notice that it is valid from Nov 19 so I'm assuming that this is the old Sub CA cert that was on the old ISE cluster? I believe that this should now show the new Sub CA that was generated when you renewed the ISE Root CA chain

When DNAC attempts the integration with ISE, it should request a new pxgrid endpoint certificate. You should see the below in the logs but its missing from the logs that you posted.

PxGrid endpoint certificate request: PUT /ers/config/endpointcert/certRequest HTTP/1.1

You can view the ISE issued pxgrid endpoint certificates under Administration -> Certificates -> Certificate Authority -> Issued Certificates. Can you check the valid from date of the last generated certificate and if the status of the certificate is good?

willwetherman · ‎07-06-2020

Actually I wonder if this is the issue:

| c.c.e.i.u.PxGridConfigurationUtilsV2 | Certificate zip already found in file-service. Downloading file endpointCert_1d0b2090-6a5a-466b-876e-c28439bd4895.zip |
2020-07-06 09:33:14,562 | INFO | coIseServiceImpl-Worker-1 | | c.c.e.i.u.PxGridConfigurationUtilsV2 | Retrieving file 57ebc001-78a9-44a3-9afa-6866f9a589d6 as an input stream |

| c.c.e.i.u.PxGridConfigurationUtilsV2 | File 57ebc001-78a9-44a3-9afa-6866f9a589d6 downloaded with size 4261 |

This suggests that DNAC is using an existing pxgrid endpoint certificate that is stored locally and not generating a request for a new one using the new ISE Root/Sub CA. This would also suggest why we are not seeing a new certificate request in the logs and the reason for the signature/trust error. From my understanding, DNAC should delete the old pxgrid endpoint certificate when the ISE integration is deleted. Maybe this is a bug?

I'm not sure where to go from here so I would advise raising a case with TAC.

Let us know how you get on

JP10 · ‎07-07-2020

You're right, under Administration->Certificates->Certificate Authority->Issued Certificates are only old issued certificates from old deployment. Under a recent date, I could find only certificates from another pxGrid connection e.g. FMC. Didn't realize that new DNAC cert should appear here (since I am not the one who built ISE deployment for the first time). Thank you for your time during troubleshooting, really appreciate it. Today I requested for opening TAC case, I'll let you about the solution.

MALi-786 · ‎07-12-2020

Hi,

Any update on this case with TAC? I am going to do ISE and DNAC Integration and willing to know more about it.

Please post if there is any update.