topic Re: DNA and ISE integration problem in Cisco Catalyst Center

DNA and ISE integration problem

Diana Karolina Rojas — Sat, 09 Mar 2019 01:27:34 GMT

Hello community,

We are having troubles doing the DNA and ISE integration, when we add the ISE in the DNA center all looks good and the device show the "ACTIVE" state, but I never see the client in pxGrid services, so ISE nevers see the DNA center as a client. In the Trusted Certificate from ISE I can see the DNA Center certificates, you have to know that DNA Center and ISE are in different domains, does that means?

All help will be appreciated.

Best Regards,

Re: DNA and ISE integration problem

Aravind Ravichandran — Fri, 28 Sep 2018 22:18:56 GMT

Hi Diana,

Please check the pxgrid settings whether it is kept as auto approval or not. Are you using self signed certificate or CA signed certificate.If CA signed,Both ISE & DNA-C certificate should be signed by same CA.Also please share the DNA-C and ISE version.

Re: DNA and ISE integration problem

AndiBuchmann157 — Mon, 08 Oct 2018 08:27:42 GMT

Did you double check that pxgrid services are enabled in your ISE settings?

Re: DNA and ISE integration problem

tom.barat@dimensiondata.com — Tue, 23 Oct 2018 15:12:31 GMT

I have the same issue, in my case :
- pxGRID is not in auto approval.
- On AAA server configuration in DNA, DNA briefly displays an error saying "expected trust phrase was not received" and the ISE server's status is "FAILED" until i refresh the page, then the status is "ACTIVE", as if the operation worked.
- In System 360 page, i see ISE as available, but pxGRID as unavailable.

- ISE uses a self-signed certificate for pxGRID.

- DNA uses the default certificate, no changes were made.

- ISE version 2.3

- DNA version 1.1.7

thank you in advance,
regards

Re: DNA and ISE integration problem

tom.barat@dimensiondata.com — Tue, 23 Oct 2018 15:07:39 GMT

I have the exact same issue, in my case pxgrid was enabled on ISE.

Re: DNA and ISE integration problem

Gaur Samal — Tue, 23 Oct 2018 15:38:42 GMT

If you see ISE server under System Settings -> Settings -> Authentication and Policy servers as ACTIVE , but under System 360 page you see PXgrid unavailable then you may want to raise a TAC case to fix this issue. there few known issues which TAC can help you with resolving those.

Re: DNA and ISE integration problem

tom.barat@dimensiondata.com — Wed, 24 Oct 2018 06:39:23 GMT

Unfortunately, opening a TAC case is not an option for me because this is a lab environment where the DNA center (and all related licenses) belong to a client, so i cannot use them for my own purpose. Additionally, the ISE appliance belongs to my organization but it is only used for lab environment so it is not licensed for technical support.

Re: DNA and ISE integration problem

Gaur Samal — Wed, 24 Oct 2018 06:44:25 GMT

If they have DNAC purchased, they by default have solution support provided. You should have a contract ID and serial number of DNAC appliance and can open case.

In any case, could you please run command : “magctl service logs -r pxgrid > pxgrid.log” in DNAC CLI. This will create a file named as pxgrid.log under path : /home/maglev/pxgrid.log

Please provide this file.

Re: DNA and ISE integration problem

tom.barat@dimensiondata.com — Wed, 24 Oct 2018 09:04:30 GMT

I will see what i can do about the TAC case. In the meantime, here is the file you requested.

EDIT : The file is about 20k lines long, so i inspected it in order to save time. As far as i can see, my issue probably comes from this error :

2018-10-23 13:33:11,437 |  ERROR | pool-1-thread-5           | identity-manager-pxgrid-service | c.c.e.i.u.PxGridConfigurationUtils | An error occurred while retrieving PxGrid endpoint certificate. Request: PUT https://10.168.196.30:9060/ers/config/endpointcert/certRequest HTTP/1.1, Response: HttpResponseProxy{HTTP/1.1 404 Not Found [Cache-Control: no-cache, no-store, must-revalidate, Expires: Thu, 01 Jan 1970 00:00:00 GMT, Set-Cookie: JSESSIONIDSSO=3EC28AEC8FF84BDA51E91F8A8BC64DF0; Path=/; Secure; HttpOnly, Set-Cookie: APPSESSIONID=E94B33C31882712A6891E5C12E615CFE; Path=/ers; Secure; HttpOnly, Pragma: no-cache, Internal Server Error: Unexpected Exeption:: 500, Content-Length: 0, Date: Tue, 23 Oct 2018 13:33:11 GMT, Server:  ] [Content-Length: 0,Chunked: false]} | 
2018-10-23 13:33:11,438 |  ERROR | pool-1-thread-5           | identity-manager-pxgrid-service | c.c.e.i.u.PxGridConfigurationUtils | Error retrieving PxGrid endpoint certificate from ISE | 
com.cisco.enc.identitymanager.exceptions.IdentityManagerException: 
	at com.cisco.enc.identitymanagerpxgrid.utils.PxGridConfigurationUtils.downloadPxGridEndPointCert(PxGridConfigurationUtils.java:350) [classes/:na]
	at com.cisco.enc.identitymanagerpxgrid.utils.PxGridConfigurationUtils.getAPICEMKeyStoreFilePath(PxGridConfigurationUtils.java:157) [classes/:na]
	at com.cisco.enc.identitymanagerpxgrid.utils.PxgridConnectionManager.createConfig(PxgridConnectionManager.java:160) [classes/:na]
	at com.cisco.enc.identitymanagerpxgrid.utils.PxgridConnectionManager.createScalableGroupGridConnection(PxgridConnectionManager.java:144) [classes/:na]
	at com.cisco.enc.identitymanagerpxgrid.utils.PxgridConnectionManager.registerScalableGroupAndSessionSubscriberWithISE(PxgridConnectionManager.java:119) [classes/:na]
	at com.cisco.enc.identitymanagerpxgrid.utils.PxgridConnectionManager.establishConnectionWithIse(PxgridConnectionManager.java:88) [classes/:na]
	at com.cisco.enc.identitymanagerpxgrid.impl.CiscoIseServiceImpl.connectWithIseAndPullData(CiscoIseServiceImpl.java:268) [classes/:na]
	at com.cisco.enc.identitymanagerpxgrid.impl.CiscoIseServiceImpl.access$000(CiscoIseServiceImpl.java:46) [classes/:na]
	at com.cisco.enc.identitymanagerpxgrid.impl.CiscoIseServiceImpl$1.run(CiscoIseServiceImpl.java:244) [classes/:na]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_112]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_112]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
Exception in thread "pool-1-thread-5" com.cisco.enc.identitymanager.exceptions.IdentityManagerException: Error retrieving PxGrid endpoint certificate from ISE
	at com.cisco.enc.identitymanagerpxgrid.utils.PxGridConfigurationUtils.downloadPxGridEndPointCert(PxGridConfigurationUtils.java:373)
	at com.cisco.enc.identitymanagerpxgrid.utils.PxGridConfigurationUtils.getAPICEMKeyStoreFilePath(PxGridConfigurationUtils.java:157)
	at com.cisco.enc.identitymanagerpxgrid.utils.PxgridConnectionManager.createConfig(PxgridConnectionManager.java:160)
	at com.cisco.enc.identitymanagerpxgrid.utils.PxgridConnectionManager.createScalableGroupGridConnection(PxgridConnectionManager.java:144)
	at com.cisco.enc.identitymanagerpxgrid.utils.PxgridConnectionManager.registerScalableGroupAndSessionSubscriberWithISE(PxgridConnectionManager.java:119)
	at com.cisco.enc.identitymanagerpxgrid.utils.PxgridConnectionManager.establishConnectionWithIse(PxgridConnectionManager.java:88)
	at com.cisco.enc.identitymanagerpxgrid.impl.CiscoIseServiceImpl.connectWithIseAndPullData(CiscoIseServiceImpl.java:268)
	at com.cisco.enc.identitymanagerpxgrid.impl.CiscoIseServiceImpl.access$000(CiscoIseServiceImpl.java:46)
	at com.cisco.enc.identitymanagerpxgrid.impl.CiscoIseServiceImpl$1.run(CiscoIseServiceImpl.java:244)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: com.cisco.enc.identitymanager.exceptions.IdentityManagerException: 
	at com.cisco.enc.identitymanagerpxgrid.utils.PxGridConfigurationUtils.downloadPxGridEndPointCert(PxGridConfigurationUtils.java:350)
	... 11 more

This error appears several times starting line ~17000. Since it looks directly related to communication and certificate exchange between ISE and DNA, it is probably the source of the issue.

Any guidance to fix this certificate error would be greatly appreciated.

Re: DNA and ISE integration problem

Gaur Samal — Wed, 24 Oct 2018 11:33:25 GMT

Please run below command :

“magctl service restart -d pxgrid” on DNAC CLI.

Before you run this command, please check if ISE is ACTIVE under AAA settings.

After running this command, wait for 3 mins and run command “magctl service logs -rf pxgrid > pxgrid_re.log”
Wait for 15-20 mins. Do control+C on CLI.
This will create a file /home/maglev/pxgrid_re.log. please provide that file.

Check under ISE->Pxgrid services if you see the subscriber.

If not, then go to ISE instance under settings->system settings->Authentication and policy servers.
Click the instance, go to edit, and update the password ( no need to change just renter the password ) and click apply.

Quickly go to CLI and run command “magctl service logs -rf k-d > k-d.log”
Wait for 10 mins and then press cntl+C, this will create file /home/maglev/k-d.log. please provide the file.
Make sure ISE become active. After that check in ISE->Pxgrid services if you see the subscriber.

Thanks,
Gaur

Re: DNA and ISE integration problem

tom.barat@dimensiondata.com — Wed, 24 Oct 2018 15:20:26 GMT

I followed the instructions you provided.

After updating the password in AAA settings in DNA, i got the same error as before:

Error establishing trust with ISE: Expected phrase [Trust establishment completed successfully] wasn't received from ise

After the update, the server's status displayed in DNA AAA settings is FAILED. It never changed back to ACTIVE.

At no point during the process did i see the DNAC subscriber appear in ISE pxGRID services.

In DNA System 360 page, the situation is still the same :

- ISE appears "available"

- PXGRID appears "unavailable"

Please find the file in attachments, which i will also inspect as soon as possible.

Re: DNA and ISE integration problem

Diana Karolina Rojas — Wed, 24 Oct 2018 20:59:13 GMT

Hello Tom,

In our case the only thing that help us in this issue was regenerate the root certificate form ISE, after that we restart the pxgrid process in DNAC and reloaded the ISE, after that we achieve the conectivity.

Best Regards,

Re: DNA and ISE integration problem

tom.barat@dimensiondata.com — Mon, 29 Oct 2018 11:00:08 GMT

I have tried the following:

- Generate self signed root CA

- Generate rootCA-signed Intermediate CA

- Generate ISE/PXGRID cert signed by intermediate CA, with Subject Alternative Name (DNS + IP)

- Generate DNA cert signed by (same) intermediate CA, with subject alternative names (DNS + IP)

- Import chain to ISE and use the ISE subject cert for PXgrid only

- Import new DNA subject cert into DNA

- Delete and create again the AAA server on DNA side.

Now the AAA server is created without any errors, and shows up as ACTIVE. But the client does not appear on ISE side, and PXGRID is still shown as unavailable on DNA System 360 page.

The issue does not appear to be fixed. DNA also displays a warning on the new certificate i imported, because it is not aware of the root and intermediate CA (for some reason only one cert can be imported in DNA, whereas the whole chain can be imported in ISE).

Any further assistance would be greatly appreciated.

Re: DNA and ISE integration problem

Aravind Ravichandran — Mon, 29 Oct 2018 11:45:10 GMT

Hi,

Try this additional things as well.

-Delete the old subscriber name in ISE.

-Create a new subscriber name(which never used before) in DNA-c

-If Proxy is used in ISE, Add DNA-C IP address in bypass list.

While integration, DNA-c certificate will automatically come under trusted certificate.

-Aravind

Re: DNA and ISE integration problem

tom.barat@dimensiondata.com — Mon, 29 Oct 2018 12:40:28 GMT

Hello,

I cannot delete DNA subscriber names from ISE because the subscriber name never appeared through all my retries. However, i try to pick a different name each time in case they are still stored somewhere hidden.

The ISE and DNA are on the same network so there should not be any proxy in use. Moreover, the ISE is used for test environments so it is not hardened with best practices = Access is not restricted to a specific set of IPs.

I deleted the old DNA certificate from ISE. Now when i try to add ISE as a AAA server in DNA i get an error that says the failure phrase was received, and that i should check the certificate chain.

However it is not specified if the issue comes from the DNA certificate chain ( in which case i cannot fix the issue since i can only import one cert, i cannot import the whole chain) or from the ISE ( in which case i do not know what the issue with the certificate chain might be since i see the three certs ( Root, Intermediate, and PXGRID) in trusted certificates).

Re: DNA and ISE integration problem

Gaur Samal — Mon, 29 Oct 2018 19:22:37 GMT

Hi Tom,

This definitely need live troubleshooting, please open up a TAC case or ask your Cisco partner/SE to open up for you.
TAC should be able to help you with this issue.

Thanks,
Gaur

Re: DNA and ISE integration problem

tom.barat@dimensiondata.com — Fri, 02 Nov 2018 16:11:43 GMT

Hello,

As i previously said i was not able to open a TAC case for this issue because our ISE appliance is not under support contract, and the DNA center i am using belongs to a client, therefore i do not own the associated support contract and cannot use it for my own purposes.

Nevertheless i kept working on this issue. I reinstalled an ISE virtual appliance from scratch and was able to make the connection between DNA and the new ISE without any issues.

We currently have 2 theories regarding the initial issue's source :

- first, reinstalling a new ISE appliance allowed me to notice that our first ISE's NTP settings were all messed up (it currently believes we are July 4th). This could very well be the problem's source. In any case, poor NTP settings probably made things worse and potentially made the issue more difficult to troubleshoot.

- second, our first ISE has been used for a long time for all our test/lab/poc purposes. This means a lot of configuration was done by many different people and there might be "poor" configuration in use in the appliance. Specifically, the appliance was downgraded and upgraded many times in many different versions and patch level.

Thank you again for the help you provided.

Re: DNA and ISE integration problem

John David — Mon, 11 Mar 2019 06:53:04 GMT

yes same here, i was also looking for such kind of issues.