ISE 2.6 MNT manual failover error

[KelvinT]

ISE 2.6 patch 7 2 nodes only. Primary/Secondary Hello, I have 2 ISE nodes setup as primary/secondary. When manually promoting ISE2 to primary MNT it is successfully promoted but the older logs do not appear and we get weird errors like "INVALID" for users. New logs do successfully appear in the live logs. Also we are unable to open context visibility. We get the error below: --------------------------------------------------------------------- Unable to load Context Visibility page. Ensure that full certificate chain of admin certificate is installed on Administration->System->Certificates->Trust ed Certificates. If not, install them and restart application services. Exception: None of the configured nodes are available: [{#transport#-1}{J....... ---------------------------------------------------------------------- Certs are good and trusted CA are available. Any idea?

[Colby LeMaire]

Sounds like your secondary MnT wasn't receiving everything initially or had issues.  You can try to reset the MnT database using the CLI command "application configure ise" and with option 4.  You can then try to restore your operational data from a backup if necessary.

[KelvinT]

Thanks for the reply.

This error happen in 3 different deployment.  Is there a bug?

[KelvinT]

Hello,

As I mention this error (MNT log not updating and INVALID error) happened on 3 different deployment. 

Is this a possible bug?

Has anyone else tested MNT failover using 2 nodes on ISE 2.6 patch 7?

[KelvinT · Accepted solution]

Hello,

Deregistering/regstering fixed the issue.  I have a feeling this with the reverse DNS lookup correction fix the issue.  At less in my lab.  I will see if it fix the issue with my client.

I will update when done.

[KelvinT]

Hello,

I tried this with the primary (ISE1) service stopped and secondary (ISE2) promoted to primary for both PAN and MNT.  Now I don't have anything showing in live logs for a whole day.  And authc is failing.  Interesting.

Any idea?

FYI....this is in my lab to reproduce the issue.

[Damien Miller]

The most common cause of the Context visibility service having issues is when DNS is not correctly configured. From the CLI of the node look up both A and PTR records. Both of these should return matching but opposite results.
nslookup <ise.domain.com> and nslookup <ise ip>

[KelvinT]

Hi Damien,

So I did not have the reverse lookup setup.  I have corrected this but I am still getting the context visibility error.

Do I need to reboot?

[Damien Miller]

A reload would work, but at least a stop/start of services from the CLI.

from the CLI;
application stop ise
application start ise

Keep in mind that doing this takes down all services on the node, the stop will take about 5 minutes, and the start about 10-15.  

[KelvinT]

That fixed the visibility issue.  Thanks Damien.

Still need to find out why MNT manual failover is not updating in live logs.  I will try Colby's suggestion.  I'm just wondering if its a bug because it happen on 3 separate deployment.  The last one being my lab.

Have any of you tested MNT manual failover?  Is the live log updating?  Does it have the old logs?  Any "INVALID" errors?

Thanks