topic Re: Identity Services Engine Unusable Domain Caused By Windows Domain Controller Reboot in Network Access Control

Identity Services Engine Unusable Domain Caused By Windows Domain Controller Reboot

Alex Pfeil — Thu, 17 Oct 2019 16:36:37 GMT

We have had an issue a few times now so it is becoming an emergency. We had a domain controller reboot cause an issue with the policy node saying that a domain is unusable. We are running ISE 2.4 patch 8.

Here are some key points:

Domain Controller has updates applied and reboots, comes online within 30 seconds.
Identity Services Engine stops authenticating clients with the following error: 24367 Skipping unusable domain - "domain name", Server not found in Kerberos database. This causes issues for hundreds of users.
For one outage, we rebooted an ISE Policy node and it re-connected to a different domain controller and started working.
For the second outage, authentications started working after approximately 2 hours with no Policy node reboot.

Any information would be appreciated.

Re: Identity Services Engine Unusable Domain Caused By Windows Domain Controller Reboot

Jason Kunst — Fri, 18 Oct 2019 06:02:38 GMT

Please see this post for general guidance on using community,
https://community.cisco.com/t5/security-documents/how-to-ask-the-community-for-help/ta-p/3704356

Re: Identity Services Engine Unusable Domain Caused By Windows Domain Controller Reboot

Alex Pfeil — Mon, 21 Oct 2019 14:33:53 GMT

I have a TAC case open. I know that this is not Cisco TAC. I always share issues that I am having here as a way to let other people know about the issue and see if they have had it before. I do not see anything wrong with that. This was also informational and you definitely did not have the right solution.

Re: Identity Services Engine Unusable Domain Caused By Windows Domain Controller Reboot

Alex Pfeil — Mon, 21 Oct 2019 16:01:50 GMT

I am working with TAC and we already had the debugs enabled on the ISE policy node. We were able to see some good logs. I am waiting to hear back if we have the exact cause. Here are some example logs in case somebody runs into a similar issue in the future.

TimeStamp VERBOSE,Transaction Log Number Hidden,AdIdentitySearcher::performSearch: domain=[Some.Domain], base=[dc=Some,dc=Domain,dc=com], filter=[(&(|(objectCategory=person)(objectCategory=computer))(sAMAccountName=Bob))],lsass/server/auth-providers/ad-open-provider/ad_identity_searcher.cpp:324

TimeStamp VERBOSE,Transaction Log Number Hidden,LsaDmLdapDirectorySearch: domain=Some.Domain, dn='dc=Some,dc=Domain,dc=com', scope=2, query=(&(|(objectCategory=person)(objectCategory=computer))(sAMAccountName=Bob)),lsass/server/auth-providers/ad-open-provider/lsadm.c:4393

TimeStamp VERBOSE,Transaction Log Number Hidden,LsaDmLdapDirectorySearch: attempt=1, error=40286(LW_ERROR_LDAP_SERVER_DOWN),lsass/server/auth-providers/ad-open-provider/lsadm.c:4420

TimeStamp VERBOSE,Transaction Log Number Hidden,LocatorLookup(0x7f5091806e90): dc=Domain-Controller1, x.x.x.x-IPA,netlogon/service_locator/service_locator.c:318

TimeStamp VERBOSE,Transaction Log Number Hidden,LocatorLookup(0x7f509176c190): dc=Domain-Controller2, x.x.x.x-IPB,netlogon/service_locator/service_locator.c:318

TimeStamp ERROR ,Transaction Log Number Hidden,LsaDmConnectDomain: domain Some.Domain is offline,lsass/server/auth-providers/ad-open-provider/lsadm.c:5011

Re: Identity Services Engine Unusable Domain Caused By Windows Domain Controller Reboot

Jason Kunst — Tue, 22 Oct 2019 05:39:18 GMT

Alex thank you, it would be good to state what your intent is so we can close out the solution. If its just information sharing then please post as a document or blog and not something needing an actual solution. We are trying to manage and make sure everything is covered.

Re: Identity Services Engine Unusable Domain Caused By Windows Domain Controller Reboot

Alex Pfeil — Wed, 23 Oct 2019 14:39:16 GMT

We ended up finding that it was a bug. It can happen when multiple domain controllers are rebooted at the same time.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvp73385

We verified that this error was being thrown: LW_ERROR_KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN.

The issue is resolved in ISE 2.4 patch 10.

Re: Identity Services Engine Unusable Domain Caused By Windows Domain Controller Reboot

Alex Pfeil — Wed, 23 Oct 2019 14:43:30 GMT

Jason,

I have found multiple discussions in the forums that are bugs that have helped me in the past. The purpose of my post was to get as many eyes on my problem as I possibly could. Sometimes, a person will reply immediately with the fix. Other times, it could be that TAC will be the final solution. And now, this thread will help somebody in the future.