10-17-2019 09:36 AM
We have had an issue a few times now so it is becoming an emergency. We had a domain controller reboot cause an issue with the policy node saying that a domain is unusable. We are running ISE 2.4 patch 8.
Here are some key points:
Any information would be appreciated.
Solved! Go to Solution.
10-21-2019 07:36 AM - edited 10-21-2019 09:01 AM
I am working with TAC and we already had the debugs enabled on the ISE policy node. We were able to see some good logs. I am waiting to hear back if we have the exact cause. Here are some example logs in case somebody runs into a similar issue in the future.
TimeStamp VERBOSE,Transaction Log Number Hidden,AdIdentitySearcher::performSearch: domain=[Some.Domain], base=[dc=Some,dc=Domain,dc=com], filter=[(&(|(objectCategory=person)(objectCategory=computer))(sAMAccountName=Bob))],lsass/server/auth-providers/ad-open-provider/ad_identity_searcher.cpp:324
TimeStamp VERBOSE,Transaction Log Number Hidden,LsaDmLdapDirectorySearch: domain=Some.Domain, dn='dc=Some,dc=Domain,dc=com', scope=2, query=(&(|(objectCategory=person)(objectCategory=computer))(sAMAccountName=Bob)),lsass/server/auth-providers/ad-open-provider/lsadm.c:4393
TimeStamp VERBOSE,Transaction Log Number Hidden,LsaDmLdapDirectorySearch: attempt=1, error=40286(LW_ERROR_LDAP_SERVER_DOWN),lsass/server/auth-providers/ad-open-provider/lsadm.c:4420
TimeStamp VERBOSE,Transaction Log Number Hidden,LocatorLookup(0x7f5091806e90): dc=Domain-Controller1, x.x.x.x-IPA,netlogon/service_locator/service_locator.c:318
TimeStamp VERBOSE,Transaction Log Number Hidden,LocatorLookup(0x7f509176c190): dc=Domain-Controller2, x.x.x.x-IPB,netlogon/service_locator/service_locator.c:318
TimeStamp ERROR ,Transaction Log Number Hidden,LsaDmConnectDomain: domain Some.Domain is offline,lsass/server/auth-providers/ad-open-provider/lsadm.c:5011
10-23-2019 07:39 AM
We ended up finding that it was a bug. It can happen when multiple domain controllers are rebooted at the same time.
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvp73385
We verified that this error was being thrown: LW_ERROR_KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN.
The issue is resolved in ISE 2.4 patch 10.
10-17-2019 11:02 PM
10-21-2019 07:33 AM
I have a TAC case open. I know that this is not Cisco TAC. I always share issues that I am having here as a way to let other people know about the issue and see if they have had it before. I do not see anything wrong with that. This was also informational and you definitely did not have the right solution.
10-21-2019 10:39 PM
10-21-2019 07:36 AM - edited 10-21-2019 09:01 AM
I am working with TAC and we already had the debugs enabled on the ISE policy node. We were able to see some good logs. I am waiting to hear back if we have the exact cause. Here are some example logs in case somebody runs into a similar issue in the future.
TimeStamp VERBOSE,Transaction Log Number Hidden,AdIdentitySearcher::performSearch: domain=[Some.Domain], base=[dc=Some,dc=Domain,dc=com], filter=[(&(|(objectCategory=person)(objectCategory=computer))(sAMAccountName=Bob))],lsass/server/auth-providers/ad-open-provider/ad_identity_searcher.cpp:324
TimeStamp VERBOSE,Transaction Log Number Hidden,LsaDmLdapDirectorySearch: domain=Some.Domain, dn='dc=Some,dc=Domain,dc=com', scope=2, query=(&(|(objectCategory=person)(objectCategory=computer))(sAMAccountName=Bob)),lsass/server/auth-providers/ad-open-provider/lsadm.c:4393
TimeStamp VERBOSE,Transaction Log Number Hidden,LsaDmLdapDirectorySearch: attempt=1, error=40286(LW_ERROR_LDAP_SERVER_DOWN),lsass/server/auth-providers/ad-open-provider/lsadm.c:4420
TimeStamp VERBOSE,Transaction Log Number Hidden,LocatorLookup(0x7f5091806e90): dc=Domain-Controller1, x.x.x.x-IPA,netlogon/service_locator/service_locator.c:318
TimeStamp VERBOSE,Transaction Log Number Hidden,LocatorLookup(0x7f509176c190): dc=Domain-Controller2, x.x.x.x-IPB,netlogon/service_locator/service_locator.c:318
TimeStamp ERROR ,Transaction Log Number Hidden,LsaDmConnectDomain: domain Some.Domain is offline,lsass/server/auth-providers/ad-open-provider/lsadm.c:5011
10-23-2019 07:39 AM
We ended up finding that it was a bug. It can happen when multiple domain controllers are rebooted at the same time.
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvp73385
We verified that this error was being thrown: LW_ERROR_KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN.
The issue is resolved in ISE 2.4 patch 10.
10-23-2019 07:43 AM
Jason,
I have found multiple discussions in the forums that are bugs that have helped me in the past. The purpose of my post was to get as many eyes on my problem as I possibly could. Sometimes, a person will reply immediately with the fix. Other times, it could be that TAC will be the final solution. And now, this thread will help somebody in the future.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide