topic ISE Distributed System - AD join issue in Network Access Control

ISE Distributed System - AD join issue

pemasirid — Mon, 11 Mar 2019 03:08:33 GMT

Hi,

We have deployed 04 ISE nodes in the following senario. (ISE ver 1.1.2.245)

1 ISE - Primary (A) Secondary (M)

2 ISE - Primary (M) Secondary (A)

3 ISE - Policy Service (PDP)

4 ISE - Policy Service (PDP)

When integrating with AD, we can only integrat to the 1 ISE only. NTP, Timezone, DNS working on all 04 boxes perfectly. We are getting the attached error while integrating AD with other ISE nodes.

In the above senario, what ISE nodes should have the AD joined, only the PDP or all 04 nodes should have joined..?

Can someone please advise. Please see the attached screenprints for the deployment and detailed error while joining to AD.

Thanks in advance.

ISE Distributed System - AD join issue

nspasov — Fri, 01 Mar 2013 17:01:19 GMT

To answer your question: You need to join your PDP nodes for user/machine authentications coming from NAD devices (Switches, Firewalls, WLCs, etc). If you want to integrate ISE admin, lobby admin to AD then you need the Admin nodes joined as well.

For the AD join error: Can you bump up the logging to "debugging" for AD and post the outputs from the log file again? Also, did you make sure that you have the proper permissions for the AD account that you are trying to use to join the nodes?

Thank you for rating!

ISE Distributed System - AD join issue

pemasirid — Sun, 03 Mar 2013 16:50:17 GMT

Hi Neno,

Below is the debug logs for AD joining. I can see the below two issues, but dont know how to find the solution..

•1) (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)
•2) SASL bind to ldap/hqv-dcs-02.xxxx.gov.qa@xxxx.GOV.QA - GSSAPI Mechanism with Kerberos error ": Cannot contact any KDC for requested realm"

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state ProbePorts complete for hqv-dcs-02.xxx.gov.qa. Elapsed time 0.014737 secs

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findkdc KDC locator for xxx.GOV.QA

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domaincontroller: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.dc.xxx.gov.qa: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findsrv FindSrvFromDns(0): _kerberos._tcp.xxxsite._sites.xxx.gov.qa

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state NST: SniffList: postfailsort=hqv-dcs-02.xxx.gov.qa, hqp-dcs-01.xxx.gov.qa

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.kerberos.keytab GetSaltFromKDC returns: xxx.GOV.QAAdmin-Asif

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.aduser getSalt update: user:admin-asif@xxx.GOV.QA salt:xxx.GOV.QAAdmin-Asif

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findkdc KDC locator for xxx.GOV.QA

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domaincontroller: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.dc.xxx.gov.qa: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findsrv FindSrvFromDns(0): _kerberos._tcp.xxxsite._sites.xxx.gov.qa

Mar 3 09:53:47 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state NST: SniffList: postfailsort=hqv-dcs-02.xxx.gov.qa, hqp-dcs-01.xxx.gov.qa

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findkdc KDC locator for xxx.GOV.QA

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domaincontroller: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.dc.xxx.gov.qa: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findsrv FindSrvFromDns(0): _kerberos._tcp.xxxsite._sites.xxx.gov.qa

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state NST: SniffList: postfailsort=hqv-dcs-02.xxx.gov.qa, hqp-dcs-01.xxx.gov.qa

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.bind.ad Performing LDAP binding with GSSAPI mechanisms to server - hqp-dcs-01.xxx.gov.qa

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findkdc KDC locator for xxx.GOV.QA

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domaincontroller: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.dc.xxx.gov.qa: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findsrv FindSrvFromDns(0): _kerberos._tcp.xxxsite._sites.xxx.gov.qa

Mar 3 09:53:49 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state NST: SniffList: postfailsort=hqv-dcs-02.xxx.gov.qa, hqp-dcs-01.xxx.gov.qa

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.osutil Module=Kerberos : SASL bind to ldap/hqp-dcs-01.xxx.gov.qa@xxx.GOV.QA - GSSAPI Mechanism with Kerberos error ": Cannot contact any KDC for requested realm" (reference base/adbind.cpp:495 rc: -1765328228)

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state NST:reportFailure: hqp-dcs-01.xxx.gov.qa

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DIAG base.bind.ad connectToServiceInDomain: Failed to connect to hqp-dcs-01.xxx.gov.qa:389: SASL bind to ldap/hqp-dcs-01.xxx.gov.qa@xxx.GOV.QA - GSSAPI Mechanism with Kerberos error ": Cannot contact any KDC for requested realm"

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domaincontroller: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.dc.xxx.gov.qa: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findsrv FindSrvFromDns(0): _ldap._tcp.xxxsite._sites.xxx.gov.qa

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state NST: SniffList: postfailsort=hqv-dcs-02.xxx.gov.qa, hqp-dcs-01.xxx.gov.qa

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.bind.ad Attempting to connect to a DC in site 'xxxsite'

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.bind.ad Connecting to hqv-dcs-02.xxx.gov.qa:389

Mar 3 09:54:04 xxx-TW-ISE-2 adjoin[27660]: DIAG base.bind.ldap 10.0.11.52:389 fetch dn="" filter="(objectclass=*)" timeout=11

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG lrpc.adobject new object:

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.bind.ad Connected root=DC=xxx,DC=gov,DC=qa, domain=xxx.GOV.QA functionality=3

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.bind.ad Address of hqv-dcs-02.xxx.gov.qa is 10.0.11.52

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.bind.ad Performing LDAP binding with GSSAPI mechanisms to server - hqv-dcs-02.xxx.gov.qa

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findkdc KDC locator for xxx.GOV.QA

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domaincontroller: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.dc.xxx.gov.qa: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG dns.findsrv FindSrvFromDns(0): _kerberos._tcp.xxxsite._sites.xxx.gov.qa

Mar 3 09:54:06 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state NST: SniffList: postfailsort=hqv-dcs-02.xxx.gov.qa, hqp-dcs-01.xxx.gov.qa

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.osutil Module=Kerberos : SASL bind to ldap/hqv-dcs-02.xxx.gov.qa@xxx.GOV.QA - GSSAPI Mechanism with Kerberos error ": Cannot contact any KDC for requested realm" (reference base/adbind.cpp:495 rc: -1765328228)

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG network.state NST:reportFailure: hqv-dcs-02.xxx.gov.qa

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.bind.ad connectToList: Failed to connect to hqv-dcs-02.xxx.gov.qa:389: SASL bind to ldap/hqv-dcs-02.xxx.gov.qa@xxx.GOV.QA - GSSAPI Mechanism with Kerberos error ": Cannot contact any KDC for requested realm"

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.osutil Module=LDAP : reconnect failed (reference base/adbind.cpp:785 rc: -11)

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG base.bind.ad Destroying binding to 'xxx.GOV.QA'

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting zonename to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting schema to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting zone to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.dc.xxx.gov.qa: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting domaincontroller to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting site to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting domain to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting prew2k.host to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting host to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG cli.adjoin Unexpected LDAP Error Connect error

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG cli.adjoin due to unexpected configuration or network error.

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG cli.adjoin Please try the --verbose option or run 'adinfo --diag' to diagnose the problem.

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting host to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: DEBUG util.settings Setting prew2k.host to

Mar 3 09:54:21 xxx-TW-ISE-2 adjoin[27660]: INFO cli.adjoin Join to domain 'xxx.gov.qa', zone 'null' failed.

Mar 3 09:54:23 xxx-TW-ISE-2 adinfo[27666]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:23 xxx-TW-ISE-2 adinfo[27668]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:33 xxx-TW-ISE-2 adinfo[28164]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:44 xxx-TW-ISE-2 adinfo[28172]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:54:54 xxx-TW-ISE-2 adinfo[28900]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:55:05 xxx-TW-ISE-2 adinfo[28905]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:55:16 xxx-TW-ISE-2 adinfo[28907]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:55:27 xxx-TW-ISE-2 adinfo[28911]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:55:38 xxx-TW-ISE-2 adinfo[28913]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:55:49 xxx-TW-ISE-2 adinfo[28920]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:56:00 xxx-TW-ISE-2 adinfo[28988]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

Mar 3 09:56:11 xxx-TW-ISE-2 adinfo[29010]: DEBUG util.except (IO) : Cannot open file /var/centrifydc/kset.domain: No such file or directory (reference util/setting.cpp:106 rc: 2)

ISE Distributed System - AD join issue

chris_day — Mon, 04 Mar 2013 05:36:34 GMT

Make sure you are joining with a user ID that has permissions to create a machine into the domain. Also make sure the subnet that you are joining the PSN devices on have AD sites and services set up so that the ISE knows what domain controller to contact that is closest to it. If you run the detailed test before joining the node that will usually tell you the problem that is getting in the way. It also helps to have your DNS entries done prior to joining the nodes, make sure you populate your PTR records as well.

ISE Distributed System - AD join issue

Naveen Kumar — Wed, 06 Mar 2013 20:32:58 GMT

Integrating Cisco ISE with Active Directory

Prerequisites:

Before you connect your Cisco ISE server with the Active Directory domain, you must check the

following:

• Ensure that your Cisco ISE server and Active Directory are time synchronized. Time in the Cisco

ISE is set according to the Network Time Protocol (NTP) server. It is recommended that you use the

NTP to synchronize time between the Cisco ISE and Active Directory.

• If there is a firewall between Cisco ISE and Active Directory, certain ports need to be opened to

allow Cisco ISE to communicate with Active Directory. Ensure that the following default ports are

open:

• If your Active Directory source has a multidomain forest, ensure that trust relationships exist

between the domain to which Cisco ISE is connected and the other domains with resources to which

you need access

• The DNS server that is configured in Cisco ISE using the ip name-server command should be able

to resolve the domain names in your Active Directory identity source. Typically, the DNS server that

is part of the Active Directory deployment is configured in Cisco ISE.

Protocol Port Number

LDAP 389 (UDP)

SMB1

1. SMB = Server Message Block

445 (TCP)

KDC2

2. KDC = Kerberos Key Distribution Center

88 (TCP)

Global Catalog 3268 (TCP), 3289

KPASS 464 (TCP)

NTP 123 (UDP)

LDAP 389 (TCP)

LDAPS3

3. LDAPS = Lightweight Directory Access Protocol over TLS/SSL

636 (TCP)

• The Active Directory username that you provide while joining to an Active Directory domain should

be predefined in Active Directory and should have any one of the following permissions:

– Add the workstation to the domain to which you are trying to connect.

– On the computer where the Cisco ISE account was created, establish permissions for creating

computer objects or deleting computer objects before you join Cisco ISE to the domain.

– Permissions for searching users and groups that are required for authentication.

After you join your Cisco ISE server to the Active Directory domain, you might still need the

permissions discussed previously to do the following:

– Join any secondary Cisco ISE servers to this domain

– Back up or restore data

– Upgrade the Cisco ISE to a higher version if the upgrade process involves backup and restore

• If your Cisco ISE deployment has multiple nodes in a distributed setup, you must first define the

Active Directory domain on the primary administration node and then explicitly join each of the

secondary policy service nodes to that domain.

• Every Cisco ISE administrator account is assigned one or more administrative roles. To perform the

operations that are described in the following procedures, you must have one of the following roles

assigned: Super Admin or System Admin

• Ensure that your Microsoft Active Directory Server does not reside behind a network address

translator and does not have a Network Address Translation (NAT) address. “

ISE Distributed System - AD join issue

nspasov — Fri, 08 Mar 2013 02:32:54 GMT

Hmm, I am not sure either. Did you turn on the highest level of logging for active directory in ISE? Also, you said that you were able to join the first node, if so were you able to pull any groups?

Re: ISE Distributed System - AD join issue

kylerossd — Fri, 08 Mar 2013 16:54:58 GMT

In addition to what everyone has posted above you can also check the following:

From the command line run a nslookup of your domain. Ensure that all NS records are your correct domain controllers and that they are active. Remove any NS records for domain controller that are no longer active.

Kyle

Re: ISE Distributed System - AD join issue

pemasirid — Sun, 10 Mar 2013 18:54:17 GMT

Hi Chris,

AD username is a member of Domain Admin. All the ISE nodes have been added to DNS and were able to resolve with hostname and IP address wise verse.

Re: ISE Distributed System - AD join issue

pemasirid — Sun, 10 Mar 2013 18:56:06 GMT

Hi Kyle,

When we give nslookup on ISE CLI, it resolve the ip address on all ISE nodes..

I have open a TAC case and we are working on this and will update once fixed.

ISE Distributed System - AD join issue

Richard Atkin — Tue, 15 Oct 2013 14:36:14 GMT

Hi,

I have exactly the same problem - did TAC ever find a solution for you?

Thanks,

Richard

ISE Distributed System - AD join issue

Ravi Singh — Thu, 17 Oct 2013 23:49:43 GMT

Please open a TAC case for the same. They will help you out.

ISE Distributed System - AD join issue

blenka — Fri, 18 Oct 2013 22:54:05 GMT

Your deployment design is correct and just verify the below activity

All nodes • View and configure system time and NTP server settings. • Install server certificate, manage certificate signing request.

Note The server certificate operations must be performed directly on each individual node. The private keys are not stored in the local database and are not copied from the relevant node; the private keys are stored in the local file system.