cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
15429
Views
20
Helpful
10
Replies

DNA Center LAN Automation Failing due to PnP Error "Connection Error" Certificate is not valid. SSL Server ID check failed after cert-install.

bradwilson75
Level 1
Level 1

LAN Automation Failing due to PnP Error
State: Error
Onboarding State: Connection Error
History Details: NCOB02068: Certificate is not valid (ip/host name does not the match the Common Name or Subject Alternative Name): SSL Server ID check failed after cert-install

We have replaced the DNA Center server's self-signed SSL certificate, with a signed certificate, which I am assuming is causing the issue.

The subordinate certificate is in a certificate chain, with both the subordinate certificate as well as the root certificate  appended together into a single file and imported into DNA Center. This looks all good.

Does anyone know the workaround to get PnP working with DNAC having a signed certificate?

2 Accepted Solutions

Accepted Solutions

For LAN automation you need to include the DNAC virtual IP addresses in the certificate SAN as DNAC uses IP address to automate the DHCP option 43 ascii string for pnp discovery. Example DHCP pool below that DNAC provisions for LAN automation. 10.64.101.100 is the Enterprise VIP on my DNAC server. The pnp agent on the Cisco device will try and validate the presence of this IP address in the certificate SAN, and if its missing, will fail with a certificate validation error.

 

ip dhcp pool nw_orchestration_pool
network 10.99.100.0 255.255.255.192
option 43 ascii 5A1D;B2;K4;I10.64.101.100;J80;
default-router 10.99.100.1
class ciscopnp
address range 10.99.100.2 10.99.100.62

 

For standard PnP (not LAN automation), if you dont include option 43 in your DHCP scope, then Cisco devices will attempt to resolve pnpserver.localdomain (localdomain equalling the domain name set using DHCP) to discover DNAC. As this FQDN is present in your certificate, PnP certificate validation will succeed. Unfortunately with LAN automation, DNAC doesnt provide you with the option to remove option 43, nor does it configure domain-name/dns-servers under the automated DHCP scope to fallback to DNS based discovery.

 

For all of my DNAC/SD-Access deployments to date, I have had to include the DNAC IP addresses in the certificate SAN to support LAN automation or integrations such as wireless telemetry (which is deployed on the WLCs by DNAC using IP address and also uses SAN validation from my understanding)

 

As noted by Arne, for a public signed certificate, including all IP address as SAN entries will be expensive. Also more importantly, there are not many reputable certificate providers that will sign a certificate that contains private RFC1918 IP addresses anyway (as most deployments will use private IP addressing). All of my deployments so far have used internal Microsoft CA signed certificates to overcome this restriction.

 

Until LAN automation and other connections to DNAC are deployed using FQDN, then I think that IP addresses in the SAN will be mandatory (hopefully Cisco will address this in a future release of DNAC). 

 

 

 

View solution in original post

For reference, please see below openssl config file that I use for generating CSRs. I have had no issues with this configuration. This is using a 3-node DNAC cluster with Enterprise and Cluster ports only (no dedicated management/GUI port).

 

# OpenSSL.cnf
req_extensions = v3_req
distinguished_name = req_distinguished_name
default_bits = 2048
default_md = sha256
prompt = no

[req_distinguished_name]
C = <Country Code>
L = <Location>
O = <Organisation>
OU = <Organisational Unit>
CN = dnac-cluster.domain.com -> resolves to Enterprise port VIP for GUI access

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage=serverAuth,clientAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = dnac-cluster.domain.com -> resolves to Enterprise port VIP for GUI access
DNS.2 = pnpserver.domain.com -> resolves to Enterprise port VIP for PnP
IP.1 = 10.1.0.10 -> Enterprise port VIP
IP.2 = 10.1.0.11 -> Enterprise port IP of node 1
IP.3 = 10.1.0.12 -> Enterprise port IP of node 2
IP.4 = 10.1.0.13 -> Enterprise port IP of node 3
IP.5 = 10.1.1.10 -> Cluster port VIP
IP.6 = 10.1.1.11 -> Cluster port IP of node 1
IP.7 = 10.1.1.12 -> Cluster port IP of node 2
IP.8 = 10.1.1.13 -> Cluster port IP of Node 3

View solution in original post

10 Replies 10

Arne Bier
VIP
VIP

Hi @bradwilson75 

 

I am almost about to embark on a similar journey of discovery. I have not found the answer to the question.

Is your error reported on an IOS/IOS-XE device?

 

As far as I know, Cisco want you to put all the VIP FQDNs into the cert's SAN field, along with the IP address of the VIP. The way I see it, if this cert has to come from a public CA then it's going to be a bit of an expensive exercise. I don't see why Cisco is putting us through all this pain - the PNP engine on DNAC should run on a web server whose cert is signed by Cisco's own internal CA. The plug and play capable Cisco Router/Switch should then trust a Cisco factory certificate that the DNAC plug and play system uses.

 

I had a call with a Cisco guy the other day who told me that plug and play is easy. I have yet to see it.

Hi Arne,

C9300-48U-A (IOS-XE 16.12.2)

DNAC 1.3.3.7

SAN (DNAC VIP and 3 hosts, PnP):

DNS Name=dnac01.<university>.edu.au
DNS Name=dnac02.<university>.edu.au
DNS Name=dnac03.<university>.edu.au
DNS Name=dnac04.<university>.edu.au
DNS Name=pnpserver.<university>.edu.au

Yep, would have thought having PnP remain on Cisco self-signed certificate would have been better.

PnP over the last 4 years has worked well (some stack issues) started with APIC-EM (4 years ago) and DNAC (over the last year). This is the first time we have added the cert at the start of the process...

I'll be following this thread with some interest. Have you had any joy with the TAC so far? 

I assume also that the DNS entry pnpserver.<university>.edu.au points to DNAC?

Could it be that the C9300 devices don't have the Root CA cert of the CA that you got your DNAC cert from? By all accounts, the client only needs to have the Root CA cert, and it can request the intermediates from the SSL server and compute the authenticity.

No idea mate. I feel like I will be in the same boat as you very soon :)

For LAN automation you need to include the DNAC virtual IP addresses in the certificate SAN as DNAC uses IP address to automate the DHCP option 43 ascii string for pnp discovery. Example DHCP pool below that DNAC provisions for LAN automation. 10.64.101.100 is the Enterprise VIP on my DNAC server. The pnp agent on the Cisco device will try and validate the presence of this IP address in the certificate SAN, and if its missing, will fail with a certificate validation error.

 

ip dhcp pool nw_orchestration_pool
network 10.99.100.0 255.255.255.192
option 43 ascii 5A1D;B2;K4;I10.64.101.100;J80;
default-router 10.99.100.1
class ciscopnp
address range 10.99.100.2 10.99.100.62

 

For standard PnP (not LAN automation), if you dont include option 43 in your DHCP scope, then Cisco devices will attempt to resolve pnpserver.localdomain (localdomain equalling the domain name set using DHCP) to discover DNAC. As this FQDN is present in your certificate, PnP certificate validation will succeed. Unfortunately with LAN automation, DNAC doesnt provide you with the option to remove option 43, nor does it configure domain-name/dns-servers under the automated DHCP scope to fallback to DNS based discovery.

 

For all of my DNAC/SD-Access deployments to date, I have had to include the DNAC IP addresses in the certificate SAN to support LAN automation or integrations such as wireless telemetry (which is deployed on the WLCs by DNAC using IP address and also uses SAN validation from my understanding)

 

As noted by Arne, for a public signed certificate, including all IP address as SAN entries will be expensive. Also more importantly, there are not many reputable certificate providers that will sign a certificate that contains private RFC1918 IP addresses anyway (as most deployments will use private IP addressing). All of my deployments so far have used internal Microsoft CA signed certificates to overcome this restriction.

 

Until LAN automation and other connections to DNAC are deployed using FQDN, then I think that IP addresses in the SAN will be mandatory (hopefully Cisco will address this in a future release of DNAC). 

 

 

 

For reference, please see below openssl config file that I use for generating CSRs. I have had no issues with this configuration. This is using a 3-node DNAC cluster with Enterprise and Cluster ports only (no dedicated management/GUI port).

 

# OpenSSL.cnf
req_extensions = v3_req
distinguished_name = req_distinguished_name
default_bits = 2048
default_md = sha256
prompt = no

[req_distinguished_name]
C = <Country Code>
L = <Location>
O = <Organisation>
OU = <Organisational Unit>
CN = dnac-cluster.domain.com -> resolves to Enterprise port VIP for GUI access

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage=serverAuth,clientAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = dnac-cluster.domain.com -> resolves to Enterprise port VIP for GUI access
DNS.2 = pnpserver.domain.com -> resolves to Enterprise port VIP for PnP
IP.1 = 10.1.0.10 -> Enterprise port VIP
IP.2 = 10.1.0.11 -> Enterprise port IP of node 1
IP.3 = 10.1.0.12 -> Enterprise port IP of node 2
IP.4 = 10.1.0.13 -> Enterprise port IP of node 3
IP.5 = 10.1.1.10 -> Cluster port VIP
IP.6 = 10.1.1.11 -> Cluster port IP of node 1
IP.7 = 10.1.1.12 -> Cluster port IP of node 2
IP.8 = 10.1.1.13 -> Cluster port IP of Node 3

Thanks for your replies willwetherman!

We did follow the guide and had the IP Addresses included originally, and we didn't realize they were removed when the certificate was signed. The certificate signing was done by a 3rd party.

If 3rd Parties are refusing to include private IP Addresses in signed certificates, Cisco need to note that in their "Security Best Practices Guide" and not recommend it.

https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/hardening_guide/b_dnac_security_best_practices_guide.html#id_90320

Also, the PnP process should be separate and use the DNAC self-signed certificate.

Hi @willwetherman 

 

great feedback.  I was getting hung up on the fact that I read in the docs that for PnP you must sign the DNAC cert with a public CA. So that is not the case it seems?    That must mean that the PnP devices don't need to trust the DNAC certificate at all, since they would not be able to if one uses a private CA like Microsoft Windows Server CA.

 

I am confused about "LAN Automation" and "Standard PnP" - what is the difference?

 

regards

Arne

I wouldn't use a public CA, as they will not include the private IP Addresses.

DNAC downloads the certificate to devices as part of the PnP process, which is why devices will trust the certificate.

The PnP process only uses IP Addresses and not names, which is why the IP Addresses must be in the certificate.

LAN Automation uses PnP, and provisions the LAN Underlay (Layer 3) in preparation for the SDA Overlay, without the need for configuration templates.

No, a public CA signed certificate is not required for PnP or any other feature/function of DNAC as far as I'm aware. As stated by @bradwilson75, at the start of the pnp process the pnp agent downloads the root certificate in the DNAC signed certificate chain (downloads from DNAC) and installs it as a temporary trustpoint that is then used for cert validation. 

 

I agree that Cisco's docs are a little confusing as the Security Best Practices guide states 'Replace the self-signed server certificate from Cisco DNA Center with one signed by a well-known certificate authority (CA)', however in reality this cannot be a public CA as they will not accept a certificate signing request that includes private IP addresses. Internal Microsoft signed certificates have worked well for me so far.

 

Also regarding LAN automation and standard PnP, I was just trying to find a way of wording the difference between LAN automation that is used to provision the routed underlay on new devices in preparation for SD-Access (using an inbuilt template that cannot be customised) and non-LAN automation PnP which is used to provision new devices with a customised configuration (that the administrator defines using the template editor). Both are forms of PnP/day-zero provisioning if that makes sense.

 

With LAN automation, DNAC provisions a temporary DHCP pool on a 'seed switch' that is used to allow connected devices to discover DNAC. This auto provisioned DHCP pool uses option 43 with IP for DNAC discovery which is why the IP address must be in the certificate SAN

 

For standard/non-LAN automation PnP, you have a number of options to enable a device to discover DNAC such as DHCP with option 43 IP (again IP must be in the certificate SAN for this to work) or DHCP with DNS based discovery, which by excluding option 43, forces the device to perform a DNS lookup on pnpserver.localdomain. For DNS based discovery, the pnpserver.localdomain FQDN must be present in the certificate SAN.

 

As you may need to do both in a single DNAC deployment, you will need include both the IP addresses and pnpserver.localdomain in the certificate SAN

 

Hope this helps

 

EDIT

Correction on the above regarding cert validation. I have researched and the pnp agent actually downloads a Trustpool bundle from DNAC (found under System Settings -> Settings -> Trustpool) which contains a number of Cisco and third party root certificates. You will see this in the PnP logs

 

%PKI-6-TRUSTPOOL_DOWNLOAD_SUCCESS: Trustpool Download is successful

 

The following explains the purpose of the Trustpool bundle:

 

"DNA Center supports the import and storage of a PKI trustpool bundle from the Cisco cloud. This PKI trustpool bundle enables supported Cisco networking devices to authenticate the controller and its applications such as Network Plug and Play, upon the presentation of a valid third-party vendor issued certificate.Cisco DNA Center already contains a pre-installed PKI trustpool bundle. This is an ios.p7b file that only supported Cisco networking devices can unbundle and use"

 

I have checked and the chain that is used to sign the DNAC certificate is also included in this Trustpool bundle (in my case, this includes the Microsoft CA chain that I uploaded to DNAC during the cert replacement process). As the bundle also includes public CA cert chains, such as for GoDaddy, Entrust etc. it would suggest that the DNAC cert could be signed by a public CA, however as outlined in my previous posts, I dont see how this is possible if private IP addresses are required in the SAN. Cisco may need to comment on this.

 

Sorry for the long post, but I think that this has helped make more sense of the PnP cert validation process for myself as well :-)

Thanks for all the useful info. I can confirm that I managed to get Plug and Play working on my DNAC (using a CSR1000V as a test). My DNAC cert was signed by our internal Microsoft CA and it contained the VIP FQDN and the VIP IP. I used DNS as the DNAC discovery method. I have not tried with DHCP option 43 because the DNS route was sufficient for my needs.

Review Cisco Networking for a $25 gift card