Check: #13:Expiry of registry CA cert

rasmus.elmholt · ‎09-20-2021

Hi,

I am trying to upgrade a DNAC 2.1.2.7 to 2.2.2.4 and the AURA tool says I might be hitting bug CSCvy55791/CSCvx83602.

Check: #13:Expiry of registry CA cert
Error:#01:Registry CA not found - check CSCvy55791 - file
/etc/docker/certs.d/maglev-registry.maglev-system.svc.cluster.local\:5000/registry-ca
.crt

Can anybody tell me what to do about it besides opening a TAC case?

I have verified that the Cert is not present on the DNAC. I have an old 1.3.3.9 with the cert, and another 2.2.2.3 without the Certificate.

Preston Chilcote · ‎09-20-2021

Hey Rasmus,

The remediation will require messing around in the maglev shell. This is not something we want customers doing on their own. Please let TAC help you with this.

rasmus.elmholt · ‎09-21-2021

Hi Preston,

Then how do I verify that this is actually a problem?

When I try to connect with openssl I get a valid response.

$ openssl s_client -showcerts -connect maglev-registry.maglev-system.svc.cluster.local:5000
CONNECTED(00000005)
depth=1 CN = 3eee9f37-cea0-888b-fb43-ac27d594e5d6, O = Cisco Systems, OU = Cisco DNA Center
verify return:1
depth=0 CN = maglev-registry
verify return:1
---
Certificate chain
 0 s:CN = maglev-registry
   i:CN = 3eee9f37-cea0-888b-fb43-ac27d594e5d6, O = Cisco Systems, OU = Cisco DNA Center
-----BEGIN CERTIFICATE-----
MIIEETCCAvmgAwIBAgIJAIcp3cl0JY/CMA0GCSqGSIb3DQEBCwUAMGIxLTArBgNV
BAMMJDNlZWU5ZjM3LWNlYTAtODg4Yi1mYjQzLWFjMjdkNTk0ZTVkNjEWMBQGA1UE
CgwNQ2lzY28gU3lzdGVtczEZMBcGA1UECwwQQ2lzY28gRE5BIENlbnRlcjAeFw0y
MTA3MjcxMDIxMTdaFw0yMjA3MjcxMDIxMTdaMBoxGDAWBgNVBAMMD21hZ2xldi1y
ZWdpc3RyeTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAK6e0RpLBT73
xDjakhlUw//PcbI/jTuOyI+kZOCK4epCwDDYnon6isSEkVybe9ugg1JzzIISE1Ed
b/luorF5/Q4AE5YlmUy3nup2mtiWbyw2ndSzEA2LWu2M9WsyjjJoo7EhbgrVPEXT
ZKUkm5riL0G2ZxfWLCnbLHrt8VMfW+RhUh1eLMF4LKwqpr6XdJ2luYmijGhrHheg
L7VfTYl1/Qj+uYZc9Pn3u/37w1CRgVE44yx44tiCaduSQXvu0WYiQe8Yee+nsIff
ZnqprfhOYhRwKtrrejbGPEogCiSoj6knhM0Ck//wHhR1uB8n0SfFlnnys0YZ0aPt
12CFlK1wiikCAwEAAaOCARAwggEMMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgXgMB0G
A1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjCB0gYDVR0RBIHKMIHHgg9tYWds
ZXYtcmVnaXN0cnmCHW1hZ2xldi1yZWdpc3RyeS5tYWdsZXYtc3lzdGVtgiFtYWds
ZXYtcmVnaXN0cnkubWFnbGV2LXN5c3RlbS5zdmOCKW1hZ2xldi1yZWdpc3RyeS5t
YWdsZXYtc3lzdGVtLnN2Yy5jbHVzdGVygi9tYWdsZXYtcmVnaXN0cnkubWFnbGV2
LXN5c3RlbS5zdmMuY2x1c3Rlci5sb2NhbIcEqf4AAYcEfwAAAYcECgoIC4cEZEAA
CzANBgkqhkiG9w0BAQsFAAOCAQEAbNDvEAy0dfEPcSUo03drwHxBvmxtHVlRMkQ6
Mwxq3pfQL2EKYGeDgalYSii3mIUbu3eO+wwwxcDT/U581qzcUioWee9/DrBko/t3
AVmwd+8htXAG6w4GE0bnMYBpvV95Fd8DhGy41gpfIVXt+hljlnKYgN5ietlnQcCV
c97OAru6F2bUhOghUgj+9TNhrJ7czlW9ztOBJSKUZLRgDYRCi9vpkP8TeKrypjDu
khvDgyVk9mq7d2kIwH/66CEFG1iWgGxnp+wWMjLOxxrKFoVNb1IWR7rKDw0p6kge
lWHq/UevCsm4By+XH4cOs6Vx+Ka0cRCrUGR4sIOHg1xfsXv95w==
-----END CERTIFICATE-----
---
Server certificate
subject=CN = maglev-registry

issuer=CN = 3eee9f37-cea0-888b-fb43-ac27d594e5d6, O = Cisco Systems, OU = Cisco DNA Center

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1618 bytes and written 442 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 097215E972C426D7E2AF857EF177E6E3159513328DD5B0538EEA5C42289C62DF
    Session-ID-ctx: 
    Master-Key: A4B8222C45EE28133662A10DFB781944A9D7FBB8B48EA051B05B8819CEDF1FE5CE08986B21AC444B57F9D5C7D19716AF
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - fc e3 36 4c a0 a8 79 d3-e7 b0 f1 0b bf 38 3d df   ..6L..y......8=.
    0010 - ad e8 f4 22 cf dd 2c ac-91 0f da 2c e3 fa af 05   ..."..,....,....
    0020 - 95 67 a1 7c 8f 06 06 10-5e 5b e1 5f 4e 04 9d 0f   .g.|....^[._N...
    0030 - da c8 62 79 41 d6 6c c4-46 2c ec 28 5a b7 53 0c   ..byA.l.F,.(Z.S.
    0040 - e8 c4 f6 52 d6 82 3e a2-5f b7 d2 69 fd 02 51 03   ...R..>._..i..Q.
    0050 - 06 97 77 25 9b 17 36 28-24 f4 37 ac f3 92 de 3a   ..w%..6($.7....:
    0060 - 38 a5 89 a0 23 4b 0f 19-30 59 91 93 d1 f6 b7 f4   8...#K..0Y......
    0070 - b2 2f 02 1c b3 72 58 3b-                          ./...rX;

    Start Time: 1632244478
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close

400 Bad Requestclosed

rasmus.elmholt · ‎09-21-2021

$ curl -vvvv https://maglev-registry.maglev-system.svc.cluster.local:5000/v2/
*   Trying 169.254.53.195...
* TCP_NODELAY set
* Connected to maglev-registry.maglev-system.svc.cluster.local (169.254.53.195) port 5000 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=maglev-registry
*  start date: Jul 27 10:21:17 2021 GMT
*  expire date: Jul 27 10:21:17 2022 GMT
*  subjectAltName: host "maglev-registry.maglev-system.svc.cluster.local" matched cert's "maglev-registry.maglev-system.svc.cluster.local"
*  issuer: CN=3eee9f37-cea0-888b-fb43-ac27d594e5d6; O=Cisco Systems; OU=Cisco DNA Center
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56515636c5c0)
> GET /v2/ HTTP/2
> Host: maglev-registry.maglev-system.svc.cluster.local:5000
> User-Agent: curl/7.58.0
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200 
< content-type: application/json; charset=utf-8
< docker-distribution-api-version: registry/2.0
< x-content-type-options: nosniff
< content-length: 2
< date: Tue, 21 Sep 2021 19:22:58 GMT
< 
* Connection #0 to host maglev-registry.maglev-system.svc.cluster.local left intact
{}

rasmus.elmholt · ‎09-21-2021

It seems like the CA certificate can already be found and extracted from the system:

$ openssl crl2pkcs7 -nocrl -certfile /etc/ssl/certs/ca-certificates.crt | openssl pkcs7 -print_certs -noout | grep Cisco
subject=CN = 3eee9f37-cea0-888b-fb43-ac27d594e5d6, O = Cisco Systems, OU = Cisco DNA Center
issuer=CN = 3eee9f37-cea0-888b-fb43-ac27d594e5d6, O = Cisco Systems, OU = Cisco DNA Center