cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
7427
Views
29
Helpful
26
Replies

PNP no longer working

zlantz
Level 4
Level 4

We have been using PNP for a while now. We have had some issues, but we have been slowly figuring them out one by one and giving feedback when possible. However, recently I have been running into an issue where my deployments can get to the controller and the controller updates the rule status to "Getting Device Info" and never continues after that point. 16 minutes go by and the deployment fails. I have mentioned this issue before in a long extended thread and wanted to move it over to a new thread. Old thread for reference: {PNP}Streamlining Zero-Touch

We are using option 43 for deployment:

"5A1D;B2;K5;I10.255.72.116;J443"

We were recently experimenting with pnp startup vlan and having issues as our 6Ks are still running 15.1, for now. I am also having this issue in my lab with a 6K that is running 15.4.


Output of sh pnp trace:

Switch#sh pnp trace

[03/13/17 17:14:47.254 UTC 1 398] Info: Startup config does not exists

[03/13/17 17:14:47.254 UTC 2 398] start_pnpa_discovery: PnP Discovery trial number[1]

[03/13/17 17:14:47.254 UTC 3 398] start_pnpa_discovery: Initiating PnP discovery manager

[03/13/17 17:14:47.254 UTC 4 398] pnpa_discovery_autoinstall_pid_create: waiting for autoinstall

[03/13/17 17:15:22.882 UTC 5 398] pnpa_discovery_autoinstall_pid_create:Received autoinstall complete status

[03/13/17 17:15:27.884 UTC 6 398] pnpa_autonomic_discovery: Starting autonomic discovery

[03/13/17 17:15:27.884 UTC 7 398] pnpa_autonomic_discovery: Starting autonomic discovery

[03/13/17 17:15:32.886 UTC 8 398] pnpa_disc_dhcp_option_43: op43 strict protocol: Yes, must secure: No

[03/13/17 17:15:32.886 UTC 9 398] pnpa_disc_dhcp_option_43: op43 profile pnp-zero-touch

[03/13/17 17:15:32.887 UTC A 398] pnpa_disc_dhcp_option_43: op43 ipaddr 10.255.72.116

[03/13/17 17:15:32.887 UTC B 398] pnpa_disc_dhcp_option_43: op43 transport 2

[03/13/17 17:15:32.887 UTC C 398] pnpa_disc_dhcp_option_43: transport https

[03/13/17 17:15:32.887 UTC D 398] pnpa_validate_port_type: Port is 443

[03/13/17 17:15:32.887 UTC E 398] pnpa_disc_dhcp_option_43: op43 port 443

[03/13/17 17:15:32.887 UTC F 398] pnpa_validate_ip_type: op43 iptype ipv4

[03/13/17 17:15:32.887 UTC 10 398] pnpa_dhcp_discovery:URL[http://10.255.72.116/ca/trustpool]

[03/13/17 17:15:32.907 UTC 11 398] pnp_validate_ca_bundle_url:CA bundle [http://10.255.72.116/ca/trustpool] present

[03/13/17 17:15:32.908 UTC 12 398] pnp_setup_abort:Setup abort initiated

[03/13/17 17:15:32.908 UTC 13 398] HA registry indicates presence of standby

[03/13/17 17:15:32.908 UTC 14 398] HA, config safe check [NOT OK], for configuring[try:0]

[03/13/17 17:15:32.908 UTC 15 398] HA, config safe check [NOT OK], for configuring[try:1]

[03/13/17 17:15:32.908 UTC 16 398] pnpa_ntp_sync : Unable to configure NTP Server IP[10.255.72.116]

[03/13/17 17:15:32.909 UTC 17 398] pnpa_disc_trustpool_install: NTP sync unsuccessful

[03/13/17 17:15:32.909 UTC 18 398] pnpa_trustpool_bundle_install: Config and ha safe check [NOT OK] for trustpool installation[try:0]

[03/13/17 17:15:32.909 UTC 19 398] pnpa_trustpool_bundle_install: Config and ha safe check [NOT OK] for trustpool installation[try:1]

[03/13/17 17:15:32.909 UTC 1A 398] pnpa_disc_trustpool_install: Trustpool installation unsuccessful

[03/13/17 17:15:32.916 UTC 1B 398] pnpa_disc_dhcp_option_43: op43 strict protocol: Yes, must secure: No

[03/13/17 17:15:32.916 UTC 1C 398] pnpa_disc_dhcp_option_43: op43 profile pnp-zero-touch

[03/13/17 17:15:32.916 UTC 1D 398] pnpa_disc_dhcp_option_43: op43 ipaddr 10.255.72.116

[03/13/17 17:15:32.916 UTC 1E 398] pnpa_disc_dhcp_option_43: op43 transport 2

[03/13/17 17:15:32.916 UTC 1F 398] pnpa_disc_dhcp_option_43: transport https

[03/13/17 17:15:32.916 UTC 20 398] pnpa_validate_port_type: Port is 443

[03/13/17 17:15:32.916 UTC 21 398] pnpa_disc_dhcp_option_43: op43 port 443

[03/13/17 17:15:32.916 UTC 22 398] pnpa_validate_ip_type: op43 iptype ipv4

[03/13/17 17:15:32.916 UTC 23 398] pnpa_dhcp_discovery:URL[http://10.255.72.116/ca/trustpool]

[03/13/17 17:15:32.935 UTC 24 398] pnp_validate_ca_bundle_url:CA bundle [http://10.255.72.116/ca/trustpool] present

[03/13/17 17:15:32.935 UTC 25 398] pnp_setup_abort:Setup abort initiated

[03/13/17 17:15:32.935 UTC 26 398] HA registry indicates presence of standby

[03/13/17 17:15:32.935 UTC 27 398] HA, config safe check [NOT OK], for configuring[try:0]

[03/13/17 17:15:32.935 UTC 28 398] HA, config safe check [NOT OK], for configuring[try:1]

[03/13/17 17:15:32.935 UTC 29 398] pnpa_ntp_sync : Unable to configure NTP Server IP[10.255.72.116]

[03/13/17 17:15:32.935 UTC 2A 398] pnpa_disc_trustpool_install: NTP sync unsuccessful

[03/13/17 17:15:32.936 UTC 2B 398] 10.stdby Disabled;

[03/13/17 17:15:36.730 UTC 2C 398] pnpa_trustpool_bundle_install: Config and ha safe check [OK] for trustpool installation[try:0]

[03/13/17 17:15:36.730 UTC 2D 398] pnpa_disc_trustpool_install: Trustpool installation Successful

[03/13/17 17:15:36.730 UTC 2E 398] pnp_httpc_register: PnP httpc registered

[03/13/17 17:15:36.730 UTC 2F 398] get_pnp_work_req_url: Port is 443

[03/13/17 17:15:36.730 UTC 30 398] send_work_req: HTTPS URL = https://10.255.72.116:443/pnp/HELLO

[03/13/17 17:15:36.730 UTC 31 398] send_work_req: INITIATING HTTPS SEND GET REQUEST...

[03/13/17 17:15:36.731 UTC 32 398] pnp_httpc_send_get: url https://10.255.72.116:443/pnp/HELLO

[03/13/17 17:15:36.731 UTC 33 398] pnp_httpc_send_get: HTTP SEND SUCCESS

[03/13/17 17:15:36.737 UTC 34 311] pnp_http_resp_data_alloc: PnP response data alloc 4096 bytes

[03/13/17 17:15:36.737 UTC 35 311] pnp_resp_data: request status Response data recieved, successfully

[03/13/17 17:15:36.737 UTC 36 311] pnp_resp_data: DATA STARTS HERE

[03/13/17 17:15:36.737 UTC 37 311] pnp_resp_data: DATA ENDS HERE

[03/13/17 17:15:36.737 UTC 38 311] pnp_resp_data: Status of this transaction is 200

[03/13/17 17:15:36.737 UTC 39 311] pnp_resp_data: Length of data handed over 21

[03/13/17 17:15:36.737 UTC 3A 311] pnp_resp_data: session id       : 6

[03/13/17 17:15:36.737 UTC 3B 311] pnp_resp_data: transaction id   : 4

[03/13/17 17:15:36.737 UTC 3C 311] pnp_resp_data: status_code      : 200

[03/13/17 17:15:36.738 UTC 3D 311] pnp_resp_data: status_string    : OK

[03/13/17 17:15:36.738 UTC 3E 311] pnp_resp_data: content_type     : application/json;charset=UTF-8

[03/13/17 17:15:36.738 UTC 3F 311] pnp_resp_data: content_encoding :

[03/13/17 17:15:36.738 UTC 40 311] pnp_resp_data: content_length   : 21

[03/13/17 17:15:36.738 UTC 41 311] pnp_resp_data: Location         :

[03/13/17 17:15:36.738 UTC 42 311] pnp_resp_data: Server           : Jetty(9.0.z-SNAPSHOT)

[03/13/17 17:15:36.738 UTC 43 311] pnp_resp_data: Data has not been cached

[03/13/17 17:15:36.738 UTC 44 311] pnp_http_resp_data_free: pnp response data freed

[03/13/17 17:15:36.739 UTC 45 398] pnp_httpc_send_get: HTTPS send success()

[03/13/17 17:15:36.739 UTC 46 398] send_work_req: Free url_https 1

[03/13/17 17:15:36.739 UTC 47 398] send_work_req: HTTPS SEND GET REQUEST SUCCESS

[03/13/17 17:15:36.739 UTC 48 398] 11.stdby Disabled;

[03/13/17 17:15:36.739 UTC 49 398] HA registry indicates NO standby up

[03/13/17 17:15:36.739 UTC 4A 398] 12.stdby Disabled;

[03/13/17 17:15:37.814 UTC 4B 398] HA, config safe check [OK], for configuring[try:0]

[03/13/17 17:15:37.814 UTC 4C 398] pnpa_dhcp_discovery:Configured pnp profile

[03/13/17 17:15:37.814 UTC 4D 398] pnp_setup_abort:Setup abort initiated

[03/13/17 17:15:37.814 UTC 4E 398] start_pnpa_discovery: PnP discovery process successful

[03/13/17 17:15:37.814 UTC 4F 398] pnp_autoinstall_terminate: Terminating ip autoinstall

[03/13/17 17:15:37.814 UTC 50 398] pnp_autoinstall_terminate: Terminating DHCP autoinstall

[03/13/17 17:16:20.941 UTC 51 284] 13.stdby Disabled;

[03/13/17 17:16:20.941 UTC 52 284] 14.stdby Disabled;

[03/13/17 17:16:20.941 UTC 53 284] HA registry indicates NO standby up

[03/13/17 17:16:20.941 UTC 54 284] 15.stdby Disabled;

[03/13/17 17:29:37.129 UTC 55 284] HA, config safe check [OK], for configuring[try:0]

26 Replies 26

Nick,

Shoot, I didn't catch that in the release notes. I'm going to try one of those versions, however, I believe the C881 I was testing had a compatible version, but I'll have to double check. What was the purpose of pushing up the supported version and is this going to happen often? I ask this because our environment is currently all on 3.7.3, potentially going to 3.6.6, but I would still like to know if this is going to be a common occurrence in the future.

Peng  Xu,

Here are the logs I got from the controller as I was trying a deployment. Thanks for the script.

From what I saw, it seem that after the cert is installed the deployment switch, the switch never communicates back with the controller.

Dropbox - pnp-service.log

penxu
Cisco Employee
Cisco Employee

What is device SN, Zak?

FDO2038Z03P

penxu
Cisco Employee
Cisco Employee

did you configure this device in project/pre-provision workflow? If so, what you see in APIC-EM side?

I did configure this device rule in a project and in the GUI this is all I see:

2017-03-21 13:43:03 (Eastern Daylight Time)Failed health check since device is stuck in non-terminal state FILESYSTEM_INFO_REQUESTED for more than threshold time: 0 hours, 16 minutes, 0 seconds
2017-03-21 13:26:24 (Eastern Daylight Time)Device first contact

My rule is set up as such: 2017-03-21_1446

The EULA is accepted, its just cut out of the screen shot

penxu
Cisco Employee
Cisco Employee

Is it 2-member stack, Zak? Also, can you uncheck the "device certificate" and test again?

Yes, this example is a 2 member stack, however I have tried this with a single member stack and had the same result.
With that box unchecked, I get the same result. In the logs, the failed health check fails in the same place, right after the cert gets pushed to the switch.

penxu
Cisco Employee
Cisco Employee

Hi Zak,

Can you send us the latest pnp-service.log from server and "show pnp tech" from switch when you did the test?

Still exploring this issue with Peng and in a TAC case. We are theorizing that it is an issue with how the 3 tier cert chain that we installed on our controller is validated on the switch during the PNP process.

penxu
Cisco Employee
Cisco Employee

Zak,

The latest analysis sounds like it is related name constraints field in certificate. I will keep you posted.

Thanks, just wanted to update the thread in case others run into the same issue.