Need suggestion for PNP solution

[JPvisltgb]

Hi,

I need some suggestions for PnP solution. I have many branch offices.

Branch office it is simple topology Router Gi0/1 ---> Gi0/49 Switch.

Router is connected to HQ through MPLS network. DHCP server is in the HQ.

So we are planning to start PnP solution using Apic-EM for all network hardware in branch office.

Scenario:

First part:

Empty router is connected to  ISP CPE router. CPE router is relaying DHCP requests from router to central DHCP. Router getting  DHCP configuration. After that it is connecting to APIC-EM using url pnpserver.domain.com and getting all config.

Second part:

After router succesfully provisioned it become "router-on-stick". It have several subinterfaces with tagged traffic (Management, Users and Phones)

And here we have a problem:

Empty switch which is connected to router by default is trying to get DHCP to Vlan1 but router doesn't have untagged vlan on its link anymore.

We came up with one solution:

We created additional network (and new DHCP scope) for switch deployment. On Branch router on Gi0/1 we configured  IP address  and ip helper to our DHCP for untagged traffic. So switch can get ip and other config from newly created deployment DHCP scope and then it connects to pnpserver.

In total we are using additional subnet and configuration on router and DHCP server.

So maybe there is more elegant solution for that?

One more question:

After provisioning completed I have many messages in router :

Jul 12 09:47:08.305 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) UP. PID=218

Jul 12 09:47:08.306 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) DOWN(502).

What should be done to remove these messages ?

[Seb Rupik]

Hi Jegor,

Have you tired configuring the command 'pnp startup vlan X' on the router? In your case X will represent your management VLAN ID. Obviously your router needs to support the Open Plug-n-Play agent for this to work:

Cisco Open Plug-n-Play Agent Configuration Guide, Cisco IOS XE Release 3E - Cisco

Not sure about those other messages you are seeing.

cheers,

Seb.

[aradford]

Nice suggestion Seb.  "pnp startup-vlan" should work, unless it is not supported.

In terms of the messages, do you have pnp debug enabled?

What is the dhcp string you are using for option 43?

You have two options,

1) turn off pnp debug, probably by changing option 43, depending on what you put in it.

2) turn off pnp agent.  You can do put "no pnp profile XXX" where XXX= the pnp profile name

Adam

[ngoldwat]

What are you actually using for DHCP?  Windows? Blue Coat?

[JPvisltgb]

We are using Microsoft DHCP.

I don't use option 43 in DHCP. I'm using DNS method.

I found out that messages in console log while deployment config:

.Jul 13 07:58:48.881 CEST: %XML-SRVC: urn:cisco:pnp:config-upgrade XML Service(212) FAILURE(712). PID=609

Jul 13 08:54:32.598 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) DOWN(502).

Jul 13 08:55:32.598 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) UP. PID=609

Jul 13 08:55:32.600 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) DOWN(502).

Jul 13 08:56:32.601 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) UP. PID=609

Jul 13 08:56:32.605 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) DOWN(502).

Jul 13 08:57:32.606 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) UP. PID=609

Jul 13 08:57:32.608 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) DOWN(502).

Jul 13 08:58:32.610 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) UP. PID=609

Jul 13 08:58:32.612 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) DOWN(502).

And Config deployment process stuck till time out.

It is happened only on router ISR4321 with IOS-XE 03.16.05.

[aradford]

That looks like an issue with the configuration file you are pushing?

Jul 13 07:58:48.881 CEST: %XML-SRVC: urn:cisco:pnp:config-upgrade XML Service(212) FAILURE(712). PID=609

If you click on the serial number in the PnP app, what does the logging history show?

Adam

[JPvisltgb]

I double checked config and found one mistake. 

But situation the same.

Now I'm getting

.Jul 13 10:24:35.012 CEST: %XML-SRVC: urn:cisco:pnp:config-upgrade XML Service(212) OK. PID=364

.Jul 13 10:24:42.355 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) DOWN(502).

Jul 13 10:25:42.356 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) UP. PID=364

Jul 13 10:25:47.358 CEST: %XML-UPDOWN: pnp-zero-touch XML Interface(101) DOWN(502).

Apic-EM after time out shows me

ERROR_HEALTH_CHECK_TIMER_EXPIRED

Failed health check since device is stuck in non-terminal state PROVISIONING_CONFIG for more than threshold time: 0 hours, 10 minutes, 0 seconds

APIC-EM Version 1.4.1.1159.

[aradford]

do you have "aaa authorisation" commands in your configuration?

[JPvisltgb]

Yes. I saw issue that old APIC-EM had. But I'm using newer version of Apic-em and router.

[aradford]

Yes, it is addressed, but you need IOS-XE 16.3.2 (or later)

You can also address it with an EEM script work around  I have documented Network Automation with Plug and Play (PnP) – Part 7

[JPvisltgb]

Seems I found problem.

Providers CPE router have ip helper on it interface with untagged vlan to which our router is connected. Also there is tagged vlan on CPE interface.

So when empty router is connected to CPE it is getting dhcp configuration using untagged interface, let's say 192.168.1.2 255.255.255.0 GW 192.168.1.1 and etc.

But router production config is removing ip address dhcp from Gi0/0 and creating subinterface Gi0/0.99 with static address, let's say 10.0.0.2 255.255.255.0 GW 10.0.0.1 an etc.

As I understand after APIC-EM applied this configurations it is loosing connection with 192.168.1.2 because router doesn't have this IP anymore and config deployment process get stuck.

Is there any workaround ?

[aradford]

changing IP address during deployment is fine.

Device just needs to be able to communicate to APIC-EM.

Can the device communicate to APIC-EM once the new IP address is assigned?

[JPvisltgb]

That is the problem. From new IP address router can't communicate with APIC-EM. After configuration it will have several VRF's. One VRF will be management and from this vrf router will be able to communicate to APIC-EM

[aradford]

for deployment to succeed you will need to have some sort of IP connectivity post the config push.  This is used as a sanity check to make sure the provisioning succeeded.

Is the management vrf being provisioned by APIC-EM?

If so, you need to update the pnp-profile to use the vrf.   I can show you how to do this...

If the management vrf is not being configured, you will have a problem,.

Adam

[JPvisltgb]

All vrf are provisioned by APIC-Em, so please can you show how should pnp-profile look like in configuration?