SDA Extended Node Tips

[Mike.Cifelli]

For anyone planning to deploy an extended node in their SDA fabric here a few good-to-knows from our experience:

-Assuming you have a third party DHCP server, such as Microsoft, here is valuable information for option 43 that you will need to configure for your extended node scope in order for PnP to work:

ASCII info:

5A1N - specifies PnP

B2 - ipv4

I (dnac ip) - What ip to connect to

J = port to use

Once you have configured/confirmed that the port channel is up on the edge node you are connecting, the port-channel is configured for extended node in host onboarding, option 43 and your scope in DHCP is setup properly, physical connection to the EN it will uplink to is good you can power it up.  Note that there are other config steps such as applying/assigning ip pool to infra vn, underlay connectivity, etc.  During the boot up phase the device should pull an IP, and eventually appear in PnP inventory in DNAC.  The overview is like this:

-Device shows up in PnP 

-Device shows up in provision/inventory page

-Device gets provisioned

-Device gets added to fabric and port-channel is created

Something to note is that under Provision->Devices->Plug and Play you will see the device appear and transition through several states.  The device name will change as well and eventually finish with SN-<serial number of device>.   Once the process is finished you will see the NAD in ISE, and in the DNAC fabric topology with an 'X' for extended node.  The main thing here is to let everything go through its process without manual intervention.  

Note, In our scenario we deployed a IE-5000-12S12P-10G device so the port-channel was configured with pagp.  This will vary depending on platform type.  Also, keep an eye out on the uplink/s on the edge node because on startup/reload for the ext node the port channel will go into err-disabled.  Just shut, no shut it.  Cisco is tracking the issue.  Something else to know is that during the PnP automated process & provisioning DNAC does not push out vlan information for ip pools, excluding infra_vn, voice & critical vlans.  DNAC will push that type of information out upon provisioning interfaces in host onboarding.  Unfortunately, you have to statically assign the port, and cannot rely on an authentication template so by default the ports are setup for no authentication.  The workaround is to deploy configs via templates.  Cisco reps have mentioned that they are targeting an automated workflow in version 1.3.2. Lastly, by default all vlans are allowed on the trunk.  Hopefully this information helps anyone with extended node endeavors.  Good luck!

[dm2020]

Hi Mike,

Thanks for the info. I need to setup a couple of extended nodes myself within the next week so this is very valuable info.

Once question that I couldn't find the answer to when I have been researching - how large do you need to size the Extended Node IP pool in the Infra VN? Does it simply need to be sized to support the number of maximum number of Extended nodes that will be deployed within a given fabric site? So if I only intended to deploy 8 nodes, a subnet with a /28 will be sufficient?

Thanks

[Mike.Cifelli]

Glad to help. I think your question is determined by requirements. I would plan for growth just in case, but if your hard requirement is 8 nodes then yes a /28 will be sufficient. Just note that when assigning IP pools as either pool type extended or AP you can only assign one ip pool to each type otherwise DNAC will throw an error. Good luck & HTH!

[Cyptic man]

Thanks for the tips. Really helpful.

Will DNAC assign a static management ip address on the correct VLAN (ext node management) when the process is completed or is this a manual step that has to go through templates? In LAN automation the specific ip range is splitted up into link networks / management ip's etc. but i don't see how this is implemented here an cannot find any documentation for it.

[Mike.Cifelli]

Glad to help. For the extended node deployment you will need to create a global pool, and then a reserved pool that you will want to use for ext node management. Then under host onboarding you will assign the pool as pool type 'extended' and assign this to your INFRA_VN. Then DNAC will deploy the vlan and anycast gw info to your edge nodes. Once you start the PnP process your device will obtain a management IP from DHCP as well as the option 43 info mentioned above. Remember that the uplink/s are configured as trunk. Something else to consider is the ability to use DNAC APIs to manage some of the device attributes such as device name etc. If you simply onboard without tweaking default stuff the hostname will appear as the serial number. HTH!

[Cyptic man]

I have followed these steps. Will the dhcp request go through the INFRA_VN through that trunk or how does it work?
It seems like the extended node send it’s dhcp request native on vlan 1. Will it hit the anycast gateway anyway?

Regards
Per

[Mike.Cifelli]

So the trunk gets configured by DNAC and allows all vlans. You can use the template editor to tweak the trunk settings to make it more secure. Once you assign the ip pool to INFRA_VN the SVI on the edge node that the extended node hangs off of will look like this:
interface Vlan10xx
description Configured from Cisco DNA-Center
mac-address xxxx.xxxx.xxxx
ip address 192.x.x.1 255.255.255.0
ip helper-address <DHCP IP>
no ip redirects
ip route-cache same-interface
no lisp mobility liveness test
lisp mobility 192_x_x_0-INFRA_VN-IPV4
end
The request will traverse the trunk and rely on the helper-address assigned once hitting the AC gateway.  So yes the INFRA_VN will be used.

[TheWiz]

Hi Mike,

Great tips. Just wondering if you encountered a situation where you needed to cascade Extended Node switches. IE - from a 9300 to an IE4000, then from that IE4000 to another IE4000. I recall reading somewhere late last year that this was possible but cannot locate the document or page where I read it. Do you know if this works? Since PnP works for two layer2 hops, I thought if you plugged them into each other and then to the 9300 and initiated PnP it might discover them both and add them. It may be that what I read has changed and it is not possible to cascade extended nodes.

[jedolphi]

Hello. Cascaded fabric edges are possible, and are documented in another communities discussion, perhaps that's what you are remembering. Cascaded extended nodes are NOT possible yet. Roadmap. You can do it manually if you must, but there are a lot of tradeoffs and caveats, and it's strongly not recommended. If you must do it manually then you'll need to get an exception from your Cisco pre-sales team. Regards, Jerome

[TheWiz]

Thanks Jerome. I was pretty sure it was extended nodes I read about as the document used a car park as an example where fibre connectivity wasn't available at the furthest reaches from the FE. In any case it appears it isn't a "standard" feature yet but in my situation it is most likely necessary. What are the key tradeoffs and caveats?

[jedolphi]

This has been taken offline for discussion with appropriate Cisco representatives. Not a recommended design today, but possible with Cisco engineering interlock for critical projects. This will become a recommended design in future, roadmap pending. Jerome

[Johann Frey]

Hello,

why does the interface vlan stay on ip dhcp?

interface Vlan1029
ip address dhcp

thx

[kristoff1]

Hi Johann,

You must see an extended node like an AP. So you can place your extended node on any spot in your network and will always work.

Kind regards,

kristoff