NDFC POAP over L3

[bunjiega]

Is there a trick to getting POAP working on NDFC when the switch mgmt0 network is reachable over L3 to the ND mgmt network? I have tried everything that all the guides say.....

This is the briefing of what my network looks like:

I DO have a route added in the ND mgmt table back to the 10.0.0.0/24. I have also tried to use the IP address assigned to POAP from the 'external service pools' for the helper address.

The switches just say: "POAP DHCP discover phase failed". If I give one a static address, it has reachability, so something on the ND isn't working like I was expecting. Which IP to use is not clear, esp in the cluster config.

Some of my sources:

https://www.cisco.com/c/en/us/td/docs/dcn/ndfc/1201/configuration/fabric-controller/cisco-ndfc-fabric-controller-configuration-guide-1201/switches.html#concept_wdp_vkk_1rb

https://www.ciscolive.com/c/dam/r/ciscolive/global-event/docs/2022/pdf/BRKDCN-1119.pdf

https://www.cisco.com/c/en/us/products/collateral/cloud-systems-management/prime-data-center-network-manager/guide-c11-741508.html

Thanks!

[Tarakesh Jetti]

Hi, 

Thanks for the question.

Can you give us the Platform information:

Nexus Dashboard version:?

NDFC version:?

Cluster Form Factor?

Cisco Nexus 9000 switches software version:?

You can also learn more about Cisco NDFC through our live Ask the Experts (ATXs) session. Check out this ATXs Resources [https://community.cisco.com/t5/data-center-and-cloud-knowledge/cisco-aci-ask-the-experts-resources/ta-p/4394491] to view the latest schedule for upcoming sessions, as well as the useful references, e.g. online guides, FAQs.

Thanks and regards,

Tarakesh Jetti,

Customer Success Specialist - CX Team.

[bunjiega]

Hi,

ND Version = 2.2(2d)
NDFC Version = 12.1.1e
Cluster is 3 physical nodes all at one site.
All switches are new Nexus 9000

POAP is working at the local site (where the switches and ND are all on the same subnet). However, at the remote site, which is a different subnet, I have IP reachability, and have ip-helpers of the 3 ND mgmt IPs and have that fabric enabled for POAP, but the switches aren't showing up in NDFC.

I can always put in a TAC case, but was just making sure I understood how it was supposed to work first.

Thanks!

[Tarakesh Jetti · Accepted solution]

Hi, 

The dhcp helper address shown in your diagram does not look correct. In needs to be each of either the data or management IPs (depending on which you have configured in the Server Settings). The DHCP service will be running on one on the ND nodes but since that node can change all 3 should be added as helpers (if using management on ND that would be):

Gi0/0

ip helper 10.1.1.10

ip helper 10.1.1.11

ip helper 10.2.1.20

Only one of these will respond, so you will see a slight delay as some requests will be sent to non-responding nodes. You can check the currently running node for the dhcp pod in the ND Admin Console for informational purposes, but you shouldn’t assume it will stay there.

You can also learn more about Cisco NDFC through our live Ask the Experts (ATXs) session. Check out this ATXs Resources [https://community.cisco.com/t5/data-center-and-cloud-knowledge/cisco-aci-ask-the-experts-resources/ta-p/4394491] to view the latest schedule for upcoming sessions, as well as the useful references, e.g. online guides, FAQs.

Thanks and regards,

Tarakesh Jetti,

Customer Success Specialist - CX Team.

[bunjiega]

Thanks Tarakesh, I did get it working finally!

It was helpful to know to put ALL the data/mgmt ip's in the helper statement and not the IP you are using for POAP in the external service pool. Also, was good to know the dhcp process could be running on any node at any time.

[saif ullah]

ND L3 CLuter Design: M1-DC1, M2-DC2, M3-DC3, Standby-Node-DC4

ND Nodes: Physical Appliances (M1+M2+M3+S1)

All Sites are connected and reachable via MPLS -BGP 

ND Version = 3.1.1K

NDFC Version = 12.1.3

The cluster is 3-physical nodes. One node resides in each DC.

Firewall: Fortigate-200F (Data/Mgmt/OOB gateways)

Fortigate-200F (DHCP relay)

ND-Data and Mgmt. The interface has different Subnets: 

ND-DATA: DC1: 10.1.0.101/24, 10.1.48.101/24, 10.1.64.101/24, 10.1.80.101/24

ND-MGMT: DC1: 10.2.0.101/24, 10.2.48.101/24, 10.2.64.101/24, 10.2.80.101/24

All switches are new Nexus 9000

TEST -LAB (VM-NODES): POAP works at the local site (where the switches and ND are all on the different subnet).

Firewall: Fortigate-200F (Data/Mgmt/OOB gateways)

However, at the remote site, which is a different subnet, I have IP reachability, DHCP relay of the 3 ND DATA IPs, and that fabric enabled for POAP, but the switches need to show up in NDFC. 

The DHCP request is sent to the ND-DATA IP from the OOB-Subnets gateway, but the switch is not assigned an IP address from ND, which is configured as a DHCP server.

[saif ullah]

Temporary Solution: You will have to verify which Data-node IP is being used as a source to release DHCP IP. You can only log in via root and manually change the DHCP file in the respective node. 

We’ve opened a new bug in your current Dashboard version for this issue.

And developer team has confirmed that permanent fix will be included on the next release 12.2(2) (July/August)

Now, how to track if the bug is fix in its totality?

In the next link, you will find the bug we opened:
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwk14831

You will see a bell symbol at the left of “Notifications” legend.

If you click on that symbol, a new message will be displayed asking you for your contact email information.

Why is asking for that? This is in order to track the bug alerts regarding, for example, on the next topics:

• Title
• Description
• Known Affected and Known Fixed releases.
• Status
• Severity

On that way, you can track the work that is being doing to fix it in the new Release 12.2(2)

Having this information and given that new version will be available until (July/August)

[saif ullah]

Has fixed