@Robert Burns wrote:
Confusion #1: Which interface (DATA or Management) does the ND use to communicate with APIC OOB for the MSO? Cisco ND guide can not be more confusing on this...Like what does this statement even mean "For Multi-Site Orchestrator application, the data network can have in-band and/or out-of-band IP reachability depending on your deployment scenario."???? I do know MSO need to communicate with APIC OOB...
All this means is that the ND Data Network (that MSO uses to reach APICs) just needs reachability to either the APIC's Inband or Out of Band IP. This could be clearer, I'll get this updated. For the Day2 Apps (Insights/NAE) these have a hard requirement to reach the fabric device (APICs and Switches) using only the Inband IPs. My advice would be this: If you plan on deploying MSO + Day2 Apps, on the same ND cluster, then plan to use only the Inband IPs of the fabrics. ND_Data_Network_Interface >>> Inband_IPs_of_APICs_and_Switches. If you were only ever deploying MSO on Nexus Dashboard (bit of a waste), then you could get away with ND_Data_Network_Interface >>> OOB_IPs_of_APICs.
So the ND DATA interface is actually where MSO communicates with APIC? If so, that's cool. But I thought the MSO can only communicate with the OOB of the APIC controllers for multi-site...Is it because of the ND, MSO can use the In-Band of APIC now? OR I missed feature improvement here? Also regarding the comment of "bit of a waste" of only running MSO on ND, we were initially planning to just get 3 UCS C servers with vSphere but were told MSO will only be available on ND going forward...
Confusion #2: The guide does state "The two interfaces can be in the same or different subnets." for ND...But will there be downsides if I put the ND DATA and MANAGEMENT on the same subnet? Like for future additional Day-2 app?
Just like anything else, the Management traffic of a system should be separated from the Data traffic. Can ND interfaces (Data & Management) belong to the same subnet - sure. Is it a good idea - No. When you start using Day2 Apps (Nexus Insights, NAE etc), you're going to be pushing a considerable amount of telemetry flows to the ND. This traffic will always utilize the Data Network interface of the ND. Do you really want your Management traffic contending with this additional congestion, or would you prefer it to be separated/isolated from it?
All fair points, I guess if I had to deploy the cluster with DATA and Management on the same subnet per ND, would it be possible to re-configure IP address to different network without rebuilding the ND cluster? If so, how much effort/impact to re-assign IP address to different network for either the DATA or Management? and also which one would be easier to change?
