cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1698
Views
11
Helpful
1
Replies

ACI MultiPod Interconnection IPN usage

Maurlai
Level 1
Level 1

Hi all, 

    in a normal MultiPod configuration as picture MultiPoD.png, I'd like understand better how the connections between IPN Pod1 and IPN Pod2 works in terms of traffic flow and bandwidth usage.

 

Assuming to have a 2x10 Gbps dark fiber between IPN Pod1 and IPN Pod2, I'd like to know:

1. How network traffic is used? Active/active about both links? So 10gbps+10gbps?

2. If I assume to add other 10+10gbps between IPN Pods, can i aggrate the traffic to grow up 20+20 gbps in active/active?

3. Or is IPNs between Pods connection only active/standby mode?

 

Any documentation about that?

 

Thanks a lot

1 Accepted Solution

Accepted Solutions

Hi @Maurlai,

The IPN links are used Active/Active for the different traffic flows.

 

For Unicast traffic crossing the IPN:

Cisco Nexus 9000 Series Switches implement VXLAN in the way that a hash of the inner frame’s header is used as the VXLAN source UDP port. As a result, a VXLAN flow can be unique, with the IP addresses and UDP ports combination in its outer header while traversing the underlay transport network. Therefore, the hashed source UDP port introduces a desirable level of entropy for ECMP (Equal Cost Multi Path) which allows the Routing Table on each hop to leverage a different link depending on this hash.

As a note, VXLAN tunnels are end-to-end in Multi-Pod (i.e. from Leaf in Pod 1 to Leaf in Pod 2).

Conclusion: This is the default and expected behavior.

Link: VXLAN Overview: Cisco Nexus 9000 Series Switches

 

For Multicast traffic crossing the IPN:

An important design consideration should be made for the deployment of the Rendezvous Point (RP) in the IPN network. The role of the RP is important in a PIM Bidir deployment, as all multicast traffic in Bidir groups vectors toward the bidir RPs, branching off as necessary as it flows upstream and/or downstream. This implies that all the BUM traffic exchanged across Pods would be sent through the same IPN device acting as RP for the 225.0.0.0/15 default range used to assign multicast groups to each defined Bridge Domain. A possible design choice to balance the work load across different RPs consists in splitting the range and configure the active RP for each sub-range on a separate IPN devices, as shown in the simple example in the next Figure:
Screen Shot 2020-05-13 at 3.19.07 PM.png

Conclusion: The strategy is to manually assign the Active RP role to all the IPN Routers involved for every "subnet" of the 225.0.0.0/15 default range. At the same time, the non-Active RPs for the subnets should follow the Phantom RP approach for redundancy.

Link: ACI Multi-Pod White Paper

 

HTHs

View solution in original post

1 Reply 1

Hi @Maurlai,

The IPN links are used Active/Active for the different traffic flows.

 

For Unicast traffic crossing the IPN:

Cisco Nexus 9000 Series Switches implement VXLAN in the way that a hash of the inner frame’s header is used as the VXLAN source UDP port. As a result, a VXLAN flow can be unique, with the IP addresses and UDP ports combination in its outer header while traversing the underlay transport network. Therefore, the hashed source UDP port introduces a desirable level of entropy for ECMP (Equal Cost Multi Path) which allows the Routing Table on each hop to leverage a different link depending on this hash.

As a note, VXLAN tunnels are end-to-end in Multi-Pod (i.e. from Leaf in Pod 1 to Leaf in Pod 2).

Conclusion: This is the default and expected behavior.

Link: VXLAN Overview: Cisco Nexus 9000 Series Switches

 

For Multicast traffic crossing the IPN:

An important design consideration should be made for the deployment of the Rendezvous Point (RP) in the IPN network. The role of the RP is important in a PIM Bidir deployment, as all multicast traffic in Bidir groups vectors toward the bidir RPs, branching off as necessary as it flows upstream and/or downstream. This implies that all the BUM traffic exchanged across Pods would be sent through the same IPN device acting as RP for the 225.0.0.0/15 default range used to assign multicast groups to each defined Bridge Domain. A possible design choice to balance the work load across different RPs consists in splitting the range and configure the active RP for each sub-range on a separate IPN devices, as shown in the simple example in the next Figure:
Screen Shot 2020-05-13 at 3.19.07 PM.png

Conclusion: The strategy is to manually assign the Active RP role to all the IPN Routers involved for every "subnet" of the 225.0.0.0/15 default range. At the same time, the non-Active RPs for the subnets should follow the Phantom RP approach for redundancy.

Link: ACI Multi-Pod White Paper

 

HTHs

Review Cisco Networking for a $25 gift card

Save 25% on Day-2 Operations Add-On License