Network redundancy, VPC, VXLAN and EVPN

[mohammed01701]

Hi!

I do have a test lab about VPC, VXLAN and EVPN. When i setup the lab everything was working very good untill i do som tests. If you want to see the configration i have a link which i used.
I do have changed IP-adress, VRF name, AS-number for BGP and RD for EVP. The link is here:

https://datacenterdope.wordpress.com/2015/10/14/vxlan-mp-bgp-evpn-part-2/

I want to thank Matt Pinizzotto who writes this article.

So now i have som quations about this lab:

First see the topology:

NOTE: HOST-A is on VLAN 200  and HOST-B is on VLAN 201.

Step-1:

Everything works fine and the HOST-A and HOST-B can communicate each other without a problem!

Step-2

On LEAF-4 when I turn off the links connected to SPINE-1/2. The communication between HOST-A and HOST-B has ended, why? The answer is "vPC Loop Avoidance" but where is the peer gateway feature? or do I understand wrong? The traffic should go via VPC peer-link or?
If you want to know more about "vPC Loop Avoidance" read this link:

https://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf

Step-3

I can understand this step. The traffic is going on LEAF-3 towards SPINE-1/2 and everythings works again.

But my big question is am i thinking/doing wrong here or that is the default behavior of this design?

Thanks

/Mohammed

[nazimkha]

I do not think there is a problem in your design and test scenario.
If the leaf-4 gets disconnected from the Fabric the traffic has to go from Leaf-4 to Leaf-3 over peer-link and from Leaf-3 it should be forwarded.
vPC loop avoidance feature will only kick in if the traffic gets forwarded over the peer-link and have to traverse another vPC. In this case the traffic gets forwarded from Leaf-4 to Leaf-3 and it has to be forwarded from Leaf-3 to Spine via a Layer-3 interface
So something might not have set correctly in configuration or you might be hitting some bug

[mohammed01701]

Hi!

Thanks for your reply. But I do not think I've done anything wrong with the configuration. If I had done that, then no one would work at all, i do have a OSPF neighborship between LEAF-1/2 and LEAF-3/4! I could not find any bug related to the peer gateway. By the way I am using "nxos.7.0.3.I7.4"

/Mohammed

[nazimkha]

Curios to know what happens if you reboot Leaf-4

[mohammed01701]

Hi!

It will no be an issue, why? traffic goes through LEAF-3 to SPINE-1/2. The problem occurs only when traffic is need to go via VPC peer link!

/Mohammed

[brdewal2]

Hi Mohammed,

Do you have any backup routing configured between Leaves 3 and 4?

Leaf 4 needs to have a route to Leaves 1 & 2 if it's spine links go down.

It looks like you're using OSPF as your underlay so you should have an SVI across the peer-link in the default VRF that has an OSPF neighborship between them so that you still have routes if your uplinks go down.

Also, I assume you're doing this on a virtual switch, but in case you're doing this on physical gear I should note that 9200, 9300-EX, and 9300-FX switches require this backup SVI to be configured with the command "system nve infra-vlans X" for it to work properly.

[mohammed01701]

Hi brdewal! 

Thanks for your reply. I do have a backup link between LEAF-3 and LEAF 4 but not command "system nve infra-vlans X" I will test this command as soon as possible.

NOTE: This other question outside the topic! 

Is this command "system nve infra-vlans X" only for SPINE and LEAF topology or there is other command for classic topology? I am thinking if i would not use VXLAN what will be for command that makes LEAF-3 and LEAF-4 can communicate each other over VPC peer-link? 

/Mohammed

[brdewal2]

Hi Mohammed,

And you have OSPF running over that backup link?

system nve infra-vlans configures a VLAN as one that can pass VXLAN encapsulated packets. So if your uplink goes down, encapped packets would have to go over this SVI instead and therefore this command must be configured if you have a 9200, 9300-EX, or 9300-FX series N9k. Other N9k switches don't need this command.

This command does not have to do with Spine & leaf but rather how these specific platforms forward encapped packets in hardware.

Hope this helps!

-Bruce 

[mohammed01701]

Hi brdewal!

Thanks! Now i have configured that command and it helped.

NOTE: HSRP active for all Vlans on switch 1 and Switch 1 is also root bridge for all Vlans.

I do have another topology which i do have same problem but on this topology i am using OSPF on Core routers which connected to 6807 VSS with port-channel.

Is this design wrong or do i need another command that makes traffic can go over peer-link without using VXLAN? what i mean the previus question was the command "system nve infra-vlans" is only for VXLAN.

Is there any way that second design will work, if it is yes how? I have tested to configure peer-gateway command and it doesn´t helped. Is there another command that forces switch 1 to accept traffic from switch 2 can pass over peer-link?

Thanks again.

/Mohammed

[brdewal2]

Hi Mohammed,

I don't see any problems with this as long as STP is forwarding and you have routing setup between switches 1 and 2. 

You might need to take captures to see where the packets are actually dropping here.