VXLAN EVPN connection to classical ethernet for migration

[j.a.m.e.s]

Dear All,

I have a business requirement to keep the same IP addressing on an old classical ethernet CE environment to allow a phased migration into our new EVPN vxlan fabric. The old CE switches are using spanning tree and HSRP, as shown in the diagram (lower left). 

My plan is to create a VPC between the old CE switches and a VTEP leaf pair as shown above.

Could you confirm please:

1. Is this topology supported?

2. Are there any downsides to leaving the L3 HSRP GW in the CE environment? The idea would be to wait for the server team to migrate everything, then configure any Anycast GW on the VTEPs.

3. Would I be better off connecting the CE switches to the border-leaf pair rather than a random leaf pair?

I found a similar discussion here, but I wasn't entirely clear on the outcome, hence the above diagram.

Thank you in advance for your help.

[j.a.m.e.s]

One aspect which occurred to me is that I should probably disable arp-suppression as the default gateway will remain on the classical ethernet switches until the end of the migration.

[Adriano Gomes]

Someone can help me?

Where  I find a procedure to migrate classical ethernet to Vxlan? I have 2 WS-6509E with some vlans with HSRP and I have to migrate these vlans to VxLAN. 

How can I do the connections between WS-6509E and NX-OS 9k?

How can I migrate my vlans to 9k using anycast default gateway?

[Revelation78]

I don't know of any resources specifically going into this though I am sure there are some out there somewhere. The process is not difficult, there are just a few steps you need to take. 

1) Connect the 6509E to a 9K via normal L2 trunking configurations. 

2) Create VLANs and VXLAN configurations on 9K (assuming this is a leaf and the rest of the environment is already configured)

3) Remove Default gateway off of 6509E

4) Add anycast-gateway/DG configs to the VLANs in the new VXLAN environment. 

  4a) Reconfigure Spanning-tree so that the VXLAN environment is the root. 

That is a basic run-down of how you achieve the end result without getting into the weeds. 

[j.a.m.e.s]

In terms of documentation, "Building Data Centers with VXLAN BGP EVPN: A Cisco NX-OS Perspective" is helpful and chapter 8 covers Classic Ethernet extensions. Honestly there's not much more than we have mentioned in this thread.

I would say the most important point is to use VPC and to mention that personally I couldn't get the gateway move from the old HSRP to Anycast VXLAN environment to work - all the traffic was blocked - but I didn't try to troubleshoot this in detail. In the end I migrated the HSRP to Anycast at the point when the last host on the VLAN migrated.

[Adriano Gomes]

We have WS6509E that will be connected in any LEAF (topology below), we have some SVI that will be migrate to VXLAN fabric. We have some static route configured behind the L3 SVI, look below the VLan34 configuration in our WS6509E:

WS6509E:

interface Vlan34
ip address 10.0.22.253 255.255.255.0
standby 34 ip 10.0.22.1
standby 34 timers 2 7
standby 34 priority 150
standby 34 preempt
!

ip route 10.0.23.0 255.255.255.0 10.0.22.2

LEAF Configuration:

LEAF:

spanning-tree vlan 34 priority 4096

vlan 34
name L2-VNI-140-Tenant1
vn-segment 50140

vrf context Tenant-1
vni 50999
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn

fabric forwarding anycast-gateway-mac 0000.2222.3333

interface Vlan34
no shutdown
vrf member Tenant-1
no ip redirects
ip address 10.0.22.1/24
fabric forwarding mode anycast-gateway

interface nve1
no shutdown
source-interface loopback1
host-reachability protocol bgp
member vni 50140
mcast-group 239.0.0.140

How could I migrate the static route in WS6509E to LEAF?

Can I do this configuration in all LEAF device?

vrf context Tenant-1

ip route 10.0.23.0 255.255.255.0 10.0.22.2

This is my future VXLAN Topology:

[Revelation78]

Where are your border leafs in this topolgy? What routing method/protocol are you using on your border leafs?

You can use static routes configured on the border leaf. You don't configure a static route on every single leaf. For instance we are running nexus 9508s for our border leafs. We have BGP setup to advertise routes into and out of the fabric with the "core network" and we use a route-map to limit what IPs make it out. We also do this to ensure we don't advertise a bunch of /32s out of the fabric. 

You only need to configure a static route on the border leafs if the default route is not advertised into your fabric. If it is not, then you will need a static route configured in the vrf(s) and use a tag/route-map to advertise it to all of the leafs. 

[Jilani101]

Can you post the configurations please 

[Revelation78]

Sorry for the late reply to your PM, I had forgotten about you issue until I was going through and cleaning out my inbox for the past couple of weeks.

I don't see anything off-hand wrong about your diagram. Couple of points I'd like to make to answer some other questions.

You really want to keep the border leafs dedicated to routing into and out of the fabric and a landing zone for services like FW, NetScalers, F5's etc...

It really makes no difference whether you have a dedicated pair for the VLAN stretch or not. It is easier to troubleshoot, should an issue arise, if that is all that pair is servicing.

My suggestion is to have a pair of 5k's or 9k's off of your 6509 that support vPC and have vPC setup between the VLAN stretch switch pairs. Use traditional trunking/port-channel between the 6509 and this new set. This helps as it is least impactful on your network as far as configurations go. It also depends on how heavily utilized your 6509s are.

I apologize again for being late to the party, so to speak. Hopefully you already have everything worked out.

[j.a.m.e.s]

Thanks a lot for replying. We've gone live with this and it seems to be working. A couple of points to add:

• Arp-supression has to be disabled across the evpn fabric while the VNIs are L2 only without an SVI
• We made the N9Ks the spanning-tree root, this is apparently not mandatory but it's the best practice
• It's fine to leave the crosslink between the classic-ethernet switches as it will be STP blocked. If there was a vPC dual-active scenario, it might add to your woes, but this unlikely. Best to leave it in for ease of rollback, even if admin-shut.

The drawing of the vPCs I posted orignally was a bit weird. Here's a better version:

[j.a.m.e.s]

One final note to say that we did get ourselves into major trouble with this as some of the traffic was getting blackholed.

In the end it turned out one of the Catalysts was tagging spanning tree (pvst) BPDU's with the wrong dot1Q tag. This caused the N9k to place the vlan into inconsistent mode. The only way to find it was:

N9k> show spanning-tree internal event-history errors
24) Event:E_DEBUG, length:113, at 098318 usecs after Sat Apr  7 18:28:27 2018
    [105] stp_sstp_bad_pvid() port Po1051 SSTP BPDU rcvd on VLAN 3251, VLAN tag 3250
 making local port inconsistent

The fix was a simple "shut/no shut" on the Catalyst physical interfaces - doing the same on the N9k side was insufficient.