OTV FHRP filtering - Blocking inter-vlan routing between DCs

[Kyle C Barnes]

Hi all,

Using the above image as a reference point, you can see that with the configuration guide for OTV best practice, we'll break inter-vlan routing between DCs.  

Here are some logical steps one might take when troubleshooting this:

1. Why not remove the MAC ACL?

If we remove the MAC ACL our "DAL" ToR switches will learn the 0000.0c07.acee MAC address from two points: The OTV devices and from the DC core.  We need to make sure that from a L2 forwarding perspective, the only path to the MAC address associated with the gateway is to the DC core.  

2. Why not connect the OTV router to the core itself (with the MAC ACL removed)?  That way, there are no L2 issues with MAC table instability?

The core is 40G and the ASR1002 only supports 1G connections.  We have the ASR connected to the ToR switch as it's the only place to allow for 1G connectivity.

3. Why not disable HSRP on the 6500 side so that the MAC addresses are different?

If we have different MAC addresses, we'll have a scenario where host B will have two entries in it's local ARP table; one for the gateway in DAL and one for the gateway in CORP.

Note: I've gotten this to work by using this method, but in doing so I have to disable OTV's arp-nd-cache and configure the MAC ACL to block arp responses.  

Now, for my question (about time!):

Does anyone have any advice on how to get this to work given this setup--WITHOUT having to disable arp-nd-cache?

[Rajagopal Sivaramakrishnan]

Did you ever fix this ? We have the same issue

[Rajagopal Sivaramakrishnan]

Did you ever solve this ?

[brselzer]

Hello Kyle,

I think there is a misconception here. HSRP does not use the virtual mac when sending packets.

https://www.cisco.com/c/en/us/support/docs/ip/hot-standby-router-protocol-hsrp/10583-62.html#anc21

See the packet flow section. Step 5 in your diagram is not what actually happens. When the 6500 puts the packet into vlan 128, it will use the destination mac of PC-B but the source mac will be the BIA mac of the vlan which is different for each device participating in HSRP. This means the packet will not have the source mac of the HSRP virtual mac when it hits OTV and will not be dropped by the FHRP ACL. 

The reason for FHRP filtering is that HSRP hellos are sent with a source mac of the virtual mac. We want to block HSRP hellos from going over OTV because you do not want a device on one site to be peering with a device on the other site. The HSRP domains should remain seperated. 

Is this actually broken for you? This is a very common setup with a gateway on each side of OTV and works. If you are having issues pinging between vlans, let me know the details specific to your problem and maybe I can assist.

Hope that helps!

EDIT: Just noticed this post was old. Rajagopal, if you are having issues with OTV, feel free to post the details if your issue. Thanks!

[Rajagopal Sivaramakrishnan]

what will the border leaf with anycast gateway source the ARP request as ?  It looks like we saw ARP drops on ethanalyzer with source as anycast. In OTV , we block the anycast MACs, since we dont want any duplicates on the other side..

ARP learning is driving us crazy.. on one side we have a VXLAN fabric.. from Border leaf we have a l2 trunk to OTV router. .on the other side, we have OTV router which connects to the legacy network through a N7K.

[brselzer]

Hello Rajagopal,

I am not 100% sure with anycast. Was it dropping arp requests or gratuitous arps? Your setup sounds a little more complex than the original post. It might be worth opening a TAC case to help troubleshoot the issue. I am not as familiar with the Nexus series. Sorry I couldn't be more helpful!