Combatting unicast flooding due to MAC address table issue (due to asymmetric routing)

[brettp]

I've uncovered an issue that has been going on long before I took over the network admin position here... Needless to say, I was thoroughly confused until I eventually figured out what was going on. Without getting into the whole nitty-gritty details, topology, etc. I was wondering what you seasoned pros would do. The issue exists in a more complex fashion, but I'll only describe one instance because the cause/effect is the same.

I have two 6509 core switches (Switch A and B) set up with HSRP. An ASA used for VPN is connected to Switch A, servers are connected to Switch B, the active gateway is Switch B. What's happening is, when VPN clients connect to a server, the traffic is unicast flooded over that VLAN on Switch A (because Switch A does not know the mac address of the server -- because the route on Switch B, points to the ASA -- the ethernet header has Switch B's MAC address, not the servers MAC address. Well, it will work for 5 minutes if the ASA needs to send an ARP request... but after 5 minutes, the flooding resumes.)

So... On Switch A, I can set the MAC Address Table aging time to 4 hours -- I think that should work. What are the drawbacks of doing that because memory getting eaten up by entries that would other wise time out?

Another idea I had... which is kind if ridiculous... would be to script something on a machine connected to Switch A that literally pings everything on our network... like a ping every four minutes... so that the MAC address table on Switch B is always populated with devices that are up and operational? Of course... this would not fix the issue if some TCN BPDU came in... so maybe a ping ever 10 seconds instead of every 4 minutes. It would only be a few hundred clients, so it's not like thousands of devices.

Maybe there is a better idea? Any insight is appreciated. 

Thanks!

[marce1000]

>....what you seasoned pros would do

  - What seasoned pros would is to review the current network architecture in place, as it seems to seriously deviate from common best practices and solutions.  And that would certainly include , not writing scripts to populate mac address-tables to start with....

M.

[Remi Astruc · Accepted solution]

Hello,

There are several ways to correct (or mitigate) your issue. Choose one of them, from best to worst:

- Set a static route on the servers containing all the VPN client subnets via the ASA IP, to solve the asymmetry.

- If VPN client subnets range is not under control, set the default route towards the ASA, and some static routes towards switch B.

- Instead of raising the MAC aging to 4 hours, you'd really better decrease the ARP cache timeout of the ASA to 290seconds.

- Connect the ASA on switch B.

- Move the SVI on switch A.

- Connect the servers on switch A.

...

I would not go with your idea of a pinging machine... not high-available, not scalable, not optimized, not seriously operationable, etc...

Remi Astruc

[brettp]

By the way, I inherited the mess -- I didn't create it! Anyway, thanks for the suggestions. Quite honestly, I have multiple solutions... Just don't know the best. The easiest would be to tweak ARP timeout on the ASA. The script idea is of course a band-aid and not a real fix (to temporarily alleviate the additional traffic.) At this point, I think the real option would be to remove multiple gateways from the same vlan and create a separate VLAN connecting the firewall to the switch -- that way it wouldn't be ARPing and sending out server traffic on it's local LAN.

[brettp]

I ended up changing the ARP timeout on the ASA as that seems to be an accepted solution. It's far from perfect, but it does the trick.

[Joseph W. Doherty]

Have you read: https://www.cisco.com/c/en/us/support/docs/switches/catalyst-6000-series-switches/23563-143.html