UNIX Virtual IP Confusion

[sblavanya]

Hi All

Yet another confusion on MAC/ARP tables.. We have a unix server here, and it has a real IP eth0 - say 1.1.1.1, and Virtual IP on the same VLAN (say 1.1.1.2).. The issue is with linux servers, both real and virtual IPs have the same MAC ... x.x.x.x (say)..

Now under normal situation, everything works good.. the unix server is connected to a nexus 5k, which hoooks up to a 7k for layer 3 routing.. on the 7k, we have the IP ARP learnt for 1.1.1.2 -> x.x.x.x... All good..

The server team is now testing failover, and they are shutting down one Virtual IP (1.1.1.2) and moving to a different box on a different N5k.. Now, they will have the same IP 1.1.1.2 assigned to a VIP on the new box.. The new box obvioiusly has a different MAC z.z.z.z....

Again... during this failover process, the core nexus 7k learns z.z.z.z back on MAC table, but it doesnt update the ARP table , because it still learns the MAC x.x.x.x from the real IP configured on the server (1.1.1.1)....  Until I clear the ARP table manually, the failver virtual ip instance is not reachable..

I donno how unix/oracle desinged such solutions, but how can we make the failover quick ? static ARP ? or may be having the virtual IP on a different subnet than real ip  ? (because ARP table can have multiple IPs with the same MAC) ?

[sblavanya]

John, Giuseppe, and others . pls help

[sblavanya]

anyone ?

[colin.farley]

The reason why the ARP table is not updated is because: 1. there is no need for the L3 device to issue an ARP request, it already has an entry for this host in it's ARP table and still believes it to be valid until it times out 2. there was no GARP sent to update the ARP tables on all hosts connected to this subnet.

The easiest way to fix your problem is to tell the server team they need to have their host that's taking over the IP send our GARP (gratuitous ARP) packets to have hosts on this subnet update their ARP tables.  This assumes that the router is accept the GARP packets and there isn't security config preventing it.  You can easily test if the nix box in question is sending out GARP by using a host on that subnet to ping before and after the VIP switch or doing a packet capture on a host on that subnet. 

[sblavanya]

Hi Colin

Thanks so much. Ill check with Linux team on the GARP.. Ill test this with GARP enabled on the linux servers, and then doing the VIP failover ! Hope 7k's ARP table will be updated with the GARP from Linux... Ill let you know today on how it goes

Thanks again