Greetings everyone,
I am facing an ARP-cache update problem on a 3850-stack. The issue occurs with a MS filecluster on server 2012 R2 nodes.
In short; a gratuitous ARP (GARP) is sent by the node, which is taking the file server role. The 3850-stack doesn’t react on this GARP 9/10 times, when the file server role changes between FILE01 and FILE02.The node with the file server role will have 2 IP-addresses assigned. This IP-address is removed from one node, and assigned to the new node, hence the GARP.
All clients in the same VLAN receives the GARP, updates their local ARP cache entry, and have connectivity to the file server. Since the 3850-stack doesn’t update the ARP cache entry, then clients on other VLANs are unable to reach the fileserver until the ARP-timer expires, or we issue a “clear arp xx.xx.20.59”.
Short diagram explanation:
- The 3850-stack is connected to a HP-enclosure (VC1, VC2) with 2x 10G fiber in active-active mode.
- The hypervisors have 8 network ports which are redundantly mapped (FlexNIC) to VC1 + VC2.
- The virtual servers have 1 NIC configured. It’s mapped to a NIC-team on the hypervisor.
FILE01:
xx.xx.20.56 /24
0015.5d65.0351
FILE02:
xx.xx.20.57 /24
0015.5d65.0352
Fileserver role:
xx.xx.20.59 /24
Client:
xx.xx.20.162
Switch version:
Technology Package License Information:
-----------------------------------------------------------------
Technology-package Technology-package
Current Type Next reboot
------------------------------------------------------------------
ipservicesk9 Permanent ipservicesk9
Model Revision Number : L0
Motherboard Revision Number : D0
Model Number : WS-C3850-48T
Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 56 WS-C3850-48T Denali 16.1.1 CAT3K_CAA-UNIVERSALK9 INSTALL
2 56 WS-C3850-48T Denali 16.1.1 CAT3K_CAA-UNIVERSALK9 INSTALL
Interface configurations:
interface TenGigabitEthernet1/1/1
description xxxx_VC1_X3
switchport trunk native vlan 999
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
end
interface TenGigabitEthernet2/1/1
description xxxx_VC2_X3
switchport trunk native vlan 999
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
end
interface Vlan20
ip address xx.xx.20.240 255.255.255.0
no ip redirects
ip directed-broadcast
Spanning-tree status for VLAN20:
VLAN0020
Spanning tree enabled protocol ieee
Root ID Priority 24596
Address 3c0e.231b.7800
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 24596 (priority 24576 sys-id-ext 20)
Address 3c0e.231b.7800
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/0/2 Desg FWD 4 128.2 P2p
Gi1/0/14 Desg FWD 19 128.14 P2p
Gi1/0/16 Desg FWD 19 128.16 P2p
Gi1/0/22 Desg FWD 19 128.22 P2p
Gi1/0/29 Desg FWD 4 128.29 P2p
Gi1/0/33 Desg FWD 4 128.33 P2p
Gi1/0/39 Desg FWD 4 128.39 P2p
Gi1/0/40 Desg FWD 4 128.40 P2p
Gi1/0/43 Desg FWD 19 128.43 P2p
Gi1/0/45 Desg FWD 4 128.45 P2p
Gi1/0/46 Desg FWD 4 128.46 P2p
Gi1/0/47 Desg FWD 19 128.47 P2p
Gi1/0/48 Desg FWD 19 128.48 P2p
Te1/1/1 Desg FWD 2 128.53 P2p Edge
Te1/1/4 Desg FWD 4 128.56 P2p
Gi2/0/11 Desg FWD 19 128.75 P2p
Gi2/0/15 Desg FWD 4 128.79 P2p
Gi2/0/19 Desg FWD 4 128.83 P2p
Gi2/0/29 Desg FWD 4 128.93 P2p
Gi2/0/33 Desg FWD 4 128.97 P2p
Gi2/0/39 Desg FWD 4 128.103 P2p
Gi2/0/40 Desg FWD 4 128.104 P2p
Gi2/0/43 Back BLK 19 128.107 P2p
Gi2/0/45 Desg FWD 4 128.109 P2p
Gi2/0/46 Desg FWD 4 128.110 P2p
Gi2/0/47 Back BLK 19 128.111 P2p
Gi2/0/48 Back BLK 19 128.112 P2p
Te2/1/1 Desg FWD 2 128.117 P2p Edge
Po3 Desg FWD 3 128.2091 P2p
Po4 Desg FWD 3 128.2092 P2p
Po5 Desg FWD 3 128.2093 P2p
Po21 Desg FWD 3 128.2109 P2p
Po22 Desg FWD 3 128.2110 P2p
Po30 Desg FWD 3 128.2118 P2p
Po31 Desg FWD 3 128.2119 P2p
The file server role changes from FILE01 to FILE02 in this wireshark output. As we can see, the server which is taking the role first checks for duplicate addresses before using xx.xx.20.59. Then it sends broadcasts a GARP to update clients in the subnet. The output is taken from a client in the subnet.
The GARP is recieved and the cache is updated
It has not been updated on the 3850-stack
The 3850-stack recieves the GARP. I've used a SPAN port to capture traffic in VLAN 20
Debugging ARP on the switch reveals nothing. There's no entry about recieving the GARP or anyhing, it's just blank.
When it does update the ARP-cache, then I see an entry as this:
012400: May 20 05:47:18.041: ARP TABLE: modifying entry xx.xx.20.59/0015.5d65.0352 on Vl20 for Dynamic
Any ideas on what to try?
Feel free to ask questions
Kind regards,
Jacob H. Malling
