Packets with destination IP and MAC address of the SVI on a vPC peer switch are not forwarded over the Peer-link when received on the other switch on a Nexus 5600 running NX/OS 7.0(2)N1(1). Regardless of the "peer-gateway" setting in the VPC domain.
Since the new 5600 platform only has two NX/OS releases available: 7.0(1)N1(1) and 7.0(2)N1(1), down-grading is not an option.
Two Nexus 5600 switches with layer-3 configuration as a vPC cluster using two 40G interfaces as peer-link vPC and peer-keepalive via dedicated isolated 1G cross-link. There is no out-of-bound management network available. Management needs to be in-band.
Uplink to the core network is a Layer-2 (dot1x trunk) vPC with one link per Nexus connected to two ports on a catalyst core switch that acts as a Layer-2+3 core router/switch. The catalyst is the gateway and HSRP active router in all VLANs. All VLANs are trunked on the vPC peer link and the virtual port-channel to the catalyst.
Both Nexus 5600 have a Layer-3 config with SVI in some of the VLAN.
Problem: SVI addresses of the Nexus are not reachable when the packet is forwarded to the peer Nexus due to port-channel load-balancing (src-dst-ip on the catalyst).
This can be verified by shutting down one of the uplink vPC ports to force all traffic to one Nexus: All SVI addresses on the Nexus with the active link work while all addresses on the peer Nexus switch are dropped.
The Packets are not forwarded over the Peer-link. The "peer-gatway" setting in the vpc domain has no effect on this behaviour!
This bug is quite severe because it makes in-band management of Nexus 5600 series switches impossible.
As a workaround we have connected the mgmt0 ports to the front side, but that is not an acceptable solution (both by design and due to the high port-costs per SFP+ port on the Nexus 5600 and all SVI addresses must be reachable for monitoring).
