Nexus 5000 IP Reachability across VPC within a 4 member HSRP group

[angus_mcguire]

I have 2 pairs of Nexus 5000 units (pair 1 and pair 2).  A pair consists of 2 Nexus 5000 (A and B) connected to each other via a VPC containing 2 ports ie P1-5KA -- P1-5KB (vpc domain 6) and P2-5KA -- P2-5KB (vpc domain 10)

P1-5KA also connects to P2-5KA via a trunk port

P1-5KB also connects to P2-5KB via a trunk port

ie

P1-5KA -- P1-5KB

  |                  |

  |                  |

P2-5KA -- P2-5KB

Each 5k has an SVI addresses as follows

  P1-5KA=10.18.136.2

  P1-5KB=10.18.136.3

  P2-5KA=10.18.136.4

  P2-5KB=10.18.136.5

Hsrp exists between all four with a virtual address of 10.18.136.1.  P1-5KA is the Active with P1-5KB as Standby.

I can ping between the four using their SVI addresses.  I am unable to ping the HSRP virtual address .1 from P2-5KA or P2-5KB.

I can ping ok only if I shut the VPC between P2-5KA or P2-5KB or define another mac address under the HSRP config other than the system default.  IP Packet debugs show that ping sourced from P2-5KB to P1-5KA loop between P2-5KA -- P2-5KB.  Pings sourced from P2-5KA to P1-5KA are transmitted but none of the 4 device debugs show a receive.

both peer-gateway and delay restore 120 have been configured under all vpc domains and all units rebooted.

[owenjustin13]

I am also seeing the same problem.

The only additional info I have to add is that when I disable HSRP on the P2 switches (by removing the ip from the hsrp group on the P2 switches) then the pings start working.

It looks like something is not working with the HSRP Active-Active enhancement for vPC.  It may specifically be related to this Back-to-Back vPC topology with 4 HSRP members.

I just re-read the whitepapter "white_paper_c11-516396.pdf" -

Virtual PortChannels: Building Networks without Spanning Tree

Protocol Virtual PortChannels: Building Networks without Spanning Tree
Protocol

It contains the following quote and indicates that the Active/Standby HSRP members should be both actively forwarding packets for the HSRP Mac, but it does not mention the HSRP group members that are in "Listen" mode.  I have found that all four of the HSRP members including those ln Listen mode are forwarding traffic for the HSRP mac.  You can see in the "show mac address-table" comand that all four membrs have the HSRP virtual mac entered as type "router".

From the White Paper:

"In the case of HSRP, the improvement was made to the forwarding engine specifically to allow local Layer 3

forwarding at both the active HSRP peer and the standby HSRP peer. This enhancement provides, in effect, an

active-active HSRP configuration with no changes to current HSRP configuration recommendations or best practices

and no changes to HSRP. The HSRP control protocol still acts like an active-standby pair, so that only the active

device responds to Address Resolution Protocol (ARP) requests, but a packet destined for the shared HSRP MAC

address is accepted as local on either the active or standby HSRP device. In the case of HSRP, the improvement was made to the forwarding engine specifically to allow local Layer 3
forwarding at both the active HSRP peer and the standby HSRP peer. This enhancement provides, in effect, an
active-active HSRP configuration with no changes to current HSRP configuration recommendations or best practices
and no changes to HSRP. The HSRP control protocol still acts like an active-standby pair, so that only the active
device responds to Address Resolution Protocol (ARP) requests, but a packet destined for the shared HSRP MAC
address is accepted as local on either the active or standby HSRP device."

[angus_mcguire]

Thank you for the reply

In our installation we are running in an Active/Standby none back to back VPC link between

P1-5KA -- P2-5KA

and

P1-5KB -- P2-5KB

Just using spanning tree.  Same results here when removing the IP address from P2 side, all pings succeed.

[owenjustin13]

Ok, that is good to know.  I was planning to re-configure, and remove my Back-to-Back vPC and replace it with spanning tree to see if that made any difference.  So, we are seeing the same behavior when connected with vPC or spanning tree beteen the two sides.

[Nicholas Poole]

do hosts on these vlans on both sides of the network have any problems connecting to the HSRP address, or is it just a nexus to nexus problem?

[owenjustin13]

Hosts on the P2 side of this diagram cannot ping the HSRP Virtual IP on the P1 side.

Hosts on the P1 side can ping the HSRP Virtual IP on the P1 side.

[owenjustin13 · Accepted solution]

This looks like a known bug that will be fixed in an upcoming release 5.1(3)N1(1)  which is supposed o be out in December.  Here is the link to the BugTraq Details:

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtq61472

[angus_mcguire]

Cisco Tac must have looked up this post as they have finally after failing to understand the problem suggested the same...

Thanks for your input

[Oleksandr Nesterov]

Guys

NX-OS release with commited fix is already published in CCO:

http://www.cisco.com/cisco/software/release.html?mdfid=283444903&flowid=20342&dvdid=281717634&softwareid=282088129&release=5.1.3N1(1)&relind=AVAILABLE&rellifecycle=&reltype=latest