cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
27051
Views
30
Helpful
6
Replies
visitor68
Enthusiast

vPC Peer Gateway

Hi - with Peer gateway functionality, VRRP, HSRP, GLBP are not needed. At least I think they're not, although Cisco is not clear about whether one configures VRRP/HSRP with peer gateway functionality or in addition to it. Instead, both vPC peer switches with peer gateway routing enabled will act as active forwarders for each other regardless of who the configured default gateway is for a particular end host. So, if switch 2 receives a frame that needs to be routed, it will route it even if the frame has switch 1's MAC address in the destination field (because the end host has switch 1 as its default gateway).

From Ciscos Website:

The vPC peer-gateway functionality allows a vPC switch to act as the active gateway for packets that are addressed to the router MAC address of the vPC peer. This feature enables local forwarding of such packets without the need to cross the vPC peer-link. In this scenario, the feature optimizes use of the peer-link and avoids potential traffic loss.

You must configure the peer-gateway functionality on both vPC peer switches.

Question: Imagine two switches in a vPC, switch 1 and switch 2. OK, vPC peer switch 1 dies. So, vPC peer switch 2 will continue to forward on its behalf, as it did before the failure. BUT, what about when the end host has to renew its ARP entry for its default gateway after its expiry? Who will answer the end hosts ARP request for its configured gateway? Remember, that switch is dead now. So who will answer - vPC peer switch 2 with its own MAC address?

 

Thanks

6 REPLIES 6
Steve Fuller
Engager

Hi,

The answer is that no one will answer the ARP request. If an end system is configured with a default gateway of the real IP address assigned to an interface on the "dead" router, then game over.

The peer-gateway command does not change the way in which the router responds to ARP requests. Rather it results in a device e.g., R2, installing the MAC address of an interface on the vPC peer R1, into its own CAM table with the "Gateway" (G) flag set. This is simply to allow R2 to directly route traffic received on a vPC that was destined to the MAC address of R1.

I think it worth pointing out that the vPC peer gateway feature was not not developed as a replacement for HSRP, VRRP etc. The feature is there to allow Nexus switches with vPC to optimally route traffic from devices such as NAS and load balancers that don't necessarily use ARP to resolve the MAC address of the default gateway. Instead these devices simply used the source MAC address of the recieved frames as the destination MAC address in the return traffic.

As per the vPC Peer-Gateway section on page 110 of the Design and Configuration Guide: Best Practices for Virtual Port Channels (vPC) on Cisco Nexus 7000 Series switches:

"The vPC Peer-Gateway enhancement (Figure 91) allows vPC interoperability with some network-attached storage (NAS) or load-balancer devices that do not perform a typical default gateway ARP request at boot up."

So to your question. "Imagine two switches in a vPC, switch 1 and switch 2. OK, vPC peer switch 1 dies. So, vPC peer switch 2 will continue to forward on its behalf, as it did before the failure."

This is incorrect. When switch 1 dies the vPC peer-link will go down and the MAC addresses that were learned on S2 via the peer-link from S1 are removed, including the MAC address of the interfaces on S1.

We can see this in the following. Here we have the MAC address of the local SVI (547f.ee69.8efc), the HSRP v2 MAC and the peer SVI (547f.ee69.8d3c) via Po1 (shown in red).

n5k1# sh int vl 171
Vlan171 is up, line protocol is up
  Hardware is EtherSVI, address is  547f.ee69.8efc
  Internet Address is 172.17.1.3/24
  MTU 9000 bytes, BW 1000000 Kbit, DLY 10 usec

n5k1# sh mac add vlan 171
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY   Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 171      0000.0c9f.f0ab    static    0          F    F  Router
* 171      0011.2557.df8a    dynamic   0          F    F  Po11
* 171      547f.ee0c.183c    dynamic   0          F    F  Po11
* 171      547f.ee10.ce7c    dynamic   0          F    F  Po11
* 171      547f.ee69.8d3c    static    0          F    F  Po1
* 171      547f.ee69.8efc    static    0          F    F  Router


Note that this is a Nexus 5K with L3 and for some reaon is not showing the G flag for these entries. That aside, when I reload the peer switch I see the MAC address of the peer switch removed

n5k1#
2015 Sep 17 14:47:19.034 n5k1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet1/15 to Ethernet1/16
2015 Sep 17 14:47:19.038 n5k1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: Ethernet1/15 is down
2015 Sep 17 14:47:19.043 n5k1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet1/16 to none
2015 Sep 17 14:47:19.069 n5k1 %ETHPORT-5-IF_TRUNK_UP: Interface Ethernet1/15, vlan 1,171-175,178-179,901 up
2015 Sep 17 14:47:19.074 n5k1 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/15 is down (Link failure)
2015 Sep 17 14:47:19.116 n5k1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: Ethernet1/16 is down
2015 Sep 17 14:47:19.116 n5k1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: port-channel1 is down
2015 Sep 17 14:47:19.167 n5k1 %ETHPORT-5-IF_TRUNK_UP: Interface Ethernet1/16, vlan 1,171-175,178-179,901 up
2015 Sep 17 14:47:19.171 n5k1 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/16 is down (Link failure)
2015 Sep 17 14:47:22.814 n5k1 %VPC-2-PEER_KEEP_ALIVE_RECV_FAIL: In domain 1, VPC peer keep-alive receive has failed
[snip]
n5k1# sh mac add vlan 171
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY   Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 171      0000.0c9f.f0ab    static    0          F    F  Router
* 171      0011.2557.df8a    dynamic   10         F    F  Po11
* 171      547f.ee0c.183c    dynamic   10         F    F  Po11
* 171      547f.ee10.ce7c    dynamic   10         F    F  Po11
* 171      547f.ee69.8efc    static    0          F    F  Router


If you want to provide default gateway resilience for your end systems then you would still need to use one of the standard Fist Hop Redundancy Protocols such as HSRP, VRRP etc.

Regards

So, Steve. This is very interesting. Nowhere - or at least that I can see - does the Cisco documentation mention that peer-gateway functionality is deployed IN ADDITION to HSRP. Now, it would seem to make sense, hence my question and the scenario I painted. This was my suspicion.

Also, when I said ""Imagine two switches in a vPC, switch 1 and switch 2. OK, vPC peer switch 1 dies. So, vPC peer switch 2 will continue to forward on its behalf, as it did before the failure."

What I meant was SW2 will continue to do L3 forwarding.

Hi,

Nowhere - or at least that I can see - does the Cisco documentation mention that peer-gateway functionality is deployed IN ADDITION to HSRP.

The fact that vPC peer gateway should be run in addition to HSRP is not explicitly stated as the two features serve different purposes. HSRP provides redundant default gateway functionality. The vPC peer gateway does not.

As you know, in a working vPC environment each router has the MAC addresses of the vPC peer in its CAM table with the Gateway (G) flag set. This is to allow each router to forward at Layer-3 any traffic it receives that is destined to the MAC address of its peer. This is not providing resilience for the IP associated with that MAC.

As the example I included in the previous post shows, when SW1 dies the MAC address with the G flag is removed from the CAM of SW2. That means SW2 will forward traffic destined to the MAC of SW1 in the same way it would for any unknown unicast MAC address i.e., it will flood at Layer-2 on all links carrying the VLAN in which that traffic was received. It will not continue to forward that traffic at Layer-3.

So while there’s no documentation to say that peer gateway must be run in addition to HSRP, you won’t find any Cisco documentation that states you can disable HSRP if you run vPC and peer-gateway, because peer gateway does not provide the same functionality as HSRP.

What I meant was SW2 will continue to do L3 forwarding.

SW2 will continue to do Layer-3 forwarding for traffic destined to any IP and MAC that are configured on SW2. As above though, it will not continue to forward at Layer-3 any traffic destined to any IP and MAC that were associated with SW1. It no longer has the MAC of SW1 in its CAM table and so will flood that traffic at Layer-2.

Regards

Steve:

"As you know, in a working vPC environment each router has the MAC addresses of the vPC peer in its CAM table with the Gateway (G) flag set. This is to allow each router to forward at Layer-3 any traffic it receives that is destined to the MAC address of its peer. This is not providing resilience for the IP associated with that MAC."

Is this with peer gateway functionality set, correct?

Correct. As an example, the following shows an N7K pair without the peer-gateway command configured under the vpc domain. In this it can be seen that whilst the MAC address of the peer VLAN interface is in the CAM table, it is not programmed with the G flag in the peer device. In this example there are only two MAC addresses with the G flag, the devices own MAC address and the HSRP MAC address.

n7k1# sh int vl 2
Vlan2 is up, line protocol is up
  Hardware is EtherSVI, address is  0026.xxxx.76c2
  [..]
n7k1# sh mac add vl 2
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
G 2        0000.0c07.ac02    static       -       F    F  sup-eth1(R)
* 2        0026.xxxx.7042    static       -       F    F  vPC Peer-Link
G 2        0026.xxxx.76c2    static       -       F    F  sup-eth1(R)
[..]

n7k2# sh int vl 2
Vlan2 is up, line protocol is up
  Hardware is EtherSVI, address is  0026.xxxx.7042
  [..]
n7k2# sh mac add vl 2
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
G 2        0000.0c07.ac02    static       -       F    F  vPC Peer-Link(R)
G 2        0026.xxxx.7042    static       -       F    F  sup-eth1(R)
* 2        0026.xxxx.76c2    static       -       F    F  vPC Peer-Link

And here's another pair with the peer-gateway command configured. In this it can be seen that there are three entries with the G flag programmed, the devices own MAC, the peers MAC and the HSRP v2 MAC.

n7k3# sh int vl 10
Vlan10 is up, line protocol is up
  Hardware is EtherSVI, address is  6c9c.xxxx.c8c2
  [...]

n7k3# sh mac add vl 10
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
G 10       0000.0c9f.f00a    static       -       F    F  44.0.1054(R)
G 10       6c9c.xxxx.c8c2    static       -       F    F  sup-eth1(R)
G 10       6c9c.xxxx.f342    static       -       F    F  44.0.2605(R)
[..]

n7k4# sh int vl 10
Vlan10 is up, line protocol is up
  Hardware is EtherSVI, address is  6c9c.xxxx.f342
  [...]

n7k4# sh mac add vl 10
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
G 10       0000.0c9f.f00a    static       -       F    F  sup-eth1(R)
G 10       6c9c.xxxx.c8c2    static       -       F    F  43.0.2605(R)
G 10       6c9c.xxxx.f342    static       -       F    F  sup-eth1(R)

Note that the second pair are also running FabricPath, hence the Ports/SWID column shows differently.

Regards

asif.naveed
Beginner

Dear visitor68

First of all, your question is not related to the peer-gateway because function of Peer Gateway is different and your Question is related to HSRP.

vPC has two Control Planes and two Data Planes. In your scenario if SW1 dies then SW2 has to respond the traffic i.e HSRP/VRRP concept.

Since we have two Control Planes, we need two SVI. So due to two SVI HSRP/VRRP is must..

There are some applications like load Balancer etc that dont rely on ARP and need physical MAC and with revrse header and source/destination so to overcome this Cisco came up with a command "Peer-Gateway"

The Peer-Gateway is a vPC feature that allows vPC peer devices to act as a gateway for traffic addressed to the MAC addresses of their peers. Please note that MAC will be physical of any vPC Peer, not the virtual of HSRP etc