Hi All,
I have a lab topology like this but loop avoidance seems not to work and the traffic is duplicated on N9K-2 when the peer gateway is enabled.
Interconnections.
- Server1 connects to Router 1.
- Routers use a subinterface to connect to N3Ks.
- N3Ks are not forming VPC but work as extensions for the Router.
- N93180-1 and N93180-2 form VPC and virtual port channels with N3K.
- N93180-2 acts like a simple layer 2 switch and uses a port channel to connect with N93180-1&N93180-2.
-----------------------------------------------------------------------------------------------------------Layer2
Only N3K has trunk allow vlans setting like below.
N93180-1 and N93180-2 allow all vlans on all interfaces.
------------------------------------------------------------------------------------------------------------
Layer3
Router1 has sub-interfaces with vlan tags, 10,12,18. Subinterfaces with tags 10 and 13 are in the VRF.
Router2 has sub-interfaces with vlan tags, 11,13,19. Subinterfaces with tags 11 and 12 are in the VRF.
N93180-1 has SVIs 10,13,18. SVI 10 and 13 are in VRF.
N93180-2 has SVIs 11,12,19. SVI 11 and 12 are in VRF.
Router1 and Router2 both establish OSPF relationships with N93180-1 and N93180-2 via the three sub-interfaces.
Also, Router1 and Router2 both have sub-interfaces, 100 in Global and 101 in VRF, and establish OSPF neighbors with each other.
Router 1 connects to Server 1 using an interface in Global VRF.
N93180-1 and N93180-2 act as a gateway for Server2 with HSRP configured.
The Subnet of Server2 is configured in VRF so Router1 and Router2 leak it from VRF to Global.
------------------------------------------------------------------------------------------------------------
I run iperf3 from server1 to server2. The maximum bandwidth is 10G.
If the peer gateway is enabled and the traffic path is Server1 -> Router 1 -> N3K-1 -> N93180-2, the traffic would be sent out to N3K-2, N93180-3, and also N93180-1 via the peer link.
According to a document about loop avoidance, the traffic should not send to N93180-3 by N93180-1 but it does.
So, I can see almost 10G traffic from both links towards N93180-1(e1/1) and N93180-2(e1/3).
If I disable the peer-gateway feature, the problem is solved.
N93180-2 is not sent traffic out to N3K-2 and N93180-3 but only N93180-1.
(Real-time monitoring on Cacti)
Here are vpc settings
N93180-1
vpc domain 10
peer-switch
role priority 10
system-mac ce:cd:be:ef:ab:cd
peer-keepalive destination 10.1.1.2 source 10.1.1.1 vrf keepalive
peer-gateway
layer3 peer-router
ip arp synchronize
interface Vlan10
no shutdown
vrf member VRF
ip address 192.168.10.253/24
ip ospf passive-interface
ip router ospf 100 area 0.0.0.0
hsrp 10
ip 192.168.10.254
preempt
N93180-1
vpc domain 10
peer-switch
role priority 20
system-mac ce:cd:be:ef:ab:cd
peer-keepalive destination 10.1.1.1 source 10.1.1.2 vrf keepalive
peer-gateway
layer3 peer-router
ip arp synchronize
interface Vlan10
no shutdown
vrf member VRF
ip address 192.168.10.253/24
ip ospf passive-interface
ip router ospf 100 area 0.0.0.0
hsrp 10
ip 192.168.10.254
preempt
---
As I know, a peer gateway is used to forward the traffic to the destination Mac belonging to the vpc peer member and loop avoidance should kick in when traffic is received from the peer link and it's about to send out to another vpc member ports.
Best regards,
James