Nexus 7k vPC and orphan ports

[Network Pro]

hi

we have two nexus 7k connected via vPC peer. We have edge switch connected to the core using HSRP via vPC.

Now we have 1 orphan port connected to each Nexus (WLC).

The problem is i cant seem to connect / ping the WLC (only 1 of them) that is connected to the orphan port  and i think it is probably due to the packet arriving at the secondary HSRP and traversing through peer-link and dropping the packet.

HSRP address on Core: 10.10.10.1

vlan 10 - N7k1 - 10.10.10.2

Vlan 10 - N7k2 - 10.10.10.3

Edge Sw - 10.10.10.10 - Vlan 10

WLC 1 - Vlan 100 - 10.10.100.11

WLC 2 - Vlan 100 - 10.10.100.12

HSRP ADdress for Vlan 100 - 10.10.100.1

N7k1 - Primary vPC

n7k2 - Secondary vPC

WLC1 (orphan port)

  |

  |

N7k1 ------vPC---------N7k2 -------WLC2 (orphan port)

  |                              |

  |                              |

  |                              |

  |-------EDGE SW-------|

          (vlan 10 - 10.10.10.10)

Now what is the best practise for HSRP with vPC for orphan ports ?

The problem is i can only ping 1 wlc from a machine. on doing a traceroute i find that the packets seems reach N7k1 and reach wlc that is connected to its own port but not to the WLC that is connected to N7k2 due to the packet travesing through peer-link and dropping at the peer-link.

Now what is the best practise to sort this out and reach both WLC at the same time ?  Do i move the WLC 2  to N7k1 ?

any thoughts please?

Thanks

[pille1234]

I'm not sure what the problem is here. The best practise probably is to not have any orphan ports, as they may get isolated in case of peer link failure.

Further information regarding vpc best practices can be found here

http://www.cisco.com/en/US/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf

That being said, the design should work anyway. The vpc loop prevention is not applied on orphan ports, because depending on the traffic flow packets need to traverse the peer link and orphan devices are not attached via vpc member ports.

Make sure the ports are considered to be orphan ports:

show vpc orphan-ports

If the ports are shown correctly, can you ping the devices from the respectively directly connected N7k? Do you have an allow-list configured for the peer-link?

Regards

Pille

[Network Pro]

Thanks so are you saying that for orphan ports packets are allowed to traverse through peer - link ?

just to confirm the wlc 1 and wlc 2 are orphan ports

I am confused here because on doing a tracert i can see that the packet gets to core1 and cant go any further to reach wlc 2. but there is no problems reaching wlc 1. However if i do a extended ping from the core using svi 10 to reach wlc 2 (from core 1) i dont have any problems ?

so could you please let me know can i have orphan ports with hsrp and vpc ? also whats the best practice to get this resolved ?

This is pretty much our topology (attached diagram)

[pille1234]

I am confused here because on doing a tracert i can see that the packet gets to core1 and cant go any further to reach wlc 2. but there is no problems reaching wlc 1. However if i do a extended ping from the core using svi 10 to reach wlc 2 (from core 1) i dont have any problems ?

To rule out any connectivity issues, could you do the following and share the results:

on N7k1: ping 10.10.100.11 source 10.10.10.2

on N7k1: ping 10.10.100.12 source 10.10.10.2

on N7k2: ping 10.10.100.11 source 10.10.10.3

on N7k2: ping 10.10.100.12 source 10.10.10.3

Next question, where is the routing instance for vlan 100? Do the N7ks have a vlan interface for vlan 100 as well? In this case repeat the above with source ip from vlan 100. Different results?

[Network Pro]

hi,

i had already tried this and i can get

on N7k1: ping 10.10.100.11 source 10.10.10.2  YES

on N7k1: ping 10.10.100.12 source 10.10.10.2   YES

on N7k2: ping 10.10.100.11 source 10.10.10.3   YES

on N7k2: ping 10.10.100.12 source 10.10.10.3   YES

the strange thing is that from the edge switch i have tried ping 10.10.100.11 source 10.10.10.10 and this works and so does 10.10.100.12 source 10.10.10.10 works

only thing that does not work is from computers connected to edge switch.  when i do a tracert from a pc connected to edge switch  i can see that it goes to nexus 1 and drops if have to get to wlc2 and if i have to get to wlc1 its fine.

The strange thing is that if i try from a differenct pc connectged to same switch i can get to wlc 2. tracert says thorough nexus 2 -> wlc 2 but drops at nexus 2 if i have to get to wlc 1

the routing is on the core and vlan 10 is hsrp address  (10.10.10.1) being active on core 2. vlan 100 is also hsrp on core being active on core 1(10.10.100.1) and has SVI of 10.10.100.2 and .3 resp

this is strange.... so does it mean we cant connect single attached devices at all ?

the rule says:

vPC loop avoidance rule states that traffic coming from vPC member port, then crossing vPC peer-link is NOT

allowed to egress any vPC member port; however it can egress any other type of port (L3 port, orphan port, …).

so even if the pc does hashing and sends traffic  destined for wlc2 through nexus 1, the packets should work as per above rule because the packets are allowed to cross vpc peer link and egress to orphan port, isnt it ?

[pille1234]

the routing is on the core and vlan 10 is hsrp address  (10.10.10.1) being active on core 2. vlan 100 is also hsrp on core being active on core 1(10.10.100.1) and has SVI of 10.10.100.2 and .3 resp
this is strange.... so does it mean we cant connect single attached devices at all ? 

No, it means there is a problem that has not yet been identified.

Could you post the output of the following commands:

show vpc brief

show vpc orphan-ports

show hsrp interface vlan 10

show hsrp interface vlan 100

and from both N7k

show int po XX trunk (whatever the portchannel to the edge switch is)

I don't know the WLC, do they have a CLI? Can you check the ARP cache and MAC table? If so, clear both, do a ping from a Computer that does not work and show the output of ARP-Cache and MAC Address table from the WLC.

 so even if the pc does hashing and sends traffic  destined for wlc2 through nexus 1, the packets should work as per above rule because the packets are allowed to cross vpc peer link and egress to orphan port, isnt it ?

Correct.

[David Lucas]

Hello,

Do you have peer-gateway configured on the vPC?  This is becoming the norm for vPC installs, but want to confirm you have it enabled.

http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx-os/interfaces/configuration/guide/if_vPC.html#wp1558675

Depending on the WLC version you are running, it's possible that you need peer-gateway.

Check out:

CSCtj58064

Dave

[pille1234]

Depending on the WLC version you are running, it's possible that you need peer-gateway.

Interesting read, but does it explain the problem? Wouldn't the reverse packet be forwarded anyway even if the bug applies and even if peer-gateway is not configured, albeit with a a suboptimal path?

[David Lucas]

Interesting read, but does it explain the problem? Wouldn't the reverse packet be forwarded anyway even if the bug applies and even if peer-gateway is not configured, albeit with a a suboptimal path?

When the WLC sends to the physical mac address of the peer of the active forwarding device it will be dropped due to the loop prevention built into vPC as it states in the document I posted.

"Packets reaching a vPC device for the non-local router MAC address are sent across the peer-link and could be dropped by the built in vPC loop avoidance mechanism if the final destination is behind another vPC."

The problem is that the WLC uses the HSRP physical address and not the virtual mac address.  In vPC this breaks communication.  So, with peer-gateway enabled it allows the active gateway to forward the packet on behalf of it's peer.

Just to clarify do you have peer-gateway enabled or not?  If you don't then my recommendation would be enable it.

http://www.cisco.com/en/US/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf

Refer to page 110 and 111 for the recomendation.

Dave

[pille1234]

When the WLC sends to the physical mac address of the peer of the active forwarding device it will be dropped due to the loop prevention built into vPC as it states in the document I posted.
"Packets reaching a vPC device for the non-local router MAC address are sent across the peer-link and could be dropped by the built in vPC loop avoidance mechanism if the final destination is behind another vPC."

My confusion is not about peer-gateway or the bug-id provided, this is well understood.

My confusion is caused by another aspect: it is my understanding that the loop prevention only kicks in, if the packet enters via vPC-link, crosses the peer-link and finally is supposed to leave via another vPC link. This situation does not apply here, because the WICs are not connected via vPC, but via orphaned port, thus the return packet should be forwarded southbound via vPC. Do you disagree?

[David Lucas]

I agree that in this case that the loop prevention will not kick in.  It also sates that in that document I provided:

"vPC loop avoidance rule states that traffic coming from vPC member port, then crossing vPC peer-link is NOT allowed to egress any vPC member port; however it can egress any other type of port (L3 port, orphan port, …)."

What I'm thinking is that the WLC is not using the HSRP VIP but rather the phyical address and this can cause the problem you are seeing.  Peer-gateway will resolve this.

Hence why I'd like to know if you have peer-gateway enabled or not?

Dave

[pille1234]

What I'm thinking is that the WLC is not using the HSRP VIP but rather the phyical address and this can cause the problem you are seeing.  Peer-gateway will resolve this.

Assuming the WLC would do that, address the packet with the physical MAC of the far side N7k instead of the virtual GW-MAC, then what? The packet would be switched to the second N7k and then routed to VLAN 100 and send southbound, wouldn't it?

Hence why I'd like to know if you have peer-gateway enabled or not?

As I myself have no good explanation I agree we should check this and in case it is not configured it should be tested. However I believe it should not have any impact. If it has an impact I can't explain why, that is why I'm so pettifogging here.

[Network Pro]

hi David / pille,

We DONT have peer gateway configured. but i thought this is only need for netapp servers. i am having the same confusion as pille. These are orphan ports so they should be having no problems routing via peer-gateway.

By the way adding the above commands will they affect the production network, do they need downtime ? will this affect any other services - because we run a 365. 24/7 network with lowest ever possible downtime

[David Lucas · Accepted solution]

Hello,

Peer-gateway is needed anytime a device uses the physical MAC address rather than the HSRP virtual Mac in a vPC environment.

As I have said in previous posts, it's becoming the norm to always enable it even if all the devices always use the virtual MAC address (check the vPC document for best practices).

Lets work on getting peer-gateway enabled and then figure why it is needed after to make sure it resolves the problem (I'm betting it will).

To answer your question about enabling it in production. It should always be done in a maint window but with that said, if you read the vPC document I think your question will be answered.

Dave

Sent from Cisco Technical Support iPhone App

[Network Pro]

Thanks i will enable this and come back to you