inter-VLAN routing problems on a multilayer vPC environment

[s-durando]

Hi,

you find attached my network architecture with 2 Nexus 7010 on core layer and 2 Nexus 5020 on distribution layer, each one with 1 N2148T fabric extender switch. PC-A1 and PC-A2 are connected to one N2148T, PC-B1 is connected to the other N2148T. Nexus-7000-1 is HSRP Active for all VLANs, Nexus-7000-2 is HSRP standby.

PC-A1 and PC-A2 are connected to VLAN A, PC-B1 is connected to VLAN B. PC-A1 and PC-A2 have the same default gateway correspondent to IP HSRP on VLAN A.

It happens that PC-A1 is able to ping PC-B1 while PC-A2 is unable to ping PC-B1. If I issue a traceroute from PC-A2 I see Nexus-7000-2’s physical IP address as the first hop even if Nexus-7000-2 is HSRP standby. After the first hop the traceroute is lost.

If I shutdown Port-channel 20 on Nexus-5000-2, PC-A2 starts to ping PC-B1.

I can’t understand what’s wrong in this architecture.

Regards

Stefano

[IAN WHITMORE]

Can PC-A1 ping PC-A2?

Can you post the port configs.

Also check the port-channels are allowing the same vlans on the trunks and recheck config (although if PC-A1 works then it should be OK). Still never hurts to double check.

Could be an STP problem. What version of STP are you running? On all switches? Who is the root of each VLAN? (Again, PC-A1 is working which makes it abit of a strange problem, but double check).

Have you tried clearing the mac-addresses / arp cache?

[s-durando]

PC-A1 can ping PC-A2

Port configs are standard:

interface Ethernet100/1/22
  description PC-A1

  switchport access vlan 930

interface Ethernet100/1/24
  description PC-A2

  switchport access vlan 930

interface Ethernet101/1/21
  descriptionPC-A3

  switchport access vlan 978

I double checked all configs, per-VLAN RSTP is running on every switch and Nexus-7000-1 is root bridge for every vlan.

I also tried clearing the mac-addresses and arp cache

Stefano

[IAN WHITMORE]

OK, these ar the killers when one thing works and the other doesn't for no apparant reason. I got a couple more question / suggestions.

Do a #sh int trunk on your switches and make sure that all the vlans are being passed on the trunks as you expected. I once had an issue between Cisco and HP Blade ethernet switches where the trunks didn't negotiate properly and I had to do a shut no shut on the Cisco interface. However, like I said, they were HP, not like in this case.

If you swap PC-A1 and PC-A2 over, does the same thing happen?

Have you tried a different PC in PC-A2 port?

Have you tried PC-A2 in a different port?

You say it works if you shut down Po20, what if you leave Po20 and shut Po10, what happens then?

Make sure all your port-channel interface are up. I once had an issue similar and it was because one of my links was in err-disable after a spanning-tree issue. In theory, being an ether.channel it shouldn't matter, but it did. When I brought up the link again everything worked fine. SOme PCs worked and others didn't....beats me. So make sure that none of your channel links on the switches is in disable or err-disable.

That's a lot of questions and testing...let me know how you get on.

Regards,

Ian

If I have helped in any way, please rate me.

[s-durando]

Hi,

the problem has been solved by activating peer-gateway feature on vpc domain.

Host not reachable was sending traffic to the physical MAC address of the Nexus 7010, rather than the virtual HSRP address. That is causing packets to be dropped in the reverse direction.

Stefano

[kbyrd]

I also had this problem with a pair of Nexus 7010s in a vPC environment. I have a number of traditional top of rack switches - 2360s and 4948s - that are port-channeled to the Nexus 7010s. Only within the routing domain of the 7010 - traffic sourced from one VLAN to another on the 7010 - did I experience this problem. I could ping some devices and not others. If I went a Layer 3 hop away or more as my source or destination IP, no problems. 7010 #1 is always HSRP active for all VLANs and 7010 #2 is always HSRP standby.

I simply enabled the peer-gateway command on both Nexus 7010 under the vpc domain xx command and it magically works.

[Steven Williams]

peer-gateway and peer-switch are two very useful commands.