cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4467
Views
4
Helpful
16
Replies

Intra-VLAN traffic not passing back to Nexus 5k

rsjordan00
Level 1
Level 1

We recently extended our access layer using a pair of 5ks with extenders. We have a pair of 6509s at our core and they handle the intra-VLAN routing with SVIs. I recently noticed that access hosts connected to the extenders cannot pass traffic between each other if they are in different VLANs.

The strange thing is these same hosts can ping devices in other VLANs as long as the other devices are not connected to the 5k environment.

For example, consider the following hosts. Each host has their gateway set to the appropriate SVI on our core.

HostA - VLAN100 - connected to 5k extender

HostB - VLAN200 - connected to 5k extender

HostC - VLAN100 - connected to 2960 off our core

HostD - VLAN200 - connected to 2960 off our core

Each host can ping each other with the exception of HostA and HostB.

As for specifics, we use HSRP (no VSS) between our cores.

Core1 interface

interface Port-channel31

description 5k-vPC-101

switchport

switchport trunk encapsulation dot1q

switchport trunk native vlan 50

switchport mode trunk

no ip address

5k interface to Core1:

interface port-channel101

  description vPC-Uplink-Core1

  switchport mode trunk

  switchport trunk native vlan 50

  spanning-tree cost 24000

  speed 10000

  vpc 101

Core2 interface:

interface Port-channel31

description 5k-vPC-102

switchport

switchport trunk encapsulation dot1q

switchport trunk native vlan 50

switchport mode trunk

no ip address

5k interface to Core2:

interface port-channel102

  description vPC-Uplink-Core2

  switchport mode trunk

  switchport trunk native vlan 50

  spanning-tree cost 28000

  speed 10000

  vpc 102

I setup a span with the following sources:

5k1: po101

5k2: po102

Core1: po31

When I ping between hostA and hostB, I see the egress packets on either 5k1 or 5k2. I then see ingress AND egress on Core1. There are no ingress packets on 5k1 or 5k2.

The egress packets from Core1 show the correct destination MAC address of the target host. The mac address table shows the mac address on po31.

At this point, I'm totally stumped. Is anyone aware of known issues or additional troubleshooting steps I can take?

16 Replies 16

Reza Sharifi
Hall of Fame
Hall of Fame

Is vPC running between the 5ks?

Is vPC running between the 5ks and extenders?

What is output of "sh vpc"

What device is the root and backup root?

sh spann root

HTH

We are running vPC between the 5ks (vpc50).

vPC is running between the extenders in a dual-homed method where each extender connects to both 5ks.

Below is sh vpc on our primary 5k. The output on our secondary is identical besides the role.

5k-primary# sh vpc

Legend:

                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1  

Peer status                       : peer adjacency formed ok     

vPC keep-alive status             : peer is alive                

Configuration consistency status  : success

Per-vlan consistency status       : success                      

Type-2 consistency status         : success

vPC role                          : primary, operational secondary

Number of vPCs configured         : 394

Peer Gateway                      : Disabled

Dual-active excluded VLANs        : -

Graceful Consistency Check        : Enabled

Auto-recovery status              : Disabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans   

--   ----   ------ --------------------------------------------------

1    Po50   up     1,3,5,9-10,12,14-15,50,99-100,102,230,302,318,337,       

                   346-348,394-395,400-600                                  

vPC status

----------------------------------------------------------------------------

id     Port        Status Consistency Reason                     Active vlans

------ ----------- ------ ----------- -------------------------- -----------

101    Po101       up     success     success                    1,3,5,9-10,

                                                                 12,14-15,50

                                                                 ,99-100,102

                                                                 ,230,302,31

                                                                 8,337,34....

102    Po102       up     success     success                    1,3,5,9-10,

                                                                 12,14-15,50

                                                                 ,99-100,102

                                                                 ,230,302,31

                                                                 8,337,34....

The show span root is below. I abreviated it because of the amount of VLANs:

5k-primary# sh spanning-tree root

                                        Root  Hello Max Fwd

Vlan                   Root ID          Cost  Time  Age Dly  Root Port

---------------- -------------------- ------- ----- --- ---  ----------------

VLAN0001          8200 001f.0000.0c01   24001    2   14  10    port-channel50

VLAN0100          8200 001f.0000.0c03   24001    2   14  10    port-channel50

VLAN0200          8200 001f.0000.0c05   24001    2   14  10    port-channel50

5k-secondary# sh spanning-tree root

                                        Root  Hello Max Fwd

Vlan                   Root ID          Cost  Time  Age Dly  Root Port

---------------- -------------------- ------- ----- --- ---  ----------------

VLAN0001          8200 001f.0000.0c01   24000    2   14  10   port-channel101

VLAN0100          8200 001f.0000.0c03   24000    2   14  10   port-channel101

VLAN0200          8200 001f.0000.0c05   24000    2   14  10   port-channel101

I'm a little confused on why my primary 5k is using the peer link (po50), instead of po101. Would this cause the 5k to drop the packet to avoid some kind of loop?

Masked the actual switch name in the output

Can you post the vpc domain setup?

     is there the command peer-gateway ?

My previous post has the output of sh vpc. You can see that peer-gateway is disabled. I did a little research and it seems this is only needed when you have certain storage systems that don't follow Ethernet standards.

The only config specific to the vpc domain is as follows:

5k-primary#

vpc domain 1

  role priority 2000

  peer-keepalive destination 172.31.255.242

5k-secondary#

vpc domain 1

  peer-keepalive destination 172.31.255.241

Guido,

Peer-gateway is used when the device is running layer-3, not just layer-2, which is the case here.

HTH

deleted

EDIT:

Ups, after reading again I think this was a misunderstanding. never mind...

I'm a little confused on why my primary 5k is using the peer link  (po50), instead of po101. Would this cause the 5k to drop the packet to  avoid some kind of loop?

I have the same scenario except I use 7ks instead of 6500 for routing and was confused too until I saw this in the vPC design document:

For vPC ports only, the operational primary switch generates and processes BPDUs. The operational

secondary switch forwards BPDUs to the primary switch.

So, it is also recommended to match vPC primary with the root switch and vPC secondary with the backup root switch if the same device plays both roles.

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572834-00_STDG_NX-OS_vPC_DG.pdf

HTH

rsjordan00 schrieb:


I'm a little confused on why my primary 5k is using the peer link (po50), instead of po101. Would this cause the 5k to drop the packet to avoid some kind of loop?

Masked the actual switch name in the output

That is exactly your problem: vPC loop-prevention drops your packets. Question is, why isn't port-channel 101 the root port for 5k-primary? Do a 'show spanning-tree vlan 100', a 'show running-config int po101' and a 'show port-channel summary'  on both N5k please.

Regards.

Well, the output of sh vpc on my 5k-primary says "operational secondary". Would this explain why the vPC is the root port on this 5k? Here is the output from the requested commands:

5k-primary# sh spanning-tree vlan 100

VLAN0100

  Spanning tree enabled protocol rstp

  Root ID    Priority    8200

             Address     001f.0000.0c64

             Cost        24001

             Port        4145 (port-channel50)

             Hello Time  2  sec  Max Age 14 sec  Forward Delay 10 sec

  Bridge ID  Priority    32868  (priority 32768 sys-id-ext 100)

             Address     547f.0000.4efc

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po50             Root FWD 1         128.4145 (vPC peer-link) Network P2p

Po101            Root FWD 24000     128.4196 (vPC) P2p Peer(STP)

Po102            Altn BLK 28000     128.4197 (vPC) P2p Peer(STP)

Eth113/1/13      Desg FWD 1         128.2701 (vPC) Edge P2p

Eth114/1/5       Desg FWD 1         128.2821 (vPC) Edge P2p

Eth114/1/6       Desg FWD 1         128.2822 (vPC) Edge P2p

Eth114/1/7       Desg FWD 1         128.2823 (vPC) Edge P2p

Eth114/1/8       Desg FWD 1         128.2824 (vPC) Edge P2p

5k-primary# sh run int po101

!Command: show running-config interface port-channel101

!Time: Fri Mar 29 18:40:03 2013

version 5.2(1)N1(3)

interface port-channel101

  description vPC-Uplink-Core1

  switchport mode trunk

  switchport trunk native vlan 50

  spanning-tree cost 24000

  speed 10000

  vpc 101

5k-primary#

5k-primary#

5k-primary# sh port-channel summary

Flags:  D - Down        P - Up in port-channel (members)

        I - Individual  H - Hot-standby (LACP only)

        s - Suspended   r - Module-removed

        S - Switched    R - Routed

        U - Up (port-channel)

        M - Not in use. Min-links not met

--------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

      Channel

--------------------------------------------------------------------------------

50    Po50(SU)    Eth      LACP      Eth1/27(P)   Eth1/28(P)   Eth1/29(P)

                                     Eth1/30(P)  

101   Po101(SU)   Eth      NONE      Eth1/31(P)  

102   Po102(SU)   Eth      NONE      Eth1/32(P)  

111   Po111(SU)   Eth      NONE      Eth1/1(P)   

112   Po112(SU)   Eth      NONE      Eth1/2(P)   

113   Po113(SU)   Eth      NONE      Eth1/3(P)   

114   Po114(SU)   Eth      NONE      Eth1/4(P)   

121   Po121(SU)   Eth      NONE      Eth1/5(P)   

122   Po122(SU)   Eth      NONE      Eth1/6(P)   

123   Po123(SU)   Eth      NONE      Eth1/7(P)   

124   Po124(SU)   Eth      NONE      Eth1/8(P)   

5k-secondary# sh spanning-tree vlan 100

VLAN0100

  Spanning tree enabled protocol rstp

  Root ID    Priority    8200

             Address     001f.0000.0c64

             Cost        24000

             Port        4196 (port-channel101)

             Hello Time  2  sec  Max Age 14 sec  Forward Delay 10 sec

  Bridge ID  Priority    32868  (priority 32768 sys-id-ext 100)

             Address     547f.0000.2f01

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po50             Desg FWD 1         128.4145 (vPC peer-link) Network P2p

Po101            Root FWD 24000     128.4196 (vPC) P2p Peer(STP)

Po102            Altn BLK 28000     128.4197 (vPC) P2p Peer(STP)

Eth113/1/13      Desg FWD 1         128.2701 (vPC) Edge P2p

Eth114/1/5       Desg FWD 1         128.2821 (vPC) Edge P2p

Eth114/1/6       Desg FWD 1         128.2822 (vPC) Edge P2p

Eth114/1/7       Desg FWD 1         128.2823 (vPC) Edge P2p

Eth114/1/8       Desg FWD 1         128.2824 (vPC) Edge P2p

5k-secondary# sh running-config int po101

!Command: show running-config interface port-channel101

!Time: Fri Mar 29 18:42:49 2013

version 5.2(1)N1(3)

interface port-channel101

  description vPC-Uplink-Core1

  switchport mode trunk

  switchport trunk native vlan 50

  spanning-tree cost 24000

  speed 10000

  vpc 101

5k-secondary# sh port-channel summary

Flags:  D - Down        P - Up in port-channel (members)

        I - Individual  H - Hot-standby (LACP only)

        s - Suspended   r - Module-removed

        S - Switched    R - Routed

        U - Up (port-channel)

        M - Not in use. Min-links not met

--------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

      Channel

--------------------------------------------------------------------------------

50    Po50(SU)    Eth      LACP      Eth1/27(P)   Eth1/28(P)   Eth1/29(P)

                                     Eth1/30(P)  

101   Po101(SU)   Eth      NONE      Eth1/31(P)  

102   Po102(SU)   Eth      NONE      Eth1/32(P)  

111   Po111(SU)   Eth      NONE      Eth1/1(P)   

112   Po112(SU)   Eth      NONE      Eth1/2(P)   

113   Po113(SU)   Eth      NONE      Eth1/3(P)   

114   Po114(SU)   Eth      NONE      Eth1/4(P)   

121   Po121(SU)   Eth      NONE      Eth1/5(P)   

122   Po122(SU)   Eth      NONE      Eth1/6(P)   

123   Po123(SU)   Eth      NONE      Eth1/7(P)   

124   Po124(SU)   Eth      NONE      Eth1/8(P)   

Well, the output of sh vpc on my 5k-primary says "operational secondary". Would this explain why the vPC is the root port on this 5k? Here is the output from the requested commands:

5k-primary# sh spanning-tree vlan 100

VLAN0100

  Spanning tree enabled protocol rstp

  Root ID    Priority    8200

             Address     001f.0000.0c64

             Cost        24001

             Port        4145 (port-channel50)

             Hello Time  2  sec  Max Age 14 sec  Forward Delay 10 sec

  Bridge ID  Priority    32868  (priority 32768 sys-id-ext 100)

             Address     547f.0000.4efc

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po50             Root FWD 1         128.4145 (vPC peer-link) Network P2p

Po101            Root FWD 24000     128.4196 (vPC) P2p Peer(STP)

Po102            Altn BLK 28000     128.4197 (vPC) P2p Peer(STP)

Eth113/1/13      Desg FWD 1         128.2701 (vPC) Edge P2p

Eth114/1/5       Desg FWD 1         128.2821 (vPC) Edge P2p

Eth114/1/6       Desg FWD 1         128.2822 (vPC) Edge P2p

Eth114/1/7       Desg FWD 1         128.2823 (vPC) Edge P2p

Eth114/1/8       Desg FWD 1         128.2824 (vPC) Edge P2p

Alright, I misinterpreted the output of 'sh spanning-tree root' that you showed earlier. What we see here is exactly as it is supposed to be: peerlink and vPC to Core are root ports. This is the correct output for vPC secondary devices.

However the Type "Peer(STP)" is confusing. I don't know if it is related to your problem, but from the look of it I believe you are using a different type of STP on your Core switches. Is this done with intention?

Our access switches are set to rpvst but our core switches run pvst. Not for any particular reason, we just don't want to take the massive hit (~100 switches) to configure our core to rpvst.

I opened a TAC with Cisco and they believe it could be related to the load balancing algorithm on the core's etherchannel. On both sides of the etherchannel we are using the default src-dst-ip. They want to change it to src-dst-mac, but I need more justification before I make a change that will disrupt all my etherchannels. I'm waiting to hear back from them.

Our access switches are set to rpvst but our core switches run pvst. Not for any particular reason, we just don't want to take the massive hit (~100 switches) to configure our core to rpvst.

That's certainly another topic, but doesn't that drastically increase convergence times?

I opened a TAC with Cisco and they believe it could be related to the load balancing algorithm on the core's etherchannel. On both sides of the etherchannel we are using the default src-dst-ip. They want to change it to src-dst-mac, but I need more justification before I make a change that will disrupt all my etherchannels. I'm waiting to hear back from them.

While I don't believe that changing the lb methods causes disruption of traffic, let alone the disruption of your port channels, I agree that this 'solution' does not sound very reasonable, except Cisco has knowledge about a specific software bug that we don't know about.

Do you have a specific reason for configuring the spanning tree cost on the uplink ports?

You don't use LACP for your uplinks, but can we rule out a simple wiring error anyway? Have you checked with cdp neighbors? I'm just guessing a little...

Regards

Pille

I leaned back on them about the LB algorithm and they agreed there is not enough evidence to warrant this change. The engineer told me there would be a disruption if I were to change it.

We configured the spanning tree cost because both 5ks are uplinked to our 6500 using 10g. The interswitch link between our cores is a 4 member etherchannel on 1g. We set a higher cost on the uplinks otherwise traffic between our cores would go through the 5ks.

Tonight, I'll be shutting the interface from core1 to 5k2-secondary. If my pings work then it at least narrows the problem a little further.

You cant route to VPC peers as long as you're using a VPC port-channel, just use a plain p2p link or a L3 port-channel to a each Nexus.This should resolve ur issue.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: