VPC between N5K and 4506

axailantoulouse · ‎04-12-2013

Hi,

I have several VPC vlans on my N5k to UCS and all is going OK.

Each N5k have an uplink (via a Fex N2K) to a 4506:

4506

/ \

N2K1 N2K2

| |

N5K1-----N5K2

Here is the configuration for 4506 side:

interface GigabitEthernet3/21

description toN2K1 port 1

switchport trunk allowed vlan 300,302,701,702,2700

switchport mode trunk

channel-group 2 mode active

end

interface GigabitEthernet5/27

description toN2K2 port 1

switchport trunk allowed vlan 300,302,701,702,2700

switchport mode trunk

channel-group 2 mode active

end

interface Port-channel2

description TO_N5K

switchport

switchport trunk allowed vlan 300,302,701,702,2700

switchport mode trunk

end

-------------------

Here is the configuration for N5K1 (N2K1 port):

interface Ethernet101/1/1

description To_4506

switchport mode trunk

switchport trunk allowed vlan 300,302,701-702,2700

speed 1000

channel-group 1 mode active

interface port-channel1

descriptionTo_4506

switchport mode trunk

switchport trunk allowed vlan 300,302,701-702,2700

speed 1000

vpc 1

interface port-channel2

description vPC peer-link

switchport mode trunk

switchport trunk allowed vlan 1,300-306,701-702,999,2700

spanning-tree port type network

speed 10000

vpc peer-link

------------------------

Here is the configuration for N5K2 (N2K2 port):

interface Ethernet102/1/1

description To_4506

switchport mode trunk

switchport trunk allowed vlan 300,302,701-702,2700

speed 1000

channel-group 1 mode active

interface port-channel1

description To_4506

switchport mode trunk

switchport trunk allowed vlan 300,302,701-702,2700

speed 1000

vpc 1

interface port-channel2

description vPC peer-link

switchport mode trunk

switchport trunk allowed vlan 1,300-306,701-702,999,2700

spanning-tree port type network

speed 10000

vpc peer-link

---------------------

My issue is a consistency error.

The left trunk of my scheme (3/21 of 4506 to e 101/1/1 on N2k1) is going down each time.. I see in logs down immediatly after a no shut.

The status of a "sh int" is "Suspended by VPC".

The VPC error is (sh vpc 1) :

vPC status

----------------------------------------------------------------------------

id Port Status Consistency Reason Active vlans

------ ----------- ------ ----------- -------------------------- -----------

1 Po1 up failed vpc port channel 300,302,702

mis-config due to vpc ,2700

links in the 2 switches

connected to different

partners

I tried several action: removing from the Peer-link and add again, shut/no shut on each side.. (no reboot, switches are in production) but the error is still there..

Before that, the Two ports on the 4506 were in differents PO.

3/21 was in po1 and 5/27 was in po2.

On all the switches of our network (some N7K..), I only have an error on this uplink..

Any ideas or clues?

A command exists to show the Mac @ learn by VPC from a partner?

Thanks,

BR

Steve Fuller · ‎04-12-2013

Hi Nicolas,

The error "due to vpc links in the 2 switches connected to different partners" would seme to suggest that the Nexus switches are receiving two different LAG ID on each of the ports to the Catalyst 4500.

Can you run the command show lacp neighbor when one of the interfaces is up e.g., eth101/1/1 and check the LAG ID, then disable eth101/1/1 and enable eth102/1/1 and then run the show lacp neighbor again?

Also is the configuration from the FEX ports and the Catalyst 4506 the complete configuration? I'm intrigued how either link remains operational when the FEX host interfaces have BPDU guard enabled by default, and no configuration option to disable it.

When we connect downstream switches to FEX ports we use the spanning-tree bpdufilter enable command otherwise the links are disabled.

Regards

Steve Fuller · ‎04-12-2013

Hi Nicolas,

The other thing that looks odd when I look closely at this is that the active VLANs shown in the output of the sh vpc command does not include VLAN 701, but only 300, 302, 702 and 2700.

I can see VLAN 701 is configured on the port-channel interfaces between the FEX and the Catalyst 4506, but has it been configured with the vlan command on all three switches?

Also is it by design that VLAN 303-306 and VLAN 999 are configured on the vPC peer link only? I presume these are used on vPC to other switches, but wanted to confirm.

Regards

axailantoulouse · ‎04-12-2013

Hi Steve,

Thanks for your answer.

Sorry I'm out of the office (France, so UTC+2)

First I will test the lacp neighbor command but I have to wait that all my users are gone to swap trunks ports on Monday evening. But it seems to be the best clue according to the vpc error. Perhaps also, because the two ports were before in different Port-channels.

The N5K is linked to our UCS, so to our servers.

For Span, I totally agree, I have this span issue on other N2K with a 3750 (bpdu filter mandatory).

I have to check monday but if I remember well, the span is disable for the trunked vlans to the 4500.

Anyway, I don't see an "err-disable:bpduguard" in logs for this interface but only a "supsended by vpc".

For the vlan 701 it is well configured Layer 2 on each switch, also L3 on the 4506 with the vlan interface for it.

You're right for others vlans, our servers vlans.

Our UCS are cross-linked to the N5K and some vlans are only used for them (Vmotion for example).

Theses Vlans are also used with port-channels from N5k to our UCS but I can't post the whole config on the forum

Sorry for my english, I hope you understand me well

I'll check everything monday.

Have a good Week-end,

Regards,

axailantoulouse · ‎04-16-2013

Hi,

I just swap the uplink port (shut on 4506 gi 5/27 and no sh gi 3/21) and doing that the lacp was completly down..

After few seconds (and some Up, Down), my two trunks from Nexus to 4506 are ok

I think that the lacp neighboring was wrong because the interface I shut (gi 5/27 on 4506) was before in another Port-channel.. That's probably why even with the other one shut (Gi 3/21) the port-channel was down..

So changing a channel-group for an interface participating in a port-channel with a VPC is Wrong ! Shut and no shut is mandatory in that context..

Thanks steeve anyway for your advices.

Br,