VM host issues connected to pair of VPC switches using HSRP

CiscoBrownBelt · ‎06-20-2018

So I have a couple hosts that continuous pings (testing do to deployment issues) fail mostly from doing pings from servers.

Just have a pair of 3548 Nexus in VPC domain (configured to be active active) and routing for the subnet hosts are in is done via SVI-HSRP. Here are pertaining configs. Any help guys?

Same on both switches except:

vpc domain 1

role priority 100 (same for both switches)

peer-keepalive destination 172.168.1.1 source 172.168.1.2 vrf vpc-keepalive

dual-active exclude interface-vlan 5

peer-gateway

auto-recovery

interface port-channel1

vpc peer-link

-----------

SVI configs

Same on both switches except where noted:

interface Vlan2

description ServerHost

no shutdown

no ip redirects

ip address 172.168.2.2/24

hsrp 2

preempt

priority 105 (priority default on Switch 2)

ip 172.16.2.1

---------------

Same on both switches except

interface Ethernet1/45 and 46

description vPC Peer-Link

switchport mode trunk

channel-group 1 mode active

no shutdown

-------------------------

Interfaces for server ports:

interface Ethernet1/1 and 2

speed 1000

description ServerHost

switchport mode trunk

switchport trunk allowed vlan 2

switchport trunk native vlan 12

spanning-tree port type edge trunk

no shutdown

------------------------------------------

SHOW Commands to help T-shoot:

show mac address-table (trouble host mac)

Sw_A# sh mac address-table | inc 03a0.9756.056d

* 214 03a0.9756.056d dynamic 0 F F Po1

Sw_B# sh mac address-table | inc 03a0.9756.056d

* 214 03a0.9756.056d dynamic 0 F F Po1

Sw_A# sh port-channel traffic

ChanId Port Rx-Ucst Tx-Ucst Rx-Mcst Tx-Mcst Rx-Bcst Tx-Bcst

------ --------- ------- ------- ------- ------- ------- -------

1 Eth1/35 0.07% 0.80% 2.25% 3.75% 0.12% 0.52%

1 Eth1/36 99.92% 99.19% 97.74% 96.24% 99.87% 99.47%

Sw_B# sh port-channel traffic

ChanId Port Rx-Ucst Tx-Ucst Rx-Mcst Tx-Mcst Rx-Bcst Tx-Bcst

------ --------- ------- ------- ------- ------- ------- -------

10 Eth1/35 55.87% 19.73% 16.15% 47.77% 98.40% 99.75%

10 Eth1/36 44.12% 80.26% 83.84% 52.22% 1.59% 0.24%

balaji.bandi · ‎06-20-2018

Firstly, i would like to see full config of both the switches. some picture how your peer links connected back to back or any mediated switch ?

I do not see you allowing VLAN 2 in the vpc peer link.

what kind of spanning tree you are using (config should be network for the peer-link)

show output :

show vpc

show spanning tree

show run inter por 1

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

CiscoBrownBelt · ‎06-21-2018

Thanks bro!

Sorry I can't really just past all config on here but I have provided all that I think is applicable as there are no other real configs on them. I have attached a drawing it is very basic topology - 2 switches 1 server.

The peer link is trunk allowing all vlans (Po1). Below are output you asked for hope it helps. I am thinking problem is on server side.

vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 0 Peer Gateway : Enabled Peer gateway excluded VLANs : - Dual-active excluded VLANs : 5 Graceful Consistency Check : Enabled Operational Layer3 Peer-router : Disabled Auto-recovery status : Enabled (timeout = 240 seconds) vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 Po1 up 1,2,3,5

----------------------------------

Here is spanning tree.

Sw1# sh spanning-tree

VLAN0001

Spanning tree enabled protocol rstp

Root ID Priority 32769

Address 00d7.8faa.5cfc

This bridge is the root

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)

Address 00d7.8faa.5cfc

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1 Desg FWD 2 128.4105 (vPC peer-link) Network P2p

VLAN002

Spanning tree enabled protocol rstp

Root ID Priority 32982

Address 00d7.8faa.5cfc

This bridge is the root

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32982 (priority 32768 sys-id-ext 214)

Address 00d7.8faa.5cfc

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1 Desg FWD 2 128.4105 (vPC peer-link) Network P2p

Eth1/1 Desg FWD 4 128.135 Edge P2p

Eth1/2 Desg FWD 4 128.136 Edge P2p

VLAN003

Spanning tree enabled protocol rstp

Root ID Priority 32783

Address 00d7.8faa.5cfc

This bridge is the root

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32783 (priority 32768 sys-id-ext 15)

Address 00d7.8faa.5cfc

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1 Desg FWD 2 128.4105 (vPC peer-link) Network P2p

VLAN005

Spanning tree enabled protocol rstp

Root ID Priority 32867

Address 00d7.8faa.5cfc

This bridge is the root

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32867 (priority 32768 sys-id-ext 99)

Address 00d7.8faa.5cfc

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1 Desg FWD 2 128.4105 (vPC peer-link) Network P2p

Sw2# sh spanning-tree

VLAN0001

Spanning tree enabled protocol rstp

Root ID Priority 32769

Address 00d7.8faa.5cfc

Cost 2

Port 4105 (port-channel1)

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)

Address 00d7.8faa.60bc

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1 Root FWD 2 128.4105 (vPC peer-link) Network P2p

VLAN02

Spanning tree enabled protocol rstp

Root ID Priority 32982

Address 00d7.8faa.5cfc

Cost 2

Port 4105 (port-channel1)

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32982 (priority 32768 sys-id-ext 214)

Address 00d7.8faa.60bc

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1 Root FWD 2 128.4105 (vPC peer-link) Network P2p

Eth1/1 Desg FWD 4 128.135 Edge P2p

Eth1/2 Desg FWD 4 128.136 Edge P2p

VLAN003

Spanning tree enabled protocol rstp

Root ID Priority 32783

Address 00d7.8faa.5cfc

Cost 2

Port 4105 (port-channel1)

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32783 (priority 32768 sys-id-ext 15)

Address 00d7.8faa.60bc

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1 Root FWD 2 128.4105 (vPC peer-link) Network P2p

VLAN005

Spanning tree enabled protocol rstp

Root ID Priority 32786

Address 00d7.8faa.5cfc

Cost 2

Port 4105 (port-channel1)

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32786 (priority 32768 sys-id-ext 18)

Address 00d7.8faa.60bc

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1 Root FWD 2 128.4105 (vPC peer-link) Network P2p

interface port-channel10 (Same on both switches)

speed 10000

description VPC Domain1_PeerLink

switchport mode trunk

spanning-tree port type network

vpc peer-link

Reza Sharifi · ‎06-21-2018

Below is not correct. HSRP virtual IP and the physical address need to be in the same subnet

interface Vlan2

description ServerHost

no shutdown

no ip redirects

ip address 172.168.2.2/24

hsrp 2

preempt

priority 105 (priority default on Switch 2)

ip 172.16.2.1

CiscoBrownBelt · ‎06-21-2018

Sorry that was a type. They are both 172.16 and it pings.

CiscoBrownBelt · ‎06-21-2018

One thing I noticed there are some jumpo frames when show int 1-2 and on the peer-link interfaces. Should I configure system jumbo as below? There are hosts on 10gig ports however the trouble hosts are only on gig ports.

switch(config)#policy-map type network-qos jumboframes
class type network-qos class-default
mtu 9216

system qos
service-policy type network-qos jumboframes

switch(config)#system jumbomtu 9216

!--- Set the MTU specification for an interface.
switch(config)#interface ethernet x/x

!--- By default, Cisco NX-OS configures Layer 3 parameters. In order to configure Layer 2 parameters, use this command.
switch(config-if)#switchport

switch(config-if)#mtu 9216
switch(config-if)#exit

Reza Sharifi · ‎06-21-2018

Although it makes sense to have the same exact jumbo frame config on both switch, this should not have anything to do with Ping failure.

So, from the server, can you ping both physical IPs and also the virtual?

If that is good, can you ping the other way, meaning from the switch, can you ping the IP on the server?

HTH

CiscoBrownBelt · ‎06-21-2018

So from the server it can ping both actual IPs no problem, but when pinging the virtual .1 ip that is where ping fails here and there.

Yes from switch I can ping the two server IPs no problem.

In regards to Jumbo, that was kinda just off subject but also I wondered about. So if I got 10gig ports connected to hosts, don't I need to make sure system mtu 9216 is configured globablly and on the peer-links between both VPC switches? Should QOS for jumbo really be configured as well?

Reza Sharifi · ‎06-21-2018

but when pinging the virtual .1 ip that is where ping fails here and there

Ok, is HSRP flapping? You can find that by using "sh hsrp vlan 2". Can you post the output here?

In regards to Jumbo, that was kinda just off subject but also I wondered about. So if I got 10gig ports connected to hosts, don't I need to make sure system mtu 9216 is configured globablly and on the peer-links between both VPC switches? Should QOS for jumbo really be configured as well?

Yes, you need to enable it globally first and add it to each vlan with QOS policy. I am not sure about the 3000 platform but in some Nexus models (5ks and 6ks), you also have to do the QOS policy globally which means it applies to the whole switch.

HTH

CiscoBrownBelt · ‎06-21-2018

Ok, is HSRP flapping? You can find that by using "sh hsrp vlan 2". Can you post the output here?

I don't think so but I will have to check when I get the chance.

Yes, you need to enable it globally first and add it to each vlan with QOS policy. I am not sure about the 3000 platform but in some Nexus models (5ks and 6ks), you also have to do the QOS policy globally which means it applies to the whole switch.

Servers would have to be enabled to be doing this as well right? Basically if they were and the switches weren't would that possible be the cause of packet drops? Yes I have ping drops which should not be applicable.

The other thing too, so the switches are in a VPC domain but I don't have any of the server ports configured with vpc/port-channel. No issues with any of them except for the two host IPs I speak of. Is it mandatory to configure member ports for anything or servers that are dual connected to a pair of switches in a VPC domain (e.g. port-channel 2, vpc x)? Perhaps this is part of the problem but why is it only happening to a certain group and not everything. I am thinking if I try and do it now while things are running on the servers I will break the connection?

Reza Sharifi · ‎06-21-2018

Servers would have to be enabled to be doing this as well right?

That is correct. It should be configured on the switches as well as the server(s). They basically have to match.

Yes, add peer-switch both chassis.

Is it mandatory to configure member ports for anything or servers that are dual connected to a pair of switches in a VPC domain (e.g. port-channel 2, vpc x)

If you are going to put the server links in Portchannel, you want them to be in a vPC as well (vpc x).

The other option would be to connect the switches to the server using the physical links only (no Portchannel no vPC). This especially works well with 3rd part server vendors like Dell and HP as sometimes they don't like Cisco's Portchannel. If your servers are Cisco, then you may want to use Po with vPC.

HTH

CiscoBrownBelt · ‎06-22-2018

That is correct. It should be configured on the switches as well as the server(s). They basically have to match.

Ok. I don't think they configured servers for Jumbo. I don't see any jumbo packets on any "show int".

Yes, add peer-switch both chassis.

Ok it is only peer-gateway right now, do you think that is causing part of or a problem?

Is it mandatory to configure member ports for anything or servers that are dual connected to a pair of switches in a VPC domain (e.g. port-channel 2, vpc x)

If you are going to put the server links in Portchannel, you want them to be in a vPC as well (vpc x).

The other option would be to connect the switches to the server using the physical links only (no Portchannel no vPC). This especially works well with 3rd part server vendors like Dell and HP as sometimes they don't like Cisco's Portchannel. If your servers are Cisco, then you may want to use Po with vPC.

Ok that is how it is now. Only 2 links are having problems but everything else is fine.

Reza Sharifi · ‎06-22-2018

Ok. I don't think they configured servers for Jumbo. I don't see any jumbo packets on any "show int"

If the servers are not configured with Jumbo then you don't need it in the switches.

Ok it is only peer-gateway right now, do you think that is causing part of or a problem?

No.

Ok that is how it is now. Only 2 links are having problems but everything else is fine.

How are they connected? Po, vPC, no PO, regular physical link?

HTH

CiscoBrownBelt · ‎06-22-2018

How are they connected? Po, vPC, no PO, regular physical link?

Just like this:

int eth1-2

switchport mode trunk

switchport trunk allowed vlan 2,3

switchport trunk native vlan 4

Native VLAN is some vlan systems team asked to put, but that is not same native vlan that may be used

on trunks for switches.

Trouble hosts on the 2 ports are NetApp.

CiscoBrownBelt · ‎06-21-2018

Also I have the peer gateway command configured under the VPC domain do you think I need peer-switch too?