cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2681
Views
35
Helpful
16
Replies

VM host issues connected to pair of VPC switches using HSRP

CiscoBrownBelt
Level 6
Level 6

So I have a couple hosts  that continuous pings (testing do to deployment issues) fail mostly from doing pings from servers.

Just have a pair of 3548 Nexus in VPC domain (configured to be active active) and routing for the subnet hosts are in is done via SVI-HSRP. Here are pertaining configs. Any help guys?

 

Same on both switches except:

vpc domain 1                                                                    
  role priority 100 (same for both switches)                                                            
  peer-keepalive destination 172.168.1.1 source 172.168.1.2 vrf vpc-keepalive   
  dual-active exclude interface-vlan 5                                     
  peer-gateway                                                                  
  auto-recovery                                                                 
                                                                                
interface port-channel1                                                        
  vpc peer-link         

-----------

SVI configs

Same on both switches except where noted:

interface Vlan2                                                               
  description ServerHost                                                      
  no shutdown                                                                   
  no ip redirects                                                               
  ip address 172.168.2.2/24                                                   
  hsrp 2                                                                      
    preempt                                                                     
    priority 105 (priority default on Switch 2)                                                                
    ip 172.16.2.1
 
---------------
Same on both switches except
interface Ethernet1/45 and 46                                                           
  description vPC Peer-Link                                                     
  switchport mode trunk                                                         
  channel-group 1 mode active                                                  
  no shutdown   
-------------------------
 
Interfaces for server ports:
 
interface Ethernet1/1 and 2                                                          
  speed 1000                                                                    
  description ServerHost                                                
  switchport mode trunk                                                         
  switchport trunk allowed vlan 2
 switchport trunk native vlan 12                                             
  spanning-tree port type edge trunk                                            
  no shutdown 

------------------------------------------

 

SHOW Commands to help T-shoot:

 

show mac address-table (trouble host mac)

Sw_A# sh mac address-table | inc 03a0.9756.056d                        
* 214      03a0.9756.056d    dynamic   0          F    F  Po1   
 
 
Sw_B# sh mac address-table | inc 03a0.9756.056d                          
* 214      03a0.9756.056d   dynamic   0          F    F  Po1
 
Sw_A# sh port-channel traffic                                        
ChanId      Port Rx-Ucst Tx-Ucst Rx-Mcst Tx-Mcst Rx-Bcst Tx-Bcst                
------ --------- ------- ------- ------- ------- ------- -------                
    1   Eth1/35   0.07%   0.80%   2.25%   3.75%   0.12%   0.52%                
    1   Eth1/36  99.92%  99.19%  97.74%  96.24%  99.87%  99.47%      
 
Sw_B# sh port-channel traffic                                        
ChanId      Port Rx-Ucst Tx-Ucst Rx-Mcst Tx-Mcst Rx-Bcst Tx-Bcst                
------ --------- ------- ------- ------- ------- ------- -------                
    10   Eth1/35  55.87%  19.73%  16.15%  47.77%  98.40%  99.75%                
    10   Eth1/36  44.12%  80.26%  83.84%  52.22%   1.59%   0.24% 
16 Replies 16

balaji.bandi
Hall of Fame
Hall of Fame

Firstly, i would like to see full config of both the switches. some picture how your peer links connected back to back or any mediated switch ?

 

I do not see you allowing VLAN 2 in the vpc peer link.

what kind of spanning tree you are using (config should be network for the peer-link)

 

show output :

 

show vpc

show spanning tree

show run inter por 1

 

BB

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

 

Thanks bro!

Sorry  I can't really just past all config on here but I have provided all that I think is applicable as there are no other real configs on them. I have attached a drawing it is very basic topology - 2 switches 1 server.

The peer link is trunk allowing all vlans (Po1). Below are output you asked for hope it helps. I am thinking problem is on server side.

 

 

vPC domain id                     : 1                                           Peer status                       : peer adjacency formed ok                    vPC keep-alive status             : peer is alive                               Configuration consistency status  : success                                     Per-vlan consistency status       : success                                     Type-2 consistency status         : success                                     vPC role                          : primary                                     Number of vPCs configured         : 0                                           Peer Gateway                      : Enabled                                     Peer gateway excluded VLANs       : -                                           Dual-active excluded VLANs        : 5                                         Graceful Consistency Check        : Enabled                                     Operational Layer3 Peer-router    : Disabled                                    Auto-recovery status              : Enabled (timeout = 240 seconds)                                                                                             vPC Peer-link status                                                            ---------------------------------------------------------------------           id   Port   Status Active vlans                                                 --   ----   ------ --------------------------------------------------           1    Po1   up     1,2,3,5

----------------------------------

 

Here is spanning tree.

Sw1# sh spanning-tree                                               

                                                                                

VLAN0001                                                                        

  Spanning tree enabled protocol rstp                                           

  Root ID    Priority    32769                                                  

             Address     00d7.8faa.5cfc                                         

             This bridge is the root                                            

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

  Bridge ID  Priority    32769  (priority 32768 sys-id-ext 1)                   

             Address     00d7.8faa.5cfc                                         

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

Interface        Role Sts Cost      Prio.Nbr Type                               

---------------- ---- --- --------- -------- --------------------------------   

Po1           Desg FWD 2         128.4105 (vPC peer-link) Network P2p        

 

VLAN002                                                                      

  Spanning tree enabled protocol rstp                                           

  Root ID    Priority    32982                                                  

             Address     00d7.8faa.5cfc                                         

             This bridge is the root                                            

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

  Bridge ID  Priority    32982  (priority 32768 sys-id-ext 214)                 

             Address     00d7.8faa.5cfc                                         

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

Interface        Role Sts Cost      Prio.Nbr Type                               

---------------- ---- --- --------- -------- --------------------------------   

Po1             Desg FWD 2         128.4105 (vPC peer-link) Network P2p        

Eth1/1           Desg FWD 4         128.135  Edge P2p                           

Eth1/2           Desg FWD 4         128.136  Edge P2p                           

 

VLAN003                                                                        

  Spanning tree enabled protocol rstp                                           

  Root ID    Priority    32783                                                  

             Address     00d7.8faa.5cfc                                         

             This bridge is the root                                            

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

  Bridge ID  Priority    32783  (priority 32768 sys-id-ext 15)                  

             Address     00d7.8faa.5cfc                                         

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

Interface        Role Sts Cost      Prio.Nbr Type                               

---------------- ---- --- --------- -------- --------------------------------   

Po1             Desg FWD 2         128.4105 (vPC peer-link) Network P2p        

 

VLAN005                                                                       

  Spanning tree enabled protocol rstp                                           

  Root ID    Priority    32867                                                  

             Address     00d7.8faa.5cfc                                         

             This bridge is the root                                            

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

  Bridge ID  Priority    32867  (priority 32768 sys-id-ext 99)                  

             Address     00d7.8faa.5cfc                                         

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

Interface        Role Sts Cost      Prio.Nbr Type                               

---------------- ---- --- --------- -------- --------------------------------   

Po1             Desg FWD 2         128.4105 (vPC peer-link) Network P2p        

          

 

 

 

Sw2# sh spanning-tree                                               

 

VLAN0001                                                                        

  Spanning tree enabled protocol rstp                                           

  Root ID    Priority    32769                                                  

             Address     00d7.8faa.5cfc                                         

             Cost        2                                                      

             Port        4105 (port-channel1)                                  

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

  Bridge ID  Priority    32769  (priority 32768 sys-id-ext 1)                   

             Address     00d7.8faa.60bc                                         

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

Interface        Role Sts Cost      Prio.Nbr Type                               

---------------- ---- --- --------- -------- --------------------------------   

Po1             Root FWD 2         128.4105 (vPC peer-link) Network P2p        

 

 

 

 

VLAN02                                                                        

  Spanning tree enabled protocol rstp                                           

  Root ID    Priority    32982                                                  

             Address     00d7.8faa.5cfc                                         

             Cost        2                                                      

             Port        4105 (port-channel1)                                  

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

  Bridge ID  Priority    32982  (priority 32768 sys-id-ext 214)                 

             Address     00d7.8faa.60bc                                         

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

Interface        Role Sts Cost      Prio.Nbr Type                               

---------------- ---- --- --------- -------- --------------------------------   

Po1            Root FWD 2         128.4105 (vPC peer-link) Network P2p        

Eth1/1           Desg FWD 4         128.135  Edge P2p                           

Eth1/2           Desg FWD 4         128.136  Edge P2p         

 

 

VLAN003                                                                        

  Spanning tree enabled protocol rstp                                           

  Root ID    Priority    32783                                                  

             Address     00d7.8faa.5cfc                                         

             Cost        2                                                      

             Port        4105 (port-channel1)                                  

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

  Bridge ID  Priority    32783  (priority 32768 sys-id-ext 15)                  

             Address     00d7.8faa.60bc                                         

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

Interface        Role Sts Cost      Prio.Nbr Type                               

---------------- ---- --- --------- -------- --------------------------------   

Po1             Root FWD 2         128.4105 (vPC peer-link) Network P2p        

 

 

VLAN005                                                                        

  Spanning tree enabled protocol rstp                                           

  Root ID    Priority    32786                                                  

             Address     00d7.8faa.5cfc                                         

             Cost        2                                                      

             Port        4105 (port-channel1)                                  

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

  Bridge ID  Priority    32786  (priority 32768 sys-id-ext 18)                  

             Address     00d7.8faa.60bc                                         

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec           

                                                                                

Interface        Role Sts Cost      Prio.Nbr Type                               

---------------- ---- --- --------- -------- --------------------------------   

Po1             Root FWD 2         128.4105 (vPC peer-link) Network P2p        

 

interface port-channel10    (Same on both switches)                                                    

  speed 10000                                                                   

  description VPC Domain1_PeerLink                                             

  switchport mode trunk                                                         

  spanning-tree port type network                                               

  vpc peer-link                                                                 

                   

Reza Sharifi
Hall of Fame
Hall of Fame
Below is not correct. HSRP virtual IP and the physical address need to be in the same subnet
interface Vlan2                                                               
  description ServerHost                                                      
  no shutdown                                                                   
  no ip redirects                                                               
  ip address 172.168.2.2/24                                                   
  hsrp 2                                                                      
    preempt                                                                     
    priority 105 (priority default on Switch 2)                                                                
    ip 172.16.2.1

Sorry that was a type. They are both 172.16 and it pings.

One thing I noticed there are some jumpo frames when show int 1-2 and on the peer-link interfaces. Should I configure system jumbo as below? There are hosts on 10gig ports however the trouble hosts are only on gig ports.

 

switch(config)#policy-map type network-qos jumboframes
  class type network-qos class-default
    mtu 9216

 

system qos
  service-policy type network-qos jumboframes

 

switch(config)#system jumbomtu 9216

!--- Set the MTU specification for an interface.
switch(config)#interface ethernet x/x

!--- By default, Cisco NX-OS configures Layer 3 parameters. In order to configure Layer 2 parameters, use this command.
switch(config-if)#switchport

switch(config-if)#mtu 9216
switch(config-if)#exit

Although it makes sense to have the same exact jumbo frame config on both switch, this should not have anything to do with Ping failure. 

So, from the server, can you ping both physical IPs and also the virtual? 

If that is good, can you ping the other way, meaning from the switch, can you ping the IP on the server?

HTH

So from the server it can ping both actual IPs no problem, but when pinging the virtual .1 ip that is where ping fails here and there.

Yes from switch I can ping the two server IPs no problem.

 

In regards to Jumbo, that was kinda just off subject but also I wondered about. So if I got 10gig ports connected to hosts, don't I need to make sure system mtu 9216 is configured globablly and on the peer-links between both VPC switches? Should QOS for jumbo really be configured as well?

but when pinging the virtual .1 ip that is where ping fails here and there

Ok, is HSRP flapping? You can find that by using "sh hsrp vlan 2". Can you post the output here?

In regards to Jumbo, that was kinda just off subject but also I wondered about. So if I got 10gig ports connected to hosts, don't I need to make sure system mtu 9216 is configured globablly and on the peer-links between both VPC switches? Should QOS for jumbo really be configured as well?

Yes, you need to enable it globally first and add it to each vlan with QOS policy. I am not sure about the 3000 platform but in some Nexus models (5ks and 6ks), you also have to do the QOS policy globally which means it applies to the whole switch.

HTH

 

Ok, is HSRP flapping? You can find that by using "sh hsrp vlan 2". Can you post the output here? 

I don't think so but I will have to check when I get the chance. 

Yes, you need to enable it globally first and add it to each vlan with QOS policy. I am not sure about the 3000 platform but in some Nexus models (5ks and 6ks), you also have to do the QOS policy globally which means it applies to the whole switch.

Servers would have to be enabled to be doing this as well right? Basically if they were and the switches weren't would that possible be the cause of packet drops? Yes I have ping drops which should not be applicable.

The other thing too,  so the switches are in a VPC domain but I don't have any of the server ports configured with vpc/port-channel. No issues with any of them except for the two host IPs I speak of. Is it mandatory to configure member ports for anything or servers that are dual connected to a pair of switches in a VPC domain (e.g. port-channel 2, vpc x)? Perhaps this is part of the problem but why is it only happening to a certain group and not everything.  I am thinking if I try and do it now while things are running on the servers I will break the connection?

 

Servers would have to be enabled to be doing this as well right?

That is correct. It should be configured on the switches as well as the server(s). They basically have to match.

Yes, add peer-switch both chassis.

Is it mandatory to configure member ports for anything or servers that are dual connected to a pair of switches in a VPC domain (e.g. port-channel 2, vpc x)

If you are going to put the server links in Portchannel, you want them to be in a vPC as well (vpc x).

The other option would be to connect the switches to the server using the physical links only (no Portchannel no vPC). This especially works well with 3rd part server vendors like Dell and HP as sometimes they don't like Cisco's Portchannel. If your servers are Cisco, then you may want to use Po with vPC.

HTH

 

That is correct. It should be configured on the switches as well as the server(s). They basically have to match.

Ok. I don't think they configured servers for Jumbo. I don't see any jumbo packets on any "show int".

Yes, add peer-switch both chassis.

Ok it is only peer-gateway right now, do you think that is causing part of or a problem?

Is it mandatory to configure member ports for anything or servers that are dual connected to a pair of switches in a VPC domain (e.g. port-channel 2, vpc x)

If you are going to put the server links in Portchannel, you want them to be in a vPC as well (vpc x).

The other option would be to connect the switches to the server using the physical links only (no Portchannel no vPC). This especially works well with 3rd part server vendors like Dell and HP as sometimes they don't like Cisco's Portchannel. If your servers are Cisco, then you may want to use Po with vPC.

Ok that is how it is now.  Only 2 links are having problems but everything else is fine.

Ok. I don't think they configured servers for Jumbo. I don't see any jumbo packets on any "show int"

If the servers are not configured with Jumbo then you don't need it in the switches.

Ok it is only peer-gateway right now, do you think that is causing part of or a problem?

No.

Ok that is how it is now.  Only 2 links are having problems but everything else is fine.

How are they connected? Po, vPC, no PO, regular physical link?

HTH

 

 

How are they connected? Po, vPC, no PO, regular physical link?

 

Just like this:

int eth1-2

switchport mode trunk

switchport trunk allowed vlan 2,3

switchport trunk native vlan 4

 

Native VLAN is some vlan systems team asked to put, but that is not same native vlan that may be used

on trunks for switches.

Trouble hosts on the 2 ports are NetApp.

Also I have the peer gateway command configured under the VPC domain do you think I need peer-switch too?