Buy or Renew
Buy or Renew
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
561
Views
0
Helpful
4
Replies

Nexus 5672 core with FEXs. Server in vpc configuration looses default gateway connectivity

Robert Rauch
Level 1
Level 1

‎03-14-2018 06:26 AM - edited ‎03-08-2019 02:15 PM

Hello,
I have a very weird situation that I dont understand and need help with.

 

We have a problem with a server that looses or cant reach its default gateway for quiet a few seconds and then it can reach the default gateway again. This can happen up to 30 times a day.

 

The default gateway happens to be a interface on the nexus core router.

 

The server is configured as a port-channel with vpc on the two cores via fabric extenders. The server guys say its a network problem but I can find any thing.

 

Its a first of a kind on our new environment using fabric-extenders with the server using Vpc over two core nexus. We also have a server with only 1 uplink trunk and this works fine, no default gateway problem.

 

WSPCR100 - part of core config 


fex 100
pinning max-links 1
description "***FEX100***"
fex 101
pinning max-links 1
description "***FEX101***"
slot 100
provision model N2K-C2348TQ
slot 101
provision model N2K-C2348TQ
.
.
lacp system-priority 61440
service dhcp
ip dhcp relay
vrf context management
vpc domain 10
peer-switch
role priority 1
system-priority 61440
peer-keepalive destination 10.0.0.2 source 10.0.0.1
delay restore 180
ip arp synchronize


interface Vlan1
description VLAN1
no shutdown
ip flow monitor netflow_monitor input sampler test
no ip redirects
ip address 172.18.60.2/16
ip directed-broadcast 50
ip router ospf 65001 area 0.0.0.0
ip pim sparse-mode
hsrp 1
preempt
priority 140
timers 1 3
ip 172.18.60.1
ip dhcp relay address 172.18.64.51
ip dhcp relay address 172.18.64.52
ip dhcp relay address 172.18.64.83
ip dhcp relay address 172.18.64.12

 

 


interface port-channel100
description ***FEX100***
switchport mode fex-fabric
fex associate 100

interface port-channel101
description ***FEX101***
switchport mode fex-fabric
fex associate 101

 

interface port-channel130
description wsps560 DB Cluster
switchport mode trunk
no lacp suspend-individual
switchport trunk allowed vlan 1,90
spanning-tree port type edge trunk
vpc 130

 

interface port-channel4096
description ***VPC_PEER-LINK***
switchport mode trunk
no lacp suspend-individual
spanning-tree port type network
vpc peer-link

 

interface Ethernet1/46
description ***L3 LINK WSPCR100<->WSPCR101***
no switchport
ip address 172.24.9.61/30
ip ospf authentication message-digest
ip ospf message-digest-key 10 md5 7 1041071F0E1C425C42
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf 65001 area 0.0.0.0
ip pim sparse-mode

i

interface Ethernet2/1
description ***VPC_PEER-LINK***
switchport mode trunk
logging event port link-status
channel-group 4096 mode active

 

interface Ethernet2/2
description ***VPC_PEER-LINK***
switchport mode trunk
logging event port link-status
channel-group 4096 mode active

 

interface Ethernet2/5
description ***FEX100***
switchport mode fex-fabric
fex associate 100
logging event port link-status
logging event port trunk-status
channel-group 100

 

interface Ethernet2/6
description ***FEX101***
switchport mode fex-fabric
fex associate 101
logging event port link-status
logging event port trunk-status
channel-group 101

 

interface mgmt0
description ***VPC_KEEPALIVE***
vrf member management
ip address 10.0.0.1/30

 

interface Ethernet101/1/43
description wsps561-mp1
switchport mode trunk
switchport trunk allowed vlan 1,90
channel-group 130 mode passive

 

interface Ethernet101/1/44

interface Ethernet101/1/45

i

 

WSPCR101 - part of core config

 

version 7.1(4)N1(1)
hostname WSPCR101


fex 100
pinning max-links 1
description "***FEX100***"
fex 101
pinning max-links 1
description "***FEX101***"
slot 100
provision model N2K-C2348TQ
slot 101
provision model N2K-C2348TQ

lacp system-priority 61440
service dhcp
ip dhcp relay
vrf context management
vpc domain 10
peer-switch
role priority 2
system-priority 61440
peer-keepalive destination 10.0.0.1 source 10.0.0.2
delay restore 180
ip arp synchronize


interface Vlan1
description VLAN1
no shutdown
ip flow monitor netflow_monitor input sampler test
no ip redirects
ip address 172.18.255.253/16
ip directed-broadcast 50
ip router ospf 65001 area 0.0.0.0
ip pim sparse-mode
hsrp 1
preempt
priority 150
timers 1 3
ip 172.18.60.1
ip dhcp relay address 172.18.64.51
ip dhcp relay address 172.18.64.52
ip dhcp relay address 172.18.64.83
ip dhcp relay address 172.18.64.12

 

 

interface port-channel100
description ***FEX100***
switchport mode fex-fabric
fex associate 100

interface port-channel101
description ***FEX101***
switchport mode fex-fabric
fex associate 101

interface port-channel130
description wsps560 DB Cluster
switchport mode trunk
no lacp suspend-individual
switchport trunk allowed vlan 1,90
spanning-tree port type edge trunk
vpc 130

 

interface port-channel4096
description ***VPC_PEER-LINK***
switchport mode trunk
no lacp suspend-individual
spanning-tree port type network
vpc peer-link


interface Ethernet1/46
description ***L3 LINK WSPCR100<->WSPCR101***
no switchport
ip address 172.24.9.62/30
ip ospf authentication message-digest
ip ospf message-digest-key 10 md5 7 1041071F0E1C425C42
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf 65001 area 0.0.0.0
ip pim sparse-mode

 

interface Ethernet2/1
description ***VPC_PEER-LINK***
switchport mode trunk
logging event port link-status
channel-group 4096 mode active

interface Ethernet2/2
description ***VPC_PEER-LINK***
switchport mode trunk
logging event port link-status
channel-group 4096 mode active

interface Ethernet2/3

interface Ethernet2/4

interface Ethernet2/5
description ***FEX100***
switchport mode fex-fabric
fex associate 100
logging event port link-status
logging event port trunk-status
channel-group 100

interface Ethernet2/6
description ***FEX101***
switchport mode fex-fabric
fex associate 101
logging event port link-status
logging event port trunk-status
channel-group 101

 

interface Ethernet101/1/42

interface Ethernet101/1/43
description wsps561-mp2
switchport mode trunk
switchport trunk allowed vlan 1,90
channel-group 130 mode passive

interface Ethernet101/1/44

 

 

The fex both look fine and if I look at the vpc on the core it looks fine.

vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
130 Po130 up success success 1,90
WSPCR100#

 

Any idea what the default gateway cant be reached from the server, can this have something to do with the vpc being on two different nexus in a VPC domain.

 

thanks for any tips that might help me.

br

Robert

 

Labels:
Preview file
136 KB
0 Helpful
4 Replies 4

Reza Sharifi
Hall of Fame
Hall of Fame

‎03-14-2018 06:33 AM - edited ‎03-14-2018 06:35 AM

Hi, 

Can you post the output of "sh vpc" from both switches?

Also, for testing, you can simply remove the Portchannel and just have the uplinks from the server as trunk ports and test again. Some vendors servers don't do well with Cisco Portchannel and have the servers guys configured the server with NIC teaming (HA). The other option you can try is to use mode "on" (no LACP) and see if it makes a difference.

HTH

0 Helpful

Robert Rauch
Level 1
Level 1
In response to Reza Sharifi

‎03-14-2018 06:38 AM

WSPCR100# show vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 10
Peer status                       : peer adjacency formed ok
vPC keep-alive status             : peer is alive
Configuration consistency status  : success
Per-vlan consistency status       : success
Type-2 consistency status         : success
vPC role                          : primary
Number of vPCs configured         : 9
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled (timeout = 240 seconds)

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans
--   ----   ------ --------------------------------------------------
1    Po4096 up     1,10-11,15,22-23,25,34-36,40,57,60-62,79,81-83,85-
                   90,93,97,149-154,160,162,176-178,191,195,203,205-2
                   06,210,251-254,260-266,300-308,363-364,462,500-502
                   ,601-602,666,700-707,800-805,901-911,913,915,997

vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
5      Po5         up     success     success                    1,10-11,15,
                                                                 22-23,25,34
                                                                 -36,40,57,6
                                                                 0-62,79,81-
                                                                 83,85-90....
8      Po8         down*  Not         Consistency Check Not      -
                          Applicable  Performed
40     Po40        down*  Not         Consistency Check Not      -
                          Applicable  Performed
60     Po60        up     success     success                    1,10-11,15,
                                                                 22-23,25,34
                                                                 -36,40,57,6
                                                                 0-62,79,81-
                                                                 83,85-90....
130    Po130       up     success     success                    1,90
200    Po200       up     success     success                    1,10-11,15,
                                                                 23,35,61,15
                                                                 1-154,205,2
                                                                 51-254,260,
                                                                 263,266,....
201    Po201       down*  Not         Consistency Check Not      -
                          Applicable  Performed
202    Po202       up     success     success                    10-11,15,22
                                                                 -23,25,34-3
                                                                 6,40,57,60-
                                                                 62,79,81-83
                                                                 ,85-90,9....
4095   Po4095      up     success     success                    1,10-11,15,
                                                                 22-23,25,34
                                                                 -36,40,57,6
                                                                 0-62,79,81-
                                                                 83,85-90....
WSPCR100#

 

WSPCR101# show vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 10
Peer status                       : peer adjacency formed ok
vPC keep-alive status             : peer is alive
Configuration consistency status  : success
Per-vlan consistency status       : success
Type-2 consistency status         : success
vPC role                          : secondary
Number of vPCs configured         : 9
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled (timeout = 240 seconds)

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans
--   ----   ------ --------------------------------------------------
1    Po4096 up     1,10-11,15,22-23,25,34-36,40,57,60-62,79,81-83,85-
                   90,93,97,149-154,160,162,176-178,191,195,203,205-2
                   06,210,251-254,260-266,300-308,363-364,462,500-502
                   ,601-602,666,700-707,800-805,901-911,913,915,997

vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
5      Po5         up     success     success                    1,10-11,15,
                                                                 22-23,25,34
                                                                 -36,40,57,6
                                                                 0-62,79,81-
                                                                 83,85-90....
8      Po8         down*  Not         Consistency Check Not      -
                          Applicable  Performed
40     Po40        down*  Not         Consistency Check Not      -
                          Applicable  Performed
60     Po60        up     success     success                    1,10-11,15,
                                                                 22-23,25,34
                                                                 -36,40,57,6
                                                                 0-62,79,81-
                                                                 83,85-90....
130    Po130       up     success     success                    1,90
200    Po200       down*  Not         Consistency Check Not      -
                          Applicable  Performed
201    Po201       up     success     success                    1,10-11,15,
                                                                 23,35,61,15
                                                                 1-154,205,2
                                                                 51-254,263,
                                                                 266,307-....
202    Po202       up     success     success                    10-11,15,22
                                                                 -23,25,34-3
                                                                 6,40,57,60-
                                                                 62,79,81-83
                                                                 ,85-90,9....
4095   Po4095      up     success     success                    1,10-11,15,
                                                                 22-23,25,34
                                                                 -36,40,57,6
                                                                 0-62,79,81-
                                                                 83,85-90....
WSPCR101#

 

0 Helpful

Reza Sharifi
Hall of Fame
Hall of Fame
In response to Robert Rauch

‎03-14-2018 06:47 AM

vPC looks good. Can you try one of the options?

Also, do you have the same issue if you bring down one of the physical links and have the server communicate using one uplink only?

 

0 Helpful

Robert Rauch
Level 1
Level 1
In response to Reza Sharifi

‎03-14-2018 06:55 AM

Yes, that was my plan to take down one of the links. This I have to coordinate with the server colleagues. It is a pity that there is already quiet a lot of data bases running on this server and is already live, even with the GW connectivity problem.
0 Helpful
Customers Also Viewed These Support Documents
Review Cisco Networking for a $25 gift card