I have a network infrastructure that has VERY low tolerance / requirement for packet loss. We have implemented nexus 5672UP (running version 7.3(5)N1(1)) in a single sided VPC configuration with 15 vpc member switches with port channels back up to the VPC peers. All fiber paths are redundant between the VPC peers and out to the VPC members. Very simple layer 2 only configuration. No layer 3 routing for the closed network. We do have critical multicast traffic in this environment. Again all layer 2 so just relying on the VPC Peer switches acting as the IGMP querier for the VLAN.
What makes this deployment very interesting is the extreme packet loss requirements during failure scenarios. I can't have more than 500ms of packet loss during any of the following network failures:
1) loss of a single link to vpc member switch,
2) loss of both keep-alive links (Po1),
3) loss of both vpc-peer-links (Po2),
4) loss of the Primary Operational VPC Peer
5) loss of the Primary Secondary VPC Peer
Should I anticipate being able to meet these requirements? I am currently not, yet I have followed all the best practices for the configurations.
Regarding failure scenario 1, I am experiencing brief packet loss 600ms when I disconnect a link in one of my vpc member port channels. I would have expected to not lose any packets as I have two uplinks in port-channel configuration and loosing one of the links should not cause any reconvergence in the solution, at least not for the port-channel or spanning-tree. Is VPC reconvergence operations causing the dropping of the packets? is that normal?
Regarding keep-alive failure testing (failure scenario 2).... no packet loss. Good.
Regarding VPC-peer link (failure scenario 3)... no packet loss. Good.
Regarding VPC peer switch failures (failure scenario 4/5)... we are getting couple seconds of packet loss when the switch comes back on line, not when the switch goes down. Any ideas on this? How can we prevent?
here are the configs.
VPC PEER Switch 1
PCNCS1# show run
!Command: show running-config
!Time: Tue Jul 23 18:05:22 2019
version 7.3(5)N1(1)
hostname PCNCS1
no feature telnet
feature scp-server
cfs eth distribute
feature interface-vlan
feature lacp
feature vpc
feature lldp
no ip source-route
no ip igmp snooping mrouter vpc-peer-link
vlan configuration 2
ip igmp snooping querier 192.168.1.1
vlan 1
vlan 2
name MCCWS_network
vlan 999
name shutdown
vlan 3000
name VPC_KEEPALIVE
spanning-tree loopguard default
spanning-tree vlan 2 priority 4096
spanning-tree vlan 3000 priority 12288
vrf context management
ip route 0.0.0.0/0 192.168.10.254 name MGMT
vpc domain 1
peer-switch
role priority 2001
peer-keepalive destination 172.16.3.2 source 172.16.3.1 vrf default
ip arp synchronize
interface Vlan1
no shutdown
no ip redirects
no ip arp gratuitous request
interface Vlan2
description VPC PEER TRUNK
no shutdown
ip address 192.168.1.1/24
interface Vlan3000
description VPC_KEEP_ALIVE
no shutdown
ip address 172.16.3.1/30
interface port-channel1
description TO_CORE_SWITCH_B
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 3000
spanning-tree port type network
interface port-channel2
description VPC_PEER_TRUNK
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
spanning-tree port type network
vpc peer-link
interface port-channel40
description to_Tenant0
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
vpc 40
interface port-channel41
description to_Primary
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
vpc 41
interface port-channel42
description to_Primary
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
vpc 42
interface port-channel50
description to_CV1
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
vpc 50
interface Ethernet1/14
switchport mode trunk
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
channel-group 2 mode active
interface Ethernet1/15
switchport mode trunk
switchport trunk native vlan 998
switchport trunk allowed vlan 3000
speed 1000
channel-group 1 mode active
interface Ethernet1/47
switchport mode trunk
switchport trunk native vlan 998
switchport trunk allowed vlan 3000
speed 1000
channel-group 1 mode active
interface Ethernet1/48
switchport mode trunk
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
channel-group 2 mode active
VPC PEER 2
PCNCS2# show run
!Command: show running-config
!Time: Tue Jul 23 18:07:23 2019
version 7.3(5)N1(1)
hostname PCNCS2
no feature telnet
feature scp-server
cfs eth distribute
feature interface-vlan
feature lacp
feature vpc
feature lldp
no ip source-route
no ip igmp snooping mrouter vpc-peer-link
vlan configuration 2
ip igmp snooping querier 192.168.1.1
vlan 1
vlan 2
name MCCC_network
vlan 998
name native
vlan 999
name shutdown
vlan 3000
name VPC_KEEPALIVE
spanning-tree loopguard default
spanning-tree vlan 3000 priority 12288
vrf context management
ip route 0.0.0.0/0 192.168.10.254 name MGMT
vpc domain 1
peer-switch
role priority 2002
peer-keepalive destination 172.16.3.1 source 172.16.3.2 vrf default
ip arp synchronize
interface Vlan1
no shutdown
no ip redirects
no ip arp gratuitous request
interface Vlan2
description VPC PEER TRUNK
no shutdown
ip address 192.168.1.2/24
interface Vlan3000
description VPC_KEEP_ALIVE
no shutdown
ip address 172.16.3.2/30
interface port-channel1
description TO_CORE_SWITCH_A
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 3000
spanning-tree port type network
interface port-channel2
description VPC_PEER_TRUNK
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
spanning-tree port type network
speed 1000
vpc peer-link
interface port-channel40
description to_Tenant0
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
vpc 40
interface port-channel41
description to_Primary
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
vpc 41
interface port-channel42
description to_Primary
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
vpc 42
interface port-channel50
description to_CV1
switchport mode trunk
no lacp suspend-individual
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
vpc 50
interface Ethernet1/14
switchport mode trunk
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
channel-group 2 mode active
interface Ethernet1/15
switchport mode trunk
switchport trunk native vlan 998
switchport trunk allowed vlan 3000
speed 1000
channel-group 1 mode active
interface Ethernet1/47
switchport mode trunk
switchport trunk native vlan 998
switchport trunk allowed vlan 3000
speed 1000
channel-group 1 mode active
interface Ethernet1/48
switchport mode trunk
switchport trunk native vlan 998
switchport trunk allowed vlan 2
speed 1000
channel-group 2 mode active
