10-02-2022 06:50 PM
HI experts
There is a problem of cedge vrrp,the configuration as below.
Master device:
Current configuration : 255 bytes
!
interface GigabitEthernet4.101
encapsulation dot1Q 101
vrf forwarding 1
ip address 172.16.101.252 255.255.255.0
ip mtu 1496
vrrp 1 address-family ipv4
timers advertise 100
priority 110
vrrpv2
address 172.16.101.254 primary
exit-vrrp
end
Backup Device:
Current configuration : 240 bytes
!
interface GigabitEthernet4.101
encapsulation dot1Q 101
vrf forwarding 1
ip address 172.16.101.253 255.255.255.0
ip mtu 1496
vrrp 1 address-family ipv4
timers advertise 100
vrrpv2
address 172.16.101.254 primary
exit-vrrp
end
I can see there are lots of logging about vrrp switchover;
*Oct 3 01:34:09.883: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:34:10.217: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:34:21.638: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:34:21.661: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:34:22.033: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:34:22.081: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:35:21.721: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:35:21.758: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:36:21.727: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:36:21.764: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
I also try to issue debug command to check it, but the error log that i feel strange. Could you help me to analysis this issue ? many thanks.
*Oct 3 01:37:21.404: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:37:21.406: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:37:21.415: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:37:21.443: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:37:21.449: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct 3 01:37:21.897: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:37:21.897: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:37:21.898: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:37:21.992: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:37:21.993: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:37:22.075: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:37:22.075: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:37:22.158: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:37:22.158: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:37:22.164: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:37:22.167: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct 3 01:38:21.215: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:38:21.216: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:38:21.224: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:38:21.248: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:38:21.251: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct 3 01:38:21.769: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:38:21.769: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:38:21.770: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:38:21.863: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:38:21.864: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:38:21.937: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:38:21.940: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct 3 01:39:21.582: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:39:21.583: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:39:21.591: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:39:21.614: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:39:21.618: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct 3 01:39:21.978: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:39:21.978: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:39:21.980: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:39:21.998: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:39:21.999: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct 3 01:40:21.434: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:40:21.436: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
*Oct 3 01:40:21.446: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 3 01:40:21.473: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct 3 01:40:21.480: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct 3 01:41:21.331: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4
*Oct 3 01:41:21.332: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED
Solved! Go to Solution.
04-04-2023 08:07 AM
Hi,
try to increase VRRP timers, it seems 100msec (0.1sec) is low in your environment. Check with 2000 value (2sec), then you may lower to 1000 (which is default).
Btw, for C8K-13 you don't have priority on group 3 (this is not an issue, but I see that other groups have higher priority, so -13 should be master explicitly)
07-03-2023 06:35 AM - edited 07-03-2023 06:36 AM
HI Kanan
Yes, Your answer could be right. when i changed the vrrp advertise timers to 1000ms or 2000ms , it seems steady.
Before change the vrrp timer.
SITE5-DEV2#sho run inter Gi4.1599
Building configuration...
Current configuration : 331 bytes
!
interface GigabitEthernet4.1599
encapsulation dot1Q 1599
vrf forwarding 599
ip address 192.168.141.253 255.255.255.0
no ip redirects
ip mtu 1496
ip nbar protocol-discovery
vrrp 59 address-family ipv4
timers advertise 100
vrrpv2
track omp shutdown
address 192.168.141.254 primary
exit-vrrp
arp timeout 1200
end
SITE5-DEV2#sho run inter Gi4.2599
Building configuration...
Current configuration : 331 bytes
!
interface GigabitEthernet4.2599
encapsulation dot1Q 2599
vrf forwarding 599
ip address 192.168.142.253 255.255.255.0
no ip redirects
ip mtu 1496
ip nbar protocol-discovery
vrrp 57 address-family ipv4
timers advertise 100
vrrpv2
track omp shutdown
address 192.168.142.254 primary
exit-vrrp
arp timeout 1200
end
SITE5-DEV2#show logg last 10 | i VRRP
Jul 3 13:03:19.496: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul 3 13:03:19.499: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul 3 13:03:20.441: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul 3 13:03:20.444: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state BACKUP -> MASTER
Jul 3 13:03:20.445: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul 3 13:03:20.445: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul 3 13:04:00.660: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state BACKUP -> MASTER
Jul 3 13:04:00.690: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul 3 13:04:00.696: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul 3 13:04:00.712: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
SITE5-DEV2#
Changed the configuration of vrrp timer
SITE5-DEV2#sho run inter Gi4.1599
Building configuration...
Current configuration : 308 bytes
!
interface GigabitEthernet4.1599
encapsulation dot1Q 1599
vrf forwarding 599
ip address 192.168.141.253 255.255.255.0
no ip redirects
ip mtu 1496
ip nbar protocol-discovery
vrrp 59 address-family ipv4 // there should be command ‘ timers advertise 1000’ ,so the 1000 ms should a default
vrrpv2
track omp shutdown
address 192.168.141.254 primary
exit-vrrp
arp timeout 1200
end
SITE5-DEV2#sho run inter Gi4.2599
Building configuration...
Current configuration : 332 bytes
!
interface GigabitEthernet4.2599
encapsulation dot1Q 2599
vrf forwarding 599
ip address 192.168.142.253 255.255.255.0
no ip redirects
ip mtu 1496
ip nbar protocol-discovery
vrrp 57 address-family ipv4
timers advertise 2000
vrrpv2
track omp shutdown
address 192.168.142.254 primary
exit-vrrp
arp timeout 1200
end
SITE5-DEV2#sho logg last 30 | i VRRP
Jul 3 13:07:20.585: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul 3 13:07:20.591: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul 3 13:08:19.335: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul 3 13:08:19.654: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul 3 13:08:20.471: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul 3 13:08:20.483: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
SITE5-DEV2#
But in the feature template configuration, 100ms seems to be the default parameter value.This is very strange!
According to the above information, I found the following bug-CSCwd95581 , which is the key to the problem!
VRRP timer in vManage UI - default 100ms
CSCwd95581
Symptom: cEdge VRRP default advertise time is 100 to 40950 milliseconds. Default time should be 1000ms(not 100 ms), need to fix it in UI.
Conditions: cedge devices
Workaround: -
Further Problem Description: cEdge VRRP default advertise time is 100 to 40950 milliseconds. Default time should be 1000ms(not 100 ms), need to fix it in UI.
https://bst.cisco.com/bugsearch/bug/CSCwd95581
Maybe this problem will be fixed in Release 20.11.1. Anyway, the problem is solved! Thanks for everyone's help
10-09-2022 06:20 AM - edited 10-09-2022 06:20 AM
Hi,
Verify the interface counters for the physical interface. Is there a physical issue between the cEdges and the switch?
Both cEdges are on the same switch?
10-10-2022 06:20 AM
GigabitEthernet4 is up, line protocol is up
Hardware is vNIC, address is 5000.0008.0003 (bia 5000.0008.0003)
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set
Keepalive set (10 sec)
Full Duplex, 1000Mbps, link type is auto, media type is Virtual
output flow-control is unsupported, input flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:03, output hang never
Last clearing of "show interface" counters never
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 10000 bits/sec, 22 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
519090 packets input, 31531760 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
11362 packets output, 872357 bytes, 0 underruns
Output 0 broadcasts (0 IP multicasts)
0 output errors, 0 collisions, 0 interface resets
448 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
10-11-2022 09:52 PM
Hi,
please share "show vrrp" output of both.
Best,
Kanan
10-12-2022 08:30 AM
HQ1#show vrrp
GigabitEthernet4.101 - Group 1 - Address-Family IPv4
State is MASTER
State duration 2 days 10 hours 9 mins
Virtual IP address is 172.16.x.254
Virtual MAC address is 0000.5E00.0101
Advertisement interval is 100 msec
Preemption enabled
Priority is 110
Master Router is 172.16.x.252 (local), priority is 110
Master Advertisement interval is 100 msec (expires in 0 msec)
Master Down interval is unknown
FLAGS: 1/1
HQ1#
*Oct 12 15:27:43.300: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 12 15:27:43.563: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
HQ2#sho vrrp
GigabitEthernet4.101 - Group 1 - Address-Family IPv4
State is MASTER
State duration 0.256 secs
Virtual IP address is 172.16.x.254
Virtual MAC address is 0000.5E00.0101
Advertisement interval is 100 msec
Preemption enabled
Priority is 100
Master Router is 172.16.x.253 (local), priority is 100
Master Advertisement interval is 100 msec (expires in 0 msec)
Master Down interval is 360 msec (expires in 0 msec)
FLAGS: 1/1
HQ2#
...
*Oct 12 15:29:40.492: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct 12 15:29:40.941: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
HQ2#sho vrrp
GigabitEthernet4.101 - Group 1 - Address-Family IPv4
State is BACKUP
State duration 10.280 secs
Virtual IP address is 172.16.x.254
Virtual MAC address is 0000.5E00.0101
Advertisement interval is 100 msec
Preemption enabled
Priority is 100
Master Router is 172.16.x.252, priority is 110
Master Advertisement interval is 100 msec (learned)
Master Down interval is 360 msec (expires in 238 msec)
FLAGS: 0/1
04-01-2023 12:43 PM
Did you get that issue resolved?
04-01-2023 05:10 PM
No,not yet
04-05-2023 02:14 PM
04-02-2023 01:17 PM
Hi,
there is one bug that maybe related. Which version do you use?
https://bst.cisco.com/bugsearch/bug/CSCvy56939
04-02-2023 08:13 PM
The version of edge is Release 17.6.3 before, i have updated the version to 17.7.2, it seems to don't flapping.
And " track omp shutdown" command added in configuration.
04-02-2023 10:13 PM
Hi,
it is better to configure track omp decrement, so if router loses OMP session (none of TLOC can connect vSmart), then it decrements its priority and VRRP failover happens. Outbound traffic (from service to DIA or Overlay) will be through new master.
To avoid asymmetry (because losing OMP does not 100% mean that you lost remote router connection), you have additional option to increase TLOC value on the master with the command " tloc-change increase-preference X" (where X is addition to tloc preference).
In this case, VRRP master will increase its TLOC's preference, so remote branch chooses master as primary point for destination. For this, you should have default or the same preference (actual interface preference) on both routers. Whichever master, it will automatically increase its TLOC preference values.
04-04-2023 07:47 AM
My previous description was wrong, it seems that VRRP flap still exists.I reconfigured the interface,The current configuration is like this:
C8K-13
C8K-13#sho run inter Gi4
Building configuration...
Current configuration : 338 bytes
!
interface GigabitEthernet4
vrf forwarding 599
ip address 192.168.112.252 255.255.255.0
no ip redirects
ip nbar protocol-discovery
load-interval 30
negotiation auto
vrrp 59 address-family ipv4
timers advertise 100
priority 110
vrrpv2
track omp shutdown
address 192.168.112.254 primary
exit-vrrp
arp timeout 1200
end
C8K-13#sho run inter Gi5.10
Building configuration...
Current configuration : 307 bytes
!
interface GigabitEthernet5.10
encapsulation dot1Q 10
vrf forwarding 10
ip address 192.168.103.252 255.255.255.0
no ip redirects
ip mtu 1496
ip nbar protocol-discovery
vrrp 8 address-family ipv4
timers advertise 100
vrrpv2
track omp shutdown
address 192.168.103.254 primary
exit-vrrp
end
C8K-13#sho run inter Gi5.599
Building configuration...
Current configuration : 326 bytes
!
interface GigabitEthernet5.599
encapsulation dot1Q 599
vrf forwarding 599
ip address 192.168.111.252 255.255.255.0
no ip redirects
ip mtu 1496
ip nbar protocol-discovery
vrrp 57 address-family ipv4
timers advertise 100
priority 110
vrrpv2
track omp shutdown
address 192.168.111.254 primary
exit-vrrp
end
Apr 4 14:44:43.041: %VRRP-6-STATE: GigabitEthernet5.10 IPv4 group 8 state BACKUP -> MASTER
Apr 4 14:44:43.073: %VRRP-6-STATE: GigabitEthernet5.10 IPv4 group 8 state MASTER -> BACKUP
Apr 4 14:45:03.589: %VRRP-6-STATE: GigabitEthernet5.10 IPv4 group 8 state BACKUP -> MASTER
Apr 4 14:45:03.958: %VRRP-6-STATE: GigabitEthernet5.10 IPv4 group 8 state MASTER -> BACKUP
Apr 4 14:46:03.604: %VRRP-6-STATE: GigabitEthernet5.10 IPv4 group 8 state BACKUP -> MASTER
Apr 4 14:46:03.988: %VRRP-6-STATE: GigabitEthernet5.10 IPv4 group 8 state MASTER -> BACKUP
C8K-14
C8K-14#sho run inter GI4
Building configuration...
Current configuration : 323 bytes
!
interface GigabitEthernet4
vrf forwarding 599
ip address 192.168.112.253 255.255.255.0
no ip redirects
ip nbar protocol-discovery
load-interval 30
negotiation auto
vrrp 59 address-family ipv4
timers advertise 100
vrrpv2
track omp shutdown
address 192.168.112.254 primary
exit-vrrp
arp timeout 1200
end
C8K-14#sho run inter GI5.10
Building configuration...
Current configuration : 322 bytes
!
interface GigabitEthernet5.10
encapsulation dot1Q 10
vrf forwarding 10
ip address 192.168.103.253 255.255.255.0
no ip redirects
ip mtu 1496
ip nbar protocol-discovery
vrrp 8 address-family ipv4
timers advertise 100
priority 110
vrrpv2
track omp shutdown
address 192.168.103.254 primary
exit-vrrp
end
C8K-14#sho run inter GI5.599
Building configuration...
Current configuration : 311 bytes
!
interface GigabitEthernet5.599
encapsulation dot1Q 599
vrf forwarding 599
ip address 192.168.111.253 255.255.255.0
no ip redirects
ip mtu 1496
ip nbar protocol-discovery
vrrp 57 address-family ipv4
timers advertise 100
vrrpv2
track omp shutdown
address 192.168.111.254 primary
exit-vrrp
end
C8K-14#
Apr 4 14:46:42.822: %VRRP-6-STATE: GigabitEthernet4 IPv4 group 59 state BACKUP -> MASTER
Apr 4 14:46:43.010: %VRRP-6-STATE: GigabitEthernet4 IPv4 group 59 state MASTER -> BACKUP
Apr 4 14:46:43.012: %VRRP-6-STATE: GigabitEthernet5.599 IPv4 group 57 state MASTER -> BACKUP
Apr 4 14:47:03.994: %VRRP-6-STATE: GigabitEthernet5.599 IPv4 group 57 state BACKUP -> MASTER
Apr 4 14:47:04.014: %VRRP-6-STATE: GigabitEthernet4 IPv4 group 59 state BACKUP -> MASTER
Apr 4 14:47:04.021: %VRRP-6-STATE: GigabitEthernet5.599 IPv4 group 57 state MASTER -> BACKUP
Apr 4 14:47:04.022: %VRRP-6-STATE: GigabitEthernet4 IPv4 group 59 state MASTER -> BACKUP
04-05-2023 02:14 PM
07-03-2023 03:16 AM
what config changes you made , i have same issue , on Secondary router vrrp groups keep on flapping where as on primary there is not even a single log for vrrp.
07-03-2023 03:18 AM
Make new post'
In this way all can see new post and reply to you
Thanks
MHM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide