Solved: Re: SDWAN cEdge VRRP flapping - Page 2

Rps-Cheers · ‎10-02-2022

HI experts

There is a problem of cedge vrrp,the configuration as below.

Master device:
Current configuration : 255 bytes
!
interface GigabitEthernet4.101
 encapsulation dot1Q 101
 vrf forwarding 1
 ip address 172.16.101.252 255.255.255.0
 ip mtu 1496
 vrrp 1 address-family ipv4
  timers advertise 100
  priority 110
  vrrpv2
  address 172.16.101.254 primary
  exit-vrrp
end

Backup Device:
Current configuration : 240 bytes
!
interface GigabitEthernet4.101
 encapsulation dot1Q 101
 vrf forwarding 1
 ip address 172.16.101.253 255.255.255.0
 ip mtu 1496
 vrrp 1 address-family ipv4
  timers advertise 100
  vrrpv2
  address 172.16.101.254 primary
  exit-vrrp
end

I can see there are lots of logging about vrrp switchover;

*Oct  3 01:34:09.883: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:34:10.217: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:34:21.638: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:34:21.661: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:34:22.033: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:34:22.081: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:35:21.721: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:35:21.758: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:36:21.727: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:36:21.764: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP

I also try to issue debug command to check it, but the error log that i feel strange. Could you help me to analysis this issue ? many thanks.

*Oct  3 01:37:21.404: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:37:21.406: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:37:21.415: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:37:21.443: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:37:21.449: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct  3 01:37:21.897: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:37:21.897: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:37:21.898: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:37:21.992: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:37:21.993: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:37:22.075: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:37:22.075: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:37:22.158: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:37:22.158: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:37:22.164: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:37:22.167: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
 
*Oct  3 01:38:21.215: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:38:21.216: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:38:21.224: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:38:21.248: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:38:21.251: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct  3 01:38:21.769: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:38:21.769: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:38:21.770: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:38:21.863: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:38:21.864: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:38:21.937: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:38:21.940: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct  3 01:39:21.582: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:39:21.583: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:39:21.591: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:39:21.614: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:39:21.618: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct  3 01:39:21.978: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:39:21.978: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:39:21.980: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:39:21.998: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:39:21.999: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
*Oct  3 01:40:21.434: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:40:21.436: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

*Oct  3 01:40:21.446: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state BACKUP -> MASTER
*Oct  3 01:40:21.473: %VRRP-6-STATE: GigabitEthernet4.101 IPv4 group 1 state MASTER -> BACKUP
*Oct  3 01:40:21.480: VRRPv4 GigabitEthernet4.101 [1] Notify active address failed: Attibute value already set
 
*Oct  3 01:41:21.331: VRRPv4 GigabitEthernet4.101 [1] vrrpv3 chksum 58E4

*Oct  3 01:41:21.332: VRRPv4 GigabitEthernet4.101 [1] vrrpv2 chksum 68ED

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Kanan Huseynli · ‎04-04-2023

Hi,

try to increase VRRP timers, it seems 100msec (0.1sec) is low in your environment. Check with 2000 value (2sec), then you may lower to 1000 (which is default).

Btw, for C8K-13 you don't have priority on group 3 (this is not an issue, but I see that other groups have higher priority, so -13 should be master explicitly)

HTH,
Please rate and mark as an accepted solution if you have found any of the information provided useful.

Rps-Cheers · ‎07-03-2023

This problem is still bothering us. Whether it is C8KV or C8200, 8300, there is still such a problem. In the backup role device, there will be Master/Backup Flap, like this:

Jul  3 08:00:21.599: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul  3 08:00:21.601: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul  3 08:01:00.510: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul  3 08:01:00.559: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state BACKUP -> MASTER
Jul  3 08:01:00.742: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul  3 08:01:00.753: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul  3 08:01:13.662: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state BACKUP -> MASTER
Jul  3 08:01:13.703: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul  3 08:01:13.727: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul  3 08:01:13.728: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul  3 08:01:19.366: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul  3 08:01:19.402: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state BACKUP -> MASTER
Jul  3 08:01:19.618: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul  3 08:01:19.619: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul  3 08:01:21.247: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state BACKUP -> MASTER
Jul  3 08:01:21.260: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul  3 08:01:21.309: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP

There are some configuration under the vrrp interfaces.

SITE5-DEV2#sho run inter Gi4.1599
Building configuration...

Current configuration : 331 bytes
!
interface GigabitEthernet4.1599
 encapsulation dot1Q 1599
 vrf forwarding 599
 ip address 192.168.141.253 255.255.255.0
 no ip redirects
 ip mtu 1496
 ip nbar protocol-discovery
 vrrp 59 address-family ipv4
  timers advertise 100
  vrrpv2
  track omp shutdown
  address 192.168.141.254 primary
  exit-vrrp
 arp timeout 1200
end

SITE5-DEV2#sho run inter Gi4.2599
Building configuration...

Current configuration : 331 bytes
!
interface GigabitEthernet4.2599
 encapsulation dot1Q 2599
 vrf forwarding 599
 ip address 192.168.142.253 255.255.255.0
 no ip redirects
 ip mtu 1496
 ip nbar protocol-discovery
 vrrp 57 address-family ipv4
  timers advertise 100
  vrrpv2
  track omp shutdown
  address 192.168.142.254 primary
  exit-vrrp
 arp timeout 1200
end

SITE5-DEV2#

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

MHM Cisco World · ‎07-03-2023

I will check and update you soon
MHM

Kanan Huseynli · ‎07-03-2023

Did you check my answer? You still have timers 100 which is 0.1sec.

HTH,
Please rate and mark as an accepted solution if you have found any of the information provided useful.

Rps-Cheers · ‎07-03-2023

HI Kanan

Yes, Your answer could be right. when i changed the vrrp advertise timers to 1000ms or 2000ms , it seems steady.

Before change the vrrp timer.

SITE5-DEV2#sho run inter Gi4.1599
Building configuration...

Current configuration : 331 bytes
!
interface GigabitEthernet4.1599
 encapsulation dot1Q 1599
 vrf forwarding 599
 ip address 192.168.141.253 255.255.255.0
 no ip redirects
 ip mtu 1496
 ip nbar protocol-discovery
 vrrp 59 address-family ipv4
  timers advertise 100
  vrrpv2
  track omp shutdown
  address 192.168.141.254 primary
  exit-vrrp
 arp timeout 1200
end

SITE5-DEV2#sho run inter Gi4.2599
Building configuration...

Current configuration : 331 bytes
!
interface GigabitEthernet4.2599
 encapsulation dot1Q 2599
 vrf forwarding 599
 ip address 192.168.142.253 255.255.255.0
 no ip redirects
 ip mtu 1496
 ip nbar protocol-discovery
 vrrp 57 address-family ipv4
  timers advertise 100
  vrrpv2
  track omp shutdown
  address 192.168.142.254 primary
  exit-vrrp
 arp timeout 1200
end

SITE5-DEV2#show logg last 10 | i VRRP
Jul 3 13:03:19.496: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul 3 13:03:19.499: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul 3 13:03:20.441: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul 3 13:03:20.444: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state BACKUP -> MASTER
Jul 3 13:03:20.445: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul 3 13:03:20.445: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul 3 13:04:00.660: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state BACKUP -> MASTER
Jul 3 13:04:00.690: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul 3 13:04:00.696: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul 3 13:04:00.712: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
SITE5-DEV2#

Changed the configuration of vrrp timer

SITE5-DEV2#sho run inter Gi4.1599
Building configuration...

Current configuration : 308 bytes
!
interface GigabitEthernet4.1599
 encapsulation dot1Q 1599
 vrf forwarding 599
 ip address 192.168.141.253 255.255.255.0
 no ip redirects
 ip mtu 1496
 ip nbar protocol-discovery
 vrrp 59 address-family ipv4 // there should be command ‘ timers advertise 1000’ ,so the 1000 ms should a default
  vrrpv2
  track omp shutdown
  address 192.168.141.254 primary
  exit-vrrp
 arp timeout 1200
end

SITE5-DEV2#sho run inter Gi4.2599
Building configuration...

Current configuration : 332 bytes
!
interface GigabitEthernet4.2599
 encapsulation dot1Q 2599
 vrf forwarding 599
 ip address 192.168.142.253 255.255.255.0
 no ip redirects
 ip mtu 1496
 ip nbar protocol-discovery
 vrrp 57 address-family ipv4
  timers advertise 2000
  vrrpv2
  track omp shutdown
  address 192.168.142.254 primary
  exit-vrrp
 arp timeout 1200
end

SITE5-DEV2#sho logg last 30 | i VRRP
Jul 3 13:07:20.585: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul 3 13:07:20.591: %VRRP-6-STATE: GigabitEthernet4.2599 IPv4 group 57 state MASTER -> BACKUP
Jul 3 13:08:19.335: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul 3 13:08:19.654: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
Jul 3 13:08:20.471: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state BACKUP -> MASTER
Jul 3 13:08:20.483: %VRRP-6-STATE: GigabitEthernet4.1599 IPv4 group 59 state MASTER -> BACKUP
SITE5-DEV2#

But in the feature template configuration, 100ms seems to be the default parameter value.This is very strange!

According to the above information, I found the following bug-CSCwd95581 , which is the key to the problem!

VRRP timer in vManage UI - default 100ms
CSCwd95581

Symptom: cEdge VRRP default advertise time is 100 to 40950 milliseconds. Default time should be 1000ms(not 100 ms), need to fix it in UI.

Conditions: cedge devices

Workaround: -

Further Problem Description: cEdge VRRP default advertise time is 100 to 40950 milliseconds. Default time should be 1000ms(not 100 ms), need to fix it in UI.

https://bst.cisco.com/bugsearch/bug/CSCwd95581

Maybe this problem will be fixed in Release 20.11.1. Anyway, the problem is solved! Thanks for everyone's help

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Rps-Cheers · ‎07-03-2023

I raised this issue in October last year, but Cisco only created this bug-CSCwd95581 on December 22, 2022. Due to the time difference, and it did not have a particularly large impact later, I ignored it for the time being. Thanks again for everyone reminder and help

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Kanan Huseynli · ‎07-03-2023

Glad, my suggestion helped.

Indeed, it is fixed and it is 1000msec in 20.11 (checked for CSR8000v).

HTH,
Please rate and mark as an accepted solution if you have found any of the information provided useful.

Rps-Cheers · ‎07-03-2023

HI Kanan

Greatly appreciated .Thanks for your proof

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Pankaj.bandewar1 · ‎07-04-2023

Lets hope it works for my setup as well , we are running C8300-1N1S-6T in out env. will update the config in coming days & post it for sure about the results.

By the way , Highly appreciate your support