cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5758
Views
0
Helpful
17
Replies

HSRP Failover on Nexus 7k for no reason

We had major Network outage on April 9 and 10th due to HSRP failover on our distribution layer Nexus 7k switches. Few Nutanix hosts were unable to reach their gateway and as a result multiple applications that hosted these Nutanix environment were affected. Any idea what could be the reason of this behavior?

 

2019 Apr 9 22:22:05 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan1010 IPV4 Grp 1010 Standby router is 0.0.0.0
2019 Apr 9 22:22:05 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan1010 IPV4 Grp 1010 Standby router priority changed to 0
2019 Apr 9 22:22:05 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1010 IPV4 Grp 1010 state changed from Standby to Active reason Act Timer E
xpired.
2019 Apr 9 22:22:05 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan1010 IPV4 Grp 1010 Active router is 10.100.20.2
2019 Apr 9 22:22:05 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan1010 IPV4 Grp 1010 Active router priority changed to 120
2019 Apr 9 22:22:05 AM-US-SAT_6-DD-35-BDRLFSW01 %ARP-3-DUP_VADDR_SRC_IP: arp [10967] Source address of packet received from 0000.0c9f.f1c9 on Vlan457(Gateway Port-Ch
annel1:1100) is duplicate of local virtual ip, 172.28.157.1
2019 Apr 9 22:22:05 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan452 IPV4 Grp 452 Standby router is 172.28.152.3
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan3765 IPV4 Grp 3765 Standby router is 10.12.20.3
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan458 IPV4 Grp 458 Standby router is 172.28.158.3
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan951 IPV4 Grp 951 Standby router is 10.12.250.131
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan3763 IPV4 Grp 3763 Standby router is 10.12.18.3
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan560 IPV4 Grp 560 Standby router is 172.28.160.3
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan3014 IPV4 Grp 3014 Standby router is 10.12.14.3
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan3006 IPV4 Grp 3006 Standby router is 0.0.0.0
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan3006 IPV4 Grp 3006 Standby router priority changed to 0
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan3006 IPV4 Grp 3006 state changed from Standby to Active reason Act Timer E
xpired.
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan3006 IPV4 Grp 3006 Active router is 10.12.5.3
2019 Apr 9 22:22:06 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan3006 IPV4 Grp 3006 Active router priority changed to 100
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan439 IPV4 Grp 439 Standby router is 172.28.139.3
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan1410 IPV4 Grp 1410 Standby router is 0.0.0.0
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan1410 IPV4 Grp 1410 Standby router priority changed to 0
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1410 IPV4 Grp 1410 state changed from Standby to Active reason Act Timer E
xpired.
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan1410 IPV4 Grp 1410 Active router is 10.100.220.2
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan1410 IPV4 Grp 1410 Active router priority changed to 120
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan410 IPV4 Grp 410 Standby router is 172.28.110.3
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:07 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan3764 IPV4 Grp 3764 Standby router is 10.12.19.3
2019 Apr 9 22:22:08 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan432 IPV4 Grp 432 Standby router is 172.28.132.3
2019 Apr 9 22:22:09 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan1010 IPV4 Grp 1010 Active router is 10.100.20.3
2019 Apr 9 22:22:09 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 1 time
2019 Apr 9 22:22:09 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan1010 IPV4 Grp 1010 Active router priority changed to 120
2019 Apr 9 22:22:09 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1010 IPV4 Grp 1010 state changed from Active to Speak reason Rx Hello-Hi P
ri Act.
2019 Apr 9 22:22:10 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:11 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan1410 IPV4 Grp 1410 Active router is 10.100.220.3
2019 Apr 9 22:22:11 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 1 time
2019 Apr 9 22:22:11 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan1410 IPV4 Grp 1410 Active router priority changed to 120
2019 Apr 9 22:22:11 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1410 IPV4 Grp 1410 state changed from Active to Speak reason Rx Hello-Hi P
ri Act.
2019 Apr 9 22:22:13 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 2 times
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1010 IPV4 Grp 1010 state changed from Speak to Standby reason Sby Timer Ex
pired.
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan1010 IPV4 Grp 1010 Standby router is 10.100.20.2
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan1010 IPV4 Grp 1010 Standby router priority changed to 120
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel411: first operational port changed from Ethernet1/12 to none
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel411: Ethernet1/12 is down
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %VPC-5-INTF_CONSISTENCY_SUCCESS: In domain 10, vPC 411 configuration is consistent
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel411 is down (No operational members)
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %VPC-5-INTF_CONSISTENCY_SUCCESS: In domain 10, vPC 411 configuration is consistent
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 1 time
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %STP-6-PORT_RANGE_DELETED: Interface port-channel411 removed from vlan=110,751-752
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %STP-6-PORT_RANGE_ROLE: new_role=designated interface=port-channel411 vlan=110,751-752
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %VPC-5-INTF_CONSISTENCY_SUCCESS: In domain 10, vPC 411 configuration is consistent
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %STP-6-PORT_RANGE_STATE: new_state=disabled interface=port-channel411 vlan=110
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %STP-6-PORT_RANGE_STATE: new_state=disabled interface=port-channel411 vlan=751-752
2019 Apr 9 22:22:20 AM-US-SAT_6-DD-35-BDRLFSW01 %VPC-5-INTF_CONSISTENCY_SUCCESS: In domain 10, vPC 411 configuration is consistent
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 6 times
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1410 IPV4 Grp 1410 state changed from Speak to Standby reason Sby Timer Ex
pired.
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan1410 IPV4 Grp 1410 Standby router is 10.100.220.2
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan1410 IPV4 Grp 1410 Standby router priority changed to 120
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel411,bandwidth changed to 100000 Kbit
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/12 is down (Link failure)
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel411 is down (No operational members)
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 %CDP-5-NEIGHBOR_REMOVED: CDP Neighbor amussat-cube03.kci.com on port Ethernet1/12 has been removed
2019 Apr 9 22:22:21 AM-US-SAT_6-DD-35-BDRLFSW01 %VPC-5-INTF_CONSISTENCY_SUCCESS: In domain 10, vPC 411 configuration is consistent
2019 Apr 9 22:22:22 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:22 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan3004 IPV4 Grp 3004 Standby router is 0.0.0.0
2019 Apr 9 22:22:22 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan3004 IPV4 Grp 3004 Standby router priority changed to 0
2019 Apr 9 22:22:22 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan3004 IPV4 Grp 3004 state changed from Standby to Active reason Act Timer E
xpired.
2019 Apr 9 22:22:22 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan3004 IPV4 Grp 3004 Active router is 10.12.4.3
2019 Apr 9 22:22:22 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan3004 IPV4 Grp 3004 Active router priority changed to 100
2019 Apr 9 22:22:25 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:30 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 1 time
2019 Apr 9 22:22:30 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan1010 IPV4 Grp 1010 Standby router is 0.0.0.0
2019 Apr 9 22:22:30 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan1010 IPV4 Grp 1010 Standby router priority changed to 0
2019 Apr 9 22:22:30 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1010 IPV4 Grp 1010 state changed from Standby to Active reason Act Timer E
xpired.
2019 Apr 9 22:22:30 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan1010 IPV4 Grp 1010 Active router is 10.100.20.2
2019 Apr 9 22:22:30 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan1010 IPV4 Grp 1010 Active router priority changed to 120
2019 Apr 9 22:22:31 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:32 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan1410 IPV4 Grp 1410 Standby router is 0.0.0.0
2019 Apr 9 22:22:32 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan1410 IPV4 Grp 1410 Standby router priority changed to 0
2019 Apr 9 22:22:32 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1410 IPV4 Grp 1410 state changed from Standby to Active reason Act Timer E
xpired.
2019 Apr 9 22:22:32 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan1410 IPV4 Grp 1410 Active router is 10.100.220.2
2019 Apr 9 22:22:32 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan1410 IPV4 Grp 1410 Active router priority changed to 120
2019 Apr 9 22:22:34 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:44 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 3 times
2019 Apr 9 22:22:44 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-SPEED: Interface Ethernet1/15, operational speed changed to 10 Gbps
2019 Apr 9 22:22:44 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/15, operational duplex mode changed to Full
2019 Apr 9 22:22:44 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/15, operational Receive Flow Control state changed to off
2019 Apr 9 22:22:44 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/15, operational Transmit Flow Control state changed to off
2019 Apr 9 22:22:46 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:47 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan1010 IPV4 Grp 1010 Active router is 10.100.20.3
2019 Apr 9 22:22:47 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 1 time
2019 Apr 9 22:22:47 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan1010 IPV4 Grp 1010 Active router priority changed to 120
2019 Apr 9 22:22:47 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1010 IPV4 Grp 1010 state changed from Active to Speak reason Rx Hello-Hi P
ri Act.
2019 Apr 9 22:22:49 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:54 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 4 times
2019 Apr 9 22:22:54 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVE_CHANGE: Interface Vlan599 IPV4 Grp 599 Active router is 172.28.219.4
2019 Apr 9 22:22:54 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 1 time
2019 Apr 9 22:22:54 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-ACTIVEROUTERPRIORITY: Interface Vlan599 IPV4 Grp 599 Active router priority changed to 120
2019 Apr 9 22:22:54 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan599 IPV4 Grp 599 state changed from Active to Speak reason Rx Hello-Hi Pri
Act.
2019 Apr 9 22:22:56 AM-US-SAT_6-DD-35-BDRLFSW01 %CDP-5-NEIGHBOR_ADDED: Device amussat-p6kvss.kci.com discovered of type cisco WS-C6506-E with port TenGigabitEthernet1/
5/15 on incoming port Ethernet1/15 with ip addr 192.168.67.1 and mgmt ip 192.168.67.1
2019 Apr 9 22:22:56 AM-US-SAT_6-DD-35-BDRLFSW01 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel499: Ethernet1/15 is up
2019 Apr 9 22:22:57 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:22:57 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan1010 IPV4 Grp 1010 state changed from Speak to Standby reason Sby Timer Ex
pired.
2019 Apr 9 22:22:57 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBY_CHANGE: Interface Vlan1010 IPV4 Grp 1010 Standby router is 10.100.20.2
2019 Apr 9 22:22:57 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-STANDBYROUTERPRIORITYCHANGE: Interface Vlan1010 IPV4 Grp 1010 Standby router priority changed to 120
2019 Apr 9 22:22:57 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_TRUNK_UP: Interface Ethernet1/15, vlan 1,27,75-76,84,110,112,127,246-249,251-255,261,299,400-427,429-459,
491-505,534,553,555,557,560,599,627,649,651,654,667,700,711 up
2019 Apr 9 22:22:57 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_TRUNK_UP: Interface Ethernet1/15, vlan 751-753,777,800,866,888-890,912,950-951,1005,1010,1103-1104,1305,1
403,1410,1710,1722,1728,3004-3006,3012,3014,3751-3771 up
2019 Apr 9 22:22:57 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel499,bandwidth changed to 20000000 Kbit
2019 Apr 9 22:22:57 AM-US-SAT_6-DD-35-BDRLFSW01 %ETHPORT-5-IF_UP: Interface Ethernet1/15 is up in mode trunk
2019 Apr 9 22:22:59 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-4-DIFFVIP1: Vlan599 Grp 599 active routers virtual IP address 172.28.219.254 is different to the locally c
onfigured address 172.28.219.1
2019 Apr 9 22:23:05 AM-US-SAT_6-DD-35-BDRLFSW01 last message repeated 1 time
2019 Apr 9 22:23:05 AM-US-SAT_6-DD-35-BDRLFSW01 %HSRP_ENGINE-5-GRPSTATECHANGE: Interface Vlan599 IPV4 Grp 599 state changed from Speak to Standby reason Sby Timer Expi
red.

17 Replies 17

Hello,

 

--> VLAN 599 is spanned between our datacenter at San Antonio and DR at Scottsdale with OTV. The gateway for VLAN 599 is .1 in San Antonio but it is .254 in DR. This is by design.

 

How is San Antonio connected to your two Nexus switches ? Does it use the same HSRP group ?

Yes, we are using HSRP.

 

AM-US-SAT_6-DD-35-BDRLFSW01# sh running-config hsrp all | begin 599
interface Vlan599
no hsrp bfd
hsrp version 2
hsrp delay minimum 0 reload 0
no hsrp use-bia
no hsrp mac-refresh
hsrp 599
authentication XXXX
mac-address 0000.0C9F.F257
preempt delay minimum 0 reload 0 sync 0
priority 130
timers 3 10
ip 172.28.219.1

 

AM-US-SAT_6-GD-35-BDRLFSW02# sh run hsrp | begin 599
interface Vlan599
hsrp version 2
hsrp 599
preempt
ip 172.28.219.1

 

AM-US-SCF_01-0726-35-BDRLFSW01# sh running-config hsrp | begin 599  (This is DR center)
interface Vlan599
hsrp version 2
hsrp 599
preempt
priority 120
ip 172.28.219.254

Are we missing authentication configuration on BDRLFSW02? To be honest, I feel some configuration is missing on BDRLFSW02 or miss configuration that has caused this issue. Please see below.

 

AM-US-SAT_6-DD-35-BDRLFSW01# sh hsrp group 599 brief
*:IPv6 group #:group belongs to a bundle
P indicates configured to preempt.
|
Interface Grp Prio P State Active addr Standby addr Group addr
Vlan599 599 130 P Active local 172.28.219.4 172.28.219.1 (conf)

 

AM-US-SAT_6-GD-35-BDRLFSW02# sh hsrp group 599 brief
*:IPv6 group #:group belongs to a bundle
P indicates configured to preempt.
|
Interface Grp Prio P State Active addr Standby addr Group addr
Vlan599 599 100 P Listen 172.28.219.2 172.28.219.4 172.28.219.1 (conf)

 

Above output says 172.28.219.2 is the active device and 172.28.219.4 is the stand by device. But when we did a SSH to 172.28.219.4, it is taking me to AM-US-SCF_01-0726-35-BDRLFSW01 which is at our disaster recovery datacenter.

 

AM-US-SCF_01-0726-35-BDRLFSW01# sh hsrp group 599 brief
*:IPv6 group #:group belongs to a bundle
P indicates configured to preempt.
|
Interface Grp Prio P State Active addr Standby addr Group addr
Vlan599 599 120 P Standby 172.28.219.2 local 172.28.219.254 (conf)

 

HSRP is in stand by state for VLAN599 in our DR center. But, this should be like this, our requirement is when AM-US-SAT_6-DD-35-BDRLFSW01 fails AM-US-SAT_6-DD-35-BDRLFSW02 should take over the traffic not AM-US-SCF_01-0726-35-BDRLFSW01 becuase the group address are different at San Antonio DC and Scottsdale DR. Please advise.

Your priority needs to be readjusted.  By default, priority is 100.  Higher priority determines the order of precedence.  

See link, https://www.cisco.com/c/en/us/support/docs/switches/nexus-7000-series-switches/113002-nexus-hsrp-00.html

  • BDRLFSW01 - keep "priority 130"
  • BDRLFSW02 - add "priority 120"
  • AM-US-SCF_01-0726-35-BDRLFSW01 - remove "priority 120"; add "priority 110"

You may want to remove HSRP authentication or add authentication on each HSRP members within the same group.

I can't say what cause your outage several weeks ago.  Do you recall any change that happened prior to the outage within the Nexus or outside network related?  Your company should have a configuration change manager who can tell you what work approved to perform that day.  

Review Cisco Networking for a $25 gift card