cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
754
Views
20
Helpful
18
Replies

3750G HSRP issues

dbuckley77
Beginner
Beginner

We have two 3750Gs setup with HSRP that we're seeing the below message on.  Pretty sure it's because HSRP messages are not getting shared correctly between the two but don't know why.

 

105331: May 11 16:29:48: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Standby -> Active
105332: 2y30w: HSRP: Vl114 Grp 1 Redundancy "hsrp-Vl114-1" state Standby -> Active
105333: 2y30w: HSRP: Vl114 Grp 1 Redundancy group hsrp-Vl114-1 state Active -> Active
105334: 2y30w: HSRP: Vl114 Grp 1 Redundancy group hsrp-Vl114-1 state Active -> Active
105335: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 found
105336: 2y30w: HSRP: Vl114 Grp 1 Hello in 10.100.x.x Active pri 150 vIP 10.100.x.x
105337: 2y30w: HSRP: Vl114 Grp 1 Active router is 10.100.x.x, was local
105338: 2y30w: HSRP: Vl114 Grp 1 Active: g/Hello rcvd from higher pri Active router (150/10.100.x.x)
105339: 2y30w: HSRP: Vl114 Grp 1 Active -> Speak
105340: May 11 16:30:08: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Active -> Speak
105341: 2y30w: HSRP: Vl114 Grp 1 Redundancy "hsrp-Vl114-1" state Active -> Speak

1 Accepted Solution

Accepted Solutions

First let me address the time stamp question. David says "it looks like it happened 2 years and 30 weeks ago" Actually that is not how long ago it happened but is how long this switch has been "up" when the event occurred. One of the options in logging messages is to use the up time as the time stamp and the usual option is to use current time. I believe that up time is the default if the device does not have authoritative time. I am puzzled at the mixture in these log messages of time stamps as up time and time stamp as current time. I don't know if that might point at something that would be significant.

Actually I just figured out the inconsistency. Check the configuration of timestamps

service timestamps debug uptime
service timestamps log datetime localtime

Some of these messages are generated by debug and are using uptime while other messages are generated by syslog.

I note that all of these messages refer to vlan 114. There are no messages about vlan 112 or 113. Is there something about vlan 114 that is different?

There is another inconsistency that I notice. There are messages like this

113257: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.231 not found

This address is not in the config that was posted. Where did it come from?

Recognizing that this message was generated by debug I guess the question is really where did debug get this from? It is still not consistent with the config.

I would like to focus on this group of messages

113287: 2y30w: HSRP: Vl114 Grp 1 Standby: c/Active timer expired (10.100.114.2)
113288: 2y30w: HSRP: Vl114 Grp 1 Active router is local, was 10.100.114.2
113289: 2y30w: HSRP: Vl114 Grp 1 Standby router is unknown, was local
113290: 2y30w: HSRP: Vl114 Grp 1 Standby -> Active

The first of these messages seems to indicate that there has been a normal/successful HSRP relationship between the switches and now switch 2 is no longe receiving HSRP messages from switch 1. The other 3 messages describe a typical failover where this switch (now the active member) is not in contact with the peer switch.

The message that follows them is a syslog message with a much more helpful timestamp

113291: May 12 12:29:03: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Standby -> Active

Skipping over a series of debug messages the next syslog message is this

113299: May 12 12:29:25: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Active -> Speak

This means that at 12:29:25 switch2 received a HSRP message from switch 1. Switch 1 with higher priority and preempt configured is taking over as the active router. And switch2 is going back to standby.

If we focus on the syslog messages we see that there was a login at May 12 12:22:39. The context suggests that HSRP was operating successfully with switch2 as standby.

At May 12 12:29:03 there was a failover based on not receiving messages from switch1. (note there are no log messages about any interface state changes).

At May 12 12:29:25 switch2 has received a HSRP message from switch1 and switch1 is resuming the role of active.

I would suggest that at this point switch2 appears to be normal and whatever issue there might be is to be found on switch1.

HTH

Rick

View solution in original post

18 Replies 18

MHM Cisco World
Advisor
Advisor

can you share the log before this Peer be from standby to active ?

Reza Sharifi
Hall of Fame Expert Hall of Fame Expert
Hall of Fame Expert

Is vlan 114 part of the trunk between the 2 switches? Is there a layer-2 connection between the switches?

Can you post relevant configs from both switches?.

HTH

There is a trunk directly between the two switches passing several vlans.  There are also four ports on each switch setup as access vlan that connect to each other through a series of non-Cisco switches not running spanning-tree.  So basically port 5 creates a ring between the two switches  on vlan 114.  So does port 6 and port 7 etc...multiple rings on vlan 114 between the two switches.

MHM Cisco World
Advisor
Advisor

show spanning tree 
check if trunk VLAN is forward or BLK, 
the spanning tree for VLAN 114 in both SW must forward not BLK.

SinghRaminder
Beginner
Beginner

Provide Relevant HSRP configuration from both switches and the port configuration on the trunks as well please

Thanks
Raminder
PS: If this answered your question, please don't forget to rate and select as validated answer

These are the configs from both switches:

 

 

3750G-1#sh run
Building configuration...

Current configuration : 7643 bytes
!
! Last configuration change at 15:04:20 EDT Wed May 11 2022 by xxxxxxxx
! NVRAM config last updated at 08:27:48 EDT Tue Mar 22 2022 by xxxxxxxxx
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log datetime localtime
service password-encryption
service sequence-numbers
!
hostname CMAQ-3750G-1
!
boot-start-marker
boot-end-marker
!
!
username XXXXXXX password 7 XXXXXXXXX
!
!
aaa new-model
!
!
aaa authentication login default group radius local
aaa authentication login CONSOLE local
aaa authentication enable default none
aaa authorization exec default if-authenticated
!
!
!
aaa session-id common
clock timezone Eastern -5
clock summer-time EDT recurring
switch 1 provision ws-c3750g-12s
system mtu routing 1500
vtp domain XXXXXXX
vtp mode transparent
ip routing
ip spd queue max-threshold 600
ip spd queue min-threshold 500
ip domain-name XXXXXXXXXXX
ip name-server 10.100.X.X
ip name-server 10.100.X.X
ip dhcp excluded-address 10.100.113.1 10.100.113.5
ip dhcp excluded-address 10.100.113.9 10.100.113.254
ip dhcp excluded-address 10.100.114.1 10.100.114.5
ip dhcp excluded-address 10.100.114.9 10.100.114.254
ip dhcp excluded-address 10.100.112.1 10.100.112.5
ip dhcp excluded-address 10.100.112.9 10.100.112.254
!
ip dhcp pool vlan112
network 10.100.112.0 255.255.255.0
default-router 10.100.112.1
domain-name nashua.city
dns-server 10.100.6.4
!
ip dhcp pool vlan113
network 10.100.113.0 255.255.255.0
default-router 10.100.113.1
domain-name XXXXXXXXX
dns-server 10.100.5.11 10.100.6.4
!
ip dhcp pool vlan114
network 10.100.114.0 255.255.255.0
default-router 10.100.114.1
domain-name nashua.city
dns-server 10.100.5.11 10.100.6.4
!
ip dhcp pool vlan111
dns-server 10.100.5.11 10.100.6.4
!
!
login on-failure
login on-success
!
!
crypto pki trustpoint TP-self-signed-2415001216
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-2415001216
revocation-check none
rsakeypair TP-self-signed-2415001216
!
!
crypto pki certificate chain TP-self-signed-2415001216
certificate self-signed 01
30820250 308201B9 A0030201 02020101 300D0609 2A864886 F70D0101 04050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 32343135 30303132 3136301E 170D3933 30333031 30303031
34385A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D32 34313530
30313231 3630819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
8100D77E 568A4C6E 9C6EBC21 2A127083 D133AB16 B9BF741D 3487F382 7E980185
0F505E13 DB3EE4A6 68FAAE56 5F711C50 4D435812 4670ED84 895D9367 9C4FCB66
CF7B2B80 1DBFE7F4 A63C3C4D 2A4E72C0 9FA2F1D6 C8DCE88E 396F1137 C7DEB2F4
CF6B2D64 AB2AFAA0 3D9991AB 0B127D06 EC523A40 799746A2 6C074285 25B87655
230F0203 010001A3 78307630 0F060355 1D130101 FF040530 030101FF 30230603
551D1104 1C301A82 18434D41 512D3337 3530472D 312E6E61 73687561 2E636974
79301F06 03551D23 04183016 801424A2 95EBB4F1 68F2E8EC 1391FA0A A5415DB7
5F35301D 0603551D 0E041604 1424A295 EBB4F168 F2E8EC13 91FA0AA5 415DB75F
35300D06 092A8648 86F70D01 01040500 03818100 A7821A61 0E0A43D8 A8B86983
5BA37197 2171C1C3 83DAC90B 9BE0EF21 45B3A8E8 8B50150A 6304EC35 31A9A222
C1E7F95D F56FEDBB 46D952A4 C4A6792A D3263C7A 97ABAFC2 68545047 132B28FB
D5CB814C DAC6A97B EE865B20 006A12C9 E306E2DD A6959CFA F23187E2 F883623C
39126B5C 1B956C76 8DFDB45C 7910D5FC DB0AEABC
quit
!
!
!
!
spanning-tree mode pvst
spanning-tree logging
no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
spanning-tree vlan 112-114 priority 8192
spanning-tree vlan 112-113 max-age 35
!
vlan internal allocation policy ascending
!
vlan 9
name Interconnect
!
vlan 112-114
!
!
!
interface GigabitEthernet1/0/1
description XXXXXXX
switchport access vlan 113
!
interface GigabitEthernet1/0/2
description XXXXXXXXX
switchport access vlan 113
!
interface GigabitEthernet1/0/3
description XXXXXXXX
switchport access vlan 113
!
interface GigabitEthernet1/0/4
switchport access vlan 113
!
interface GigabitEthernet1/0/5
description XXXXXXX
switchport access vlan 114
!
interface GigabitEthernet1/0/6
description XXXXXXX
switchport access vlan 114
!
interface GigabitEthernet1/0/7
description XXXXXXX
switchport access vlan 114
!
interface GigabitEthernet1/0/8
description XXXXXXX
switchport access vlan 114
!
interface GigabitEthernet1/0/9
description DWH_Grahm
switchport access vlan 112
switchport mode access
!
interface GigabitEthernet1/0/10
switchport access vlan 114
!
interface GigabitEthernet1/0/11
description XXXXXXX
switchport access vlan 9
!
interface GigabitEthernet1/0/12
description tRUNK TO 3750G-2 Gig 1/0/12
switchport trunk encapsulation dot1q
switchport trunk native vlan 9
switchport trunk allowed vlan 9,112-114
switchport mode trunk
!
interface Vlan1
no ip address
shutdown
!
interface Vlan9
ip address 10.100.9.5 255.255.255.0
ip access-group 110 in
!
interface Vlan112
ip address 10.100.112.2 255.255.255.0
no ip redirects
standby 1 ip 10.100.112.1
standby 1 priority 150
standby 1 preempt delay minimum 3
!
interface Vlan113
ip address 10.100.113.2 255.255.255.0
no ip redirects
standby 1 ip 10.100.113.1
standby 1 priority 150
standby 1 preempt delay minimum 3
!
interface Vlan114
ip address 10.100.114.2 255.255.255.0
no ip redirects
standby 1 ip 10.100.114.1
standby 1 priority 150
standby 1 preempt delay minimum 3
hold-queue 500 in
!
router ospf 1
router-id 10.100.9.5
log-adjacency-changes
passive-interface default
no passive-interface Vlan9
no passive-interface GigabitEthernet1/0/11
network 10.100.9.0 0.0.0.255 area 0
network 10.100.112.0 0.0.0.255 area 0
network 10.100.113.0 0.0.0.255 area 0
network 10.100.114.0 0.0.0.255 area 0
!
ip classless
ip route 0.0.0.0 0.0.0.0 10.100.9.1
no ip http server
ip http access-class 5
ip http authentication local
ip http secure-server
!
ip radius source-interface Vlan9
logging trap notifications
logging 10.100.6.56
access-list 5 permit 10.100.5.34
access-list 5 permit 10.100.2.0 0.0.0.255
access-list 5 permit 10.100.95.0 0.0.0.63
access-list 99 deny any
access-list 99 remark Drop OSPF routes
access-list 110 permit ip 10.100.95.0 0.0.0.63 any
access-list 110 permit ip host 10.100.5.34 any
access-list 110 permit ip 10.100.107.0 0.0.0.255 any
access-list 110 permit icmp host 10.100.6.17 any
access-list 110 permit ip host 10.100.110.55 any
access-list 110 permit ip 10.100.9.0 0.0.0.255 host 10.100.9.5
access-list 110 permit ospf any host 224.0.0.5
access-list 110 permit ospf any host 224.0.0.6
access-list 110 permit udp host 10.100.5.11 any
access-list 110 permit ip host 10.100.5.103 any
access-list 110 permit udp host 10.100.6.4 any
access-list 110 permit ip host 10.100.6.21 any
access-list 110 permit ip host 10.100.6.16 any
access-list dynamic-extended
snmp-server community XXXXXXXXXX RO
snmp-server location "XXXXXXXXXXXX"
radius-server host X.X.X.X auth-port 1812 acct-port 1813
radius-server retransmit 5
radius-server timeout 3
radius-server key 7 XXXXXXXXXXXXX
!
banner login ^C
XXXXXXXXXXXXXXXXXXX
Un-authorized tampering with this equipment is punishable by law
Do not attempt to login if you are not authorized
.
'^C
!
line con 0
login authentication CONSOLE
line vty 0 4
session-timeout 30
access-class 5 in
privilege level 15
transport input ssh
line vty 5 15
transport input none
!
ntp clock-period 36029443
ntp

 

 

 

3750G-2#sh run
Building configuration...

Current configuration : 6966 bytes
!
! Last configuration change at 16:04:50 EDT Wed May 11 2022 by xxxxxxxxx
! NVRAM config last updated at 08:27:44 Eastern Tue Mar 22 2022 by xxxxxxx
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log datetime localtime
no service password-encryption
service sequence-numbers
no service dhcp
!
hostname 3750G-2
!
!
username XXXXXXX privilege 15 password 7 XXXXXXXXXXXX
aaa new-model
aaa authentication login default group radius local
aaa authentication login CONSOLE local
aaa authentication enable default none
aaa authorization exec default if-authenticated
!
aaa session-id common
clock timezone Eastern -4
clock summer-time EDT recurring
switch 1 provision ws-c3750g-12s
vtp domain XXXXXXX
vtp mode transparent
no ip subnet-zero
ip routing
ip spd queue max-threshold 600
ip spd queue min-threshold 500
ip domain-name XXXXXXXXXXX
ip name-server X.X.X.X
ip name-server X.X.X.X
!
login on-failure
login on-success
!
!
crypto pki trustpoint TP-self-signed-656894208
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-656894208
revocation-check none
rsakeypair TP-self-signed-656894208
!
!
crypto ca certificate chain TP-self-signed-656894208
certificate self-signed 01
308202A6 3082020F A0030201 02020101 300D0609 2A864886 F70D0101 04050030
5B312E30 2C060355 04031325 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 36353638 39343230 38312930 2706092A 864886F7 0D010902
161A5472 61666669 632D4D61 696E2D32 2E6E6173 6875612E 63697479 301E170D
31343034 33303132 34343030 5A170D32 30303130 31303030 3030305A 305B312E
302C0603 55040313 25494F53 2D53656C 662D5369 676E6564 2D436572 74696669
63617465 2D363536 38393432 30383129 30270609 2A864886 F70D0109 02161A54
72616666 69632D4D 61696E2D 322E6E61 73687561 2E636974 7930819F 300D0609
2A864886 F70D0101 01050003 818D0030 81890281 8100D6FE 7351AEFC 0D92CBF0
755B4AB8 9E51A346 D91FA7E6 A83F3A0C 08C4356A CE593DCB 53A68256 DD924571
F215B576 6F69356F 96A785C5 45CFEBB9 CAAD0A99 E605B9DD 3BB17F57 62BABE30
2281781F 6CE50B52 99FC7F4B AC9A4303 B2E62EE7 241B4FE6 0F2FCCEE 7648E41F
69A81EE0 9310F413 20C85190 6C79E773 1FFC0BAA A77B0203 010001A3 7A307830
0F060355 1D130101 FF040530 030101FF 30250603 551D1104 1E301C82 1A547261
66666963 2D4D6169 6E2D322E 6E617368 75612E63 69747930 1F060355 1D230418
30168014 5FB0C18B 8E3BDF89 EA2595F1 7EFE2838 AA76BBBF 301D0603 551D0E04
1604145F B0C18B8E 3BDF89EA 2595F17E FE2838AA 76BBBF30 0D06092A 864886F7
0D010104 05000381 81000B87 D3A80300 21DD7FEE CED6443D 04B2FFF2 2F915E1C
2F6B3A4B 5E2D4270 191779FD C4FCDAE0 0C9DB402 9445751C 738AB7B6 391CAF91
797B8D7B 4F3D8DA8 E5BA23A1 5E10332D BC2C94F1 A036CA44 DCE9EDBD 298275F4
1D4B2E40 A19CD8B8 BA63A465 71EE7CE8 2B7DA1AD 14D6B463 5342A11A 2CD2A6E1
A6FD5F7B 4AB50113 1CCD
quit
!
!
no file verify auto
!
spanning-tree mode pvst
spanning-tree logging
no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
spanning-tree vlan 112-114 priority 8192
spanning-tree vlan 112-113 max-age 35
!
vlan internal allocation policy ascending
!
vlan 9
name Interconnect
!
vlan 112-114
!
!
interface GigabitEthernet1/0/1
description xxxxxxx
switchport access vlan 113
spanning-tree cost 100
!
interface GigabitEthernet1/0/2
description XXXXXXX
switchport access vlan 113
spanning-tree cost 100
!
interface GigabitEthernet1/0/3
description XXXXXXX
switchport access vlan 113
!
interface GigabitEthernet1/0/4
description XXXXXXXX
switchport access vlan 113
!
interface GigabitEthernet1/0/5
description XXXXXXXX
switchport access vlan 114
spanning-tree cost 100
!
interface GigabitEthernet1/0/6
description XXXXXXX
switchport access vlan 114
!
interface GigabitEthernet1/0/7
description XXXXXXX
switchport access vlan 114
spanning-tree cost 100
!
interface GigabitEthernet1/0/8
description XXXXXXX
switchport access vlan 114
spanning-tree cost 100
!
interface GigabitEthernet1/0/9
description XXXXXXXXX
switchport access vlan 112
switchport mode access
spanning-tree cost 100
!
interface GigabitEthernet1/0/10
switchport access vlan 114
spanning-tree cost 100
!
interface GigabitEthernet1/0/11
description CH-Core-6807-XL Gi2/2/47
switchport access vlan 9
!
interface GigabitEthernet1/0/12
description Trunk to 3750G-1 Gi1/0/12
switchport trunk encapsulation dot1q
switchport trunk native vlan 9
switchport trunk allowed vlan 9,112-114
switchport mode trunk
spanning-tree cost 100
!
interface Vlan1
no ip address
shutdown
!
interface Vlan2
no ip address
!
interface Vlan9
ip address 10.100.9.6 255.255.255.0
ip access-group 110 in
!
interface Vlan112
ip address 10.100.112.3 255.255.255.0
no ip redirects
standby 1 ip 10.100.112.1
!
interface Vlan113
ip address 10.100.113.3 255.255.255.0
no ip redirects
standby 1 ip 10.100.113.1
!
interface Vlan114
ip address 10.100.114.3 255.255.255.0
no ip redirects
standby 1 ip 10.100.114.1
hold-queue 500 in
!
router ospf 1
router-id 10.100.9.6
log-adjacency-changes
passive-interface default
no passive-interface Vlan9
no passive-interface GigabitEthernet1/0/11
network 10.100.9.0 0.0.0.255 area 0
network 10.100.112.0 0.0.0.255 area 0
network 10.100.113.0 0.0.0.255 area 0
network 10.100.114.0 0.0.0.255 area 0
!
ip classless
ip route 0.0.0.0 0.0.0.0 10.100.9.1
no ip http server
ip http access-class 5
ip http secure-server
!
!
logging trap notifications
logging 10.100.6.56
access-list 5 permit 10.100.5.34
access-list 5 permit 10.100.2.0 0.0.0.255
access-list 5 permit 10.100.95.0 0.0.0.63
access-list 99 deny any
access-list 99 remark Drop OSPF routes
access-list 110 permit ip 10.100.95.0 0.0.0.63 any
access-list 110 permit ip host 10.100.5.34 any
access-list 110 permit ip 10.100.107.0 0.0.0.255 any
access-list 110 permit udp host 10.100.5.11 any
access-list 110 permit ip host 10.100.110.55 any
access-list 110 permit ospf any host 224.0.0.6
access-list 110 permit ospf any host 224.0.0.5
access-list 110 permit ip host 10.100.6.21 any
access-list 110 permit ip 10.100.9.0 0.0.0.255 host 10.100.9.6
access-list 110 permit ip host 10.100.5.103 any
access-list 110 permit udp host 10.100.6.4 any
access-list 110 permit icmp host 10.100.6.17 any
access-list 110 permit ip host 10.100.6.16 any
snmp-server community xxxxxxxxx RO
radius-server host x.x.x.x auth-port 1812 acct-port 1813
radius-server source-ports 1645-1646
radius-server retransmit 5
radius-server timeout 3
radius-server key XXXXXXXXX
!
control-plane
!
banner login ^C
xxxxxxxxxxxxxxxxxxxxx
Un-authorized tampering with this equipment is punishable by law
Do not attempt to login if you are not authorized
.
'^C
!
line con 0
login authentication CONSOLE
line vty 0 4
session-timeout 30
access-class 5 in
privilege level 15
transport input ssh
line vty 5 15
transport input none
!
ntp clock-period 36029457
ntp server 10.100.x.x
end

 

Hello,

 

The configuration looks fine from initial read. As @paul driver mentioned the timestamp...it looks like it happened 2 years and 30 weeks ago if we read that right? Is this a problem you are having currently. Do you have more recent logs?

 

Can you enter the command:

 

show standby or show standby brief?

 

 

-David

Here is a current log from switch 2 and sh standy from both switches:

 

3750G-2#sh logging
Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
Console logging: level debugging, 113304 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 113304 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Trap logging: level notifications, 4682 message lines logged
Logging to 10.100.6.56, 4682 message lines logged, xml disabled,
filtering disabled

Log Buffer (4096 bytes):
virtual address 10.100.114.1 not found
113253: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113254: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113255: May 12 12:22:39: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: admindana] [Source: 10.100.95.11] [localport: 22] at 12:22:39 EDT Thu May 12 2022
113256: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113257: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.231 not found
113258: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113259: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113260: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113261: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113262: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113263: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.3 not found
113264: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113265: 2y30w: HSRP: Vl112 API active virtual address 10.100.112.1 not found
113266: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113267: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113268: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113269: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113270: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113271: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113272: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113273: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113274: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113275: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113276: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113277: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113278: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113279: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113280: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113281: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.3 not found
113282: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113283: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.136 not found
113284: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113285: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113286: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.1 not found
113287: 2y30w: HSRP: Vl114 Grp 1 Standby: c/Active timer expired (10.100.114.2)
113288: 2y30w: HSRP: Vl114 Grp 1 Active router is local, was 10.100.114.2
113289: 2y30w: HSRP: Vl114 Grp 1 Standby router is unknown, was local
113290: 2y30w: HSRP: Vl114 Grp 1 Standby -> Active
113291: May 12 12:29:03: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Standby -> Active
113292: 2y30w: HSRP: Vl114 Grp 1 Redundancy "hsrp-Vl114-1" state Standby -> Active
113293: 2y30w: HSRP: Vl114 Grp 1 Redundancy group hsrp-Vl114-1 state Active -> Active
113294: 2y30w: HSRP: Vl114 Grp 1 Redundancy group hsrp-Vl114-1 state Active -> Active
113295: 2y30w: HSRP: Vl114 Grp 1 Hello in 10.100.114.2 Active pri 150 vIP 10.100.114.1
113296: 2y30w: HSRP: Vl114 Grp 1 Active router is 10.100.114.2, was local
113297: 2y30w: HSRP: Vl114 Grp 1 Active: g/Hello rcvd from higher pri Active router (150/10.100.114.2)
113298: 2y30w: HSRP: Vl114 Grp 1 Active -> Speak
113299: May 12 12:29:25: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Active -> Speak
113300: 2y30w: HSRP: Vl114 Grp 1 Redundancy "hsrp-Vl114-1" state Active -> Speak
113301: 2y30w: HSRP: Vl114 Grp 1 Speak: d/Standby timer expired (unknown)
113302: 2y30w: HSRP: Vl114 Grp 1 Standby router is local
113303: 2y30w: HSRP: Vl114 Grp 1 Speak -> Standby
113304: 2y30w: HSRP: Vl114 Grp 1 Redundancy "hsrp-Vl114-1" state Speak -> Standby

 

 

 

3750G-1#sh standby
Vlan112 - Group 1
State is Active
2 state changes, last state change 3y8w
Virtual IP address is 10.100.112.1
Active virtual MAC address is 0000.0c07.ac01
Local virtual MAC address is 0000.0c07.ac01 (v1 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 0.768 secs
Preemption enabled, delay min 3 secs
Active router is local
Standby router is 10.100.112.3, priority 100 (expires in 9.296 sec)
Priority 150 (configured 150)
Group name is "hsrp-Vl112-1" (default)
Vlan113 - Group 1
State is Active
1 state change, last state change 3y8w
Virtual IP address is 10.100.113.1
Active virtual MAC address is 0000.0c07.ac01
Local virtual MAC address is 0000.0c07.ac01 (v1 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 2.448 secs
Preemption enabled, delay min 3 secs
Active router is local
Standby router is 10.100.113.3, priority 100 (expires in 10.704 sec)
Priority 150 (configured 150)
Group name is "hsrp-Vl113-1" (default)
Vlan114 - Group 1
State is Active
1 state change, last state change 3y8w
Virtual IP address is 10.100.114.1
Active virtual MAC address is 0000.0c07.ac01
Local virtual MAC address is 0000.0c07.ac01 (v1 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 1.472 secs
Preemption enabled, delay min 3 secs
Active router is local
Standby router is 10.100.114.3, priority 100 (expires in 8.784 sec)
Priority 150 (configured 150)
Group name is "hsrp-Vl114-1" (default)

 


3750G-2#sh standby
Vlan112 - Group 1
State is Standby
1 state change, last state change 2y30w
Virtual IP address is 10.100.112.1
Active virtual MAC address is 0000.0c07.ac01
Local virtual MAC address is 0000.0c07.ac01 (v1 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 2.085 secs
Preemption disabled
Active router is 10.100.112.2, priority 150 (expires in 7.038 sec)
Standby router is local
Priority 100 (default 100)
IP redundancy name is "hsrp-Vl112-1" (default)
Vlan113 - Group 1
State is Standby
6546 state changes, last state change 3w5d
Virtual IP address is 10.100.113.1
Active virtual MAC address is 0000.0c07.ac01
Local virtual MAC address is 0000.0c07.ac01 (v1 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 2.085 secs
Preemption disabled
Active router is 10.100.113.2, priority 150 (expires in 8.414 sec)
Standby router is local
Priority 100 (default 100)
IP redundancy name is "hsrp-Vl113-1" (default)
Vlan114 - Group 1
State is Standby
150020 state changes, last state change 00:30:02
Virtual IP address is 10.100.114.1
Active virtual MAC address is 0000.0c07.ac01
Local virtual MAC address is 0000.0c07.ac01 (v1 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 2.077 secs
Preemption disabled
Active router is 10.100.114.2, priority 150 (expires in 7.399 sec)
Standby router is local
Priority 100 (default 100)
IP redundancy name is "hsrp-Vl114-1" (default)

Thank you for the info. Can you elaborate on what specific issue you are having? The changes/messages you got were a long time ago and even according to the output you just provided the last change was 2-3 years ago depending on the VLAN interface.

 

HSRP looks to be functioning and acting as normal. If you are having a specific issues can you please provide more detail.

 

There may be an issue with logging if you still have messages from years ago stuck in your log output and no new ones are being generated.

 

-David

not all VLAN face issue some have 
150020 state changes, last state change 00:30:02

We have a monitoring system losing ping to the first switch routinely even though it's not actually losing connectivity so I looked in the logs and though the output showed something wrong with HSRP.

Thank you @MHM Cisco World for pointing the change out as more recent. I missed that. 

 

@dbuckley77 - you said you have a monitoring system pinging the first switch. Does this mean you're not pinging the HSRP IP and you are pinging the actual SVI IP? If that's the case it may not be an HSRP issue, but a connectivity issue.

 

-David

First let me address the time stamp question. David says "it looks like it happened 2 years and 30 weeks ago" Actually that is not how long ago it happened but is how long this switch has been "up" when the event occurred. One of the options in logging messages is to use the up time as the time stamp and the usual option is to use current time. I believe that up time is the default if the device does not have authoritative time. I am puzzled at the mixture in these log messages of time stamps as up time and time stamp as current time. I don't know if that might point at something that would be significant.

Actually I just figured out the inconsistency. Check the configuration of timestamps

service timestamps debug uptime
service timestamps log datetime localtime

Some of these messages are generated by debug and are using uptime while other messages are generated by syslog.

I note that all of these messages refer to vlan 114. There are no messages about vlan 112 or 113. Is there something about vlan 114 that is different?

There is another inconsistency that I notice. There are messages like this

113257: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.231 not found

This address is not in the config that was posted. Where did it come from?

Recognizing that this message was generated by debug I guess the question is really where did debug get this from? It is still not consistent with the config.

I would like to focus on this group of messages

113287: 2y30w: HSRP: Vl114 Grp 1 Standby: c/Active timer expired (10.100.114.2)
113288: 2y30w: HSRP: Vl114 Grp 1 Active router is local, was 10.100.114.2
113289: 2y30w: HSRP: Vl114 Grp 1 Standby router is unknown, was local
113290: 2y30w: HSRP: Vl114 Grp 1 Standby -> Active

The first of these messages seems to indicate that there has been a normal/successful HSRP relationship between the switches and now switch 2 is no longe receiving HSRP messages from switch 1. The other 3 messages describe a typical failover where this switch (now the active member) is not in contact with the peer switch.

The message that follows them is a syslog message with a much more helpful timestamp

113291: May 12 12:29:03: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Standby -> Active

Skipping over a series of debug messages the next syslog message is this

113299: May 12 12:29:25: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Active -> Speak

This means that at 12:29:25 switch2 received a HSRP message from switch 1. Switch 1 with higher priority and preempt configured is taking over as the active router. And switch2 is going back to standby.

If we focus on the syslog messages we see that there was a login at May 12 12:22:39. The context suggests that HSRP was operating successfully with switch2 as standby.

At May 12 12:29:03 there was a failover based on not receiving messages from switch1. (note there are no log messages about any interface state changes).

At May 12 12:29:25 switch2 has received a HSRP message from switch1 and switch1 is resuming the role of active.

I would suggest that at this point switch2 appears to be normal and whatever issue there might be is to be found on switch1.

HTH

Rick

The issue has been found and was on switch one.  The issue was that we had changed the IP address of one of our servers but did not update the ACL in the switch.  I had thought it was an HSRP issue because I was unfamiliar with the entries int he log and they looked ominous so I consulted a Cisco doc that led me to believe that HSRP was the issue.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: