First let me address the time stamp question. David says "it looks like it happened 2 years and 30 weeks ago" Actually that is not how long ago it happened but is how long this switch has been "up" when the event occurred. One of the options in logging messages is to use the up time as the time stamp and the usual option is to use current time. I believe that up time is the default if the device does not have authoritative time. I am puzzled at the mixture in these log messages of time stamps as up time and time stamp as current time. I don't know if that might point at something that would be significant.
Actually I just figured out the inconsistency. Check the configuration of timestamps
service timestamps debug uptime
service timestamps log datetime localtime
Some of these messages are generated by debug and are using uptime while other messages are generated by syslog.
I note that all of these messages refer to vlan 114. There are no messages about vlan 112 or 113. Is there something about vlan 114 that is different?
There is another inconsistency that I notice. There are messages like this
113257: 2y30w: HSRP: Vl114 API active virtual address 10.100.114.231 not found
This address is not in the config that was posted. Where did it come from?
Recognizing that this message was generated by debug I guess the question is really where did debug get this from? It is still not consistent with the config.
I would like to focus on this group of messages
113287: 2y30w: HSRP: Vl114 Grp 1 Standby: c/Active timer expired (10.100.114.2)
113288: 2y30w: HSRP: Vl114 Grp 1 Active router is local, was 10.100.114.2
113289: 2y30w: HSRP: Vl114 Grp 1 Standby router is unknown, was local
113290: 2y30w: HSRP: Vl114 Grp 1 Standby -> Active
The first of these messages seems to indicate that there has been a normal/successful HSRP relationship between the switches and now switch 2 is no longe receiving HSRP messages from switch 1. The other 3 messages describe a typical failover where this switch (now the active member) is not in contact with the peer switch.
The message that follows them is a syslog message with a much more helpful timestamp
113291: May 12 12:29:03: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Standby -> Active
Skipping over a series of debug messages the next syslog message is this
113299: May 12 12:29:25: %HSRP-6-STATECHANGE: Vlan114 Grp 1 state Active -> Speak
This means that at 12:29:25 switch2 received a HSRP message from switch 1. Switch 1 with higher priority and preempt configured is taking over as the active router. And switch2 is going back to standby.
If we focus on the syslog messages we see that there was a login at May 12 12:22:39. The context suggests that HSRP was operating successfully with switch2 as standby.
At May 12 12:29:03 there was a failover based on not receiving messages from switch1. (note there are no log messages about any interface state changes).
At May 12 12:29:25 switch2 has received a HSRP message from switch1 and switch1 is resuming the role of active.
I would suggest that at this point switch2 appears to be normal and whatever issue there might be is to be found on switch1.
HTH
Rick
