08-26-2019 02:33 AM - edited 09-22-2019 02:41 AM
kindly I need your assistance on this. Having this error for the pass one month on our Distribution switch 6880x( Software (c6880x-ADVENTERPRISEK9-M), Version 15.4(1)SY1).
Note, CPU process is good, no stp instances, portchannel is up up. See attachment for detail
08-26-2019 02:51 AM
Hello,
go the switch shell:
take show standby
moving from Active --> Speak should mean a better HSRP router is heard and the local router gives up the active role.
Verify that an active router is known for each group even if it is not the local router.
No STP instances running .
all interfaces on the C68800 are routed interfaces ?
Hope to help
Giuseppe
08-26-2019 08:56 AM - edited 08-26-2019 11:43 PM
kindly see Attached.
And yes, to STP instances running and the sub PortChannel interfaces are routed.
08-26-2019 10:32 AM
Hello,
please attach as a text file the show commands or configurations
thanks.
Best Regards
Giuseppe
08-27-2019 01:40 AM
@Giuseppe Larosa Please check your inbox for the running-config. I had to do this because of the sensitivity of the info provided. Thanks in advance
08-27-2019 07:13 AM - edited 08-27-2019 07:19 AM
Hello ilenjohnx,
you are using very aggressive timers on multiple HSRP groups with MD5 authentication
Example:
standby 2009 timers msec 100 msec 300
standby 2009 priority 150
standby 2009 preempt delay minimum 60
standby 2009 authentication md5 key-chain hsrp50.2009
This can lead to HSRP change of states in presence of microbursts.
I would suggest to use less aggressive timers and to see if the network becomes more stable
something like:
standby 2009 timers msec 300 msec 950
Of course you should make these changes on both switches for all groups and then look at log messages to see if the network becomes more stable.
Edit:
you have also CoPP Control Plane policing enabled. It would be interesting to understand in which class are HSRP hello messages classified.
Hope to help
Giuseppe
08-29-2019 12:27 AM
Will try this and get back to you
09-02-2019 08:09 AM
@Giuseppe Larosa @balaji.bandi
So i have discovered that hsrp hello packets are policed on class-map: class-copp-mcast-v4-data-on-routedPort (match-any) as seen below. kindly give guide me through finding exactly with Multicast are being dropped?
Please, give me all the commands to use for my investigations.
c6880x# show policy-map control-plane
Control Plane Interface
Service-policy input: policy-default-autocopp
Hardware Counters:
class-map: class-copp-mcast-v4-data-on-routedPort (match-any)
Match: none
police :
10 pps 1 limit 1 extended limit
Earl in slot 5 :
401174895 packets
5 minute offered rate 20 pps
aggregate-forwarded 0 packets
action: drop
exceeded 401174895 packets action: drop
aggregate-forward 0 pps exceed 18 pps
09-03-2019 03:39 AM - edited 09-03-2019 03:40 AM
Hello IlenJohnx,
HSRP hello messages should be sent to the 224.0.0.2 all routers on subnet link local multicast address.
The configuration of the class-map suggests that it will drop if more the 10 multicast packets are received on an interface.
With your timers of 100 msec, 10 Hello packets are sent for each HSRP group.
Now, CoPP is applied on the internal link between the forwarding plane and the Main CPU to protect the CPU.
You can see the offered packet rate is 20 pps and not 10 pps.
You can solve this in two ways:
either you increase the HSRP hello timers to 300 msec so that you get 3 packets for each HSRP group per second.
Or you need to modify the class-map action so that the police-action happens at 30 pps instead of 10 pps.
However, the provided output show us that these packets are all dropped:
class-map: class-copp-mcast-v4-data-on-routedPort (match-any)
Match: none
police :
10 pps 1 limit 1 extended limit
Earl in slot 5 :
401174895 packets
5 minute offered rate 20 pps
aggregate-forwarded 0 packets
action: drop
exceeded 401174895 packets action: drop
aggregate-forward 0 pps exceed 18 pps
Questions:
is slot 5 the slot of the supervisor on your C6800 system ?
>> 401174895 packets
>> exceeded 401174895 packets action: drop
aggregate-forward 0 pps exceed 18 pps
All packets are dropped if these packets were Hello packets you should see standby unknown on all HSRP groups and no flapping of HSRP state = no state change.
So I am not sure that HSRP hello messages are falling in this class.
They may be dropped in part in another class of CoPP, but it is evident they are not all dropped.
Hope to help
Giuseppe
09-04-2019 02:19 AM
@Giuseppe Larosa as requested, the Supervisor is in Slot 5. See below
What is the relevant? Please share
switch#show module
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
5 20 6880-X-LE 16P SFP+ Multi-Rate (Active) C6880-X-LE-SUP SAL1848522Z
Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ------ ------------ ------------ -------
5 7c69.f6e7.ca57 to 7c69.f6e7.ca6f 1.2 15.1(02)SY01 15.4(1)SY1 Ok
Mod Sub-Module Model Serial Hw Status
---- --------------------------- ------------------ ----------- ------- -------
5 Policy Feature Card 4 C6880-X-LE-PFC SAL18453XD0 1.1 Ok
Mod Online Diag Status
---- -------------------
5 Pass
09-04-2019 03:42 AM
Hello,
I have asked what module was in slot 5 because it was listed in the output of the CoPP class.
As I have explained in my previous post all packets look like dropped in that class and this makes me to think that HSRP hello messages should not be classified / processed by this class.
However, it is possible that the HSRP hello messages are processed in another CoPP class and they can experience some drops (but not drops of all packets).
In other words if in show standby you don't see standby router unknown for all groups this means that not all HSRP packets are dropped.
On the other hand if all HSRP groups have Active local standby unknown then all HSRP messages are actully dropped.
In this case either you modify the CoPP policy to police at 30 pps instead of 10 pps or you increase the HSRP timers so that less HSRP hellos are sent per second.
If you like you can post show standby brief so that we can see what is happening.
However, I have explained what to check above.
Hope to help
Giuseppe
09-05-2019 06:34 AM
@Giuseppe Larosa - pls check your inbox too for full output
As requested,
switch# show standby brief
Interface Grp Pri P State Active Standby Virtual IP
Po50.x 100 x P Standby x.x.x.x local x.x.x.x
As it can be seen for all the group, all the Active standby have known IP
I will like to still investigate packet drops on a class map, CPU also
Kindly, share show commands to use for the this task. I want know the the type of packets drop in a class map and CPU, if necessary
I know about the show policy-map control-plane and show platform qos ip
09-27-2019 04:48 AM
Hello John,
I have examined the log files that you have sent me.
I provide here my answers :
The mentioned extended ACLs are not blocking HSRP messages:
HSRP hellos are sent to destination address 224.0.0.2 all routers on subnet. (or other link local multicast for HSRP v2).
As I have explained before you are using timers that are too much aggressive.
There is no real need to use an HSRP timer of 100 msec in each group.
Move to 300 msec HSRP hello timers and you will solve your issue.
Please be aware that I am providing just an advice, if you want more assistance you should buy professional services from cisco or a cisco gold partner.
Hope to help
Giuseppe
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide