Cisco ISR4431 HSRP issues

[Yuvi1983]

Hi All, 

I have the following issue on the Production C4431 ISR routers 

System image file is "bootflash:/isr4400-universalk9.03.16.03.S.155-3.S3-ext.SPA.bin"

Any idea what caused the HSRP to enter into "Unknown" state please?

Thanks & Best Regards

Yuvi

[davisbolaosrojas]

Check this:

1. Layer 1 or Layer 2 Issues: Ensure that the physical connections and VLAN configurations are correct. Check for any issues with the interfaces or VLANs associated with the HSRP groups.

2. HSRP Configuration Errors: Verify that the HSRP configurations on all routers are correct and consistent. Ensure that the HSRP group numbers, versions, and virtual IP addresses match across all routers.

3. Multicast Issues: HSRP uses multicast to send Hello packets. Ensure that multicast traffic is not being blocked by ACLs or firewalls. Verify that the multicast address 224.0.0.2 is allowed through the network.

4. HSRP Feature Disabled: Make sure that the HSRP feature is enabled on all participating routers. Check the HSRP configuration on each interface to ensure it is properly set up.

5. Interface Issues: Ensure that the Interface associated with the HSRP groups are up and properly configured. Check the status of the Interfaces and ensure they are not in a shutdown state.

[M02@rt37]

Hello @Yuvi1983 

Have you got a diagram?

How are connected routers ? Both on a L2 Switch ? 

[Richard Burts]

Yuvi

Given that the HSRP is configured on subinterfaces of Po1, am I correct in assuming that the connection is through a switch? If so what does the switch say about its connections? The output of show cdp neighbor might be helpful.

Since HSRP is in the init state it suggests that there is a lack of connectivity. The output of show ip interface brief might shed light on this.

[M02@rt37]

Hello @Richard Burts 

Since HSRP is in the init state it suggests that there is a lack of connectivity __ we are agree.

A diagramm will help us in all cases. I'm running out of crystal balls (LOL).

Hope cdp is not disable...

[Richard Burts]

Yes we agree that there is likely a connectivity issue. While I agree that a diagram would help us to understand the environment, I think we need more than just a diagram. The diagram may show a connection between devices. But that does not necessarily mean that the connection is up and working.

[Yuvi1983]

Thank you, I do see the large number following message , What the issues please ? 

d tcp 6c9c.ed5c.2793 156.253.252.202(46501) -> 216.75.192.165(34567), 1 packet
Mar 10 12:54:39.335: %OSPF-4-INVALIDKEY: Key ID 0 received on interface Port-channel1.3006
Mar 10 12:54:42.917: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 195.82.147.195(56831) -> 216.75.192.169(3393), 1 packet
Mar 10 12:54:43.609: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 209.97.180.8(45793) -> 216.75.192.172(7349), 1 packet
Mar 10 12:54:44.542: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 18.190.163.148(44035) -> 131.153.39.141(7500), 1 packet
Mar 10 12:54:44.895: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied udp 6c9c.ed5c.2793 162.142.125.242(46232) -> 216.75.192.146(29340), 1 packet
Mar 10 12:54:47.814: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 204.76.203.15(43228) -> 216.75.192.168(80), 1 packet
Mar 10 12:54:50.126: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 47.19.205.170(6428) -> 216.75.192.165(443), 1 packet
Mar 10 12:54:54.155: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 167.94.138.157(44131) -> 216.75.192.168(3260), 1 packet
Mar 10 12:54:59.092: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 47.19.205.170(47527) -> 216.75.192.146(443), 1 packet
Mar 10 12:54:59.640: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 149.129.249.160(41984) -> 131.153.39.141(2222), 1 packet

[sasanka1912]

Hi @Yuvi1983  First of all ,in for HSRP to work, you should have the Layer 2 ring complete and related devices should be reachable. As above posts mentioned , diagram would be helpful in troubleshooting this .

1.First step would be , check and identify the connectivity issues between related devices .

2.Check layer 2 connectivity is there .(If you are using dedicated vlan for hsrp connectivity)

3.Once two related devices are reachable , you can look in to the configuration issues for hsrp such as standby group id/standby ip etc.

4.Suggest you take a structured approach from layer 1-->2  to troubleshoot these issues where you will be able to rectify those issues by yourself. 

5. Finally if devices are reachable and still having problems with HSRP , you may do some debugging such as debug standby as it will also give some important logs. 

Regards

CK

*** Please Rate All Helpful Responses ***

[Joshqun Ismayilov]

@Yuvi1983 

The HSRP "Unknown" state in your scenario is likely tied to the IOS XE software version (03.16.03.S) running on the ISR4431 router.
I think this IOS XE version is older (2018 release). Cisco has since patched many HSRP-related bugs in newer releases. You can upgrade IOS's

Thanks!

[Yuvi1983]

Upgrade done , Version 17.12.4a, RELEASE SOFTWARE (fc2), 

I still see the following message in the console 

Log Buffer (102400 bytes):
d tcp 6c9c.ed5c.2793 156.253.252.202(46501) -> 216.75.192.165(34567), 1 packet
Mar 10 12:54:39.335: %OSPF-4-INVALIDKEY: Key ID 0 received on interface Port-channel1.3006
Mar 10 12:54:42.917: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 195.82.147.195(56831) -> 216.75.192.169(3393), 1 packet
Mar 10 12:54:43.609: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 209.97.180.8(45793) -> 216.75.192.172(7349), 1 packet
Mar 10 12:54:44.542: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 18.190.163.148(44035) -> 131.153.39.141(7500), 1 packet
Mar 10 12:54:44.895: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied udp 6c9c.ed5c.2793 162.142.125.242(46232) -> 216.75.192.146(29340), 1 packet
Mar 10 12:54:47.814: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 204.76.203.15(43228) -> 216.75.192.168(80), 1 packet
Mar 10 12:54:50.126: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 47.19.205.170(6428) -> 216.75.192.165(443), 1 packet
Mar 10 12:54:54.155: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 167.94.138.157(44131) -> 216.75.192.168(3260), 1 packet
Mar 10 12:54:59.092: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 47.19.205.170(47527) -> 216.75.192.146(443), 1 packet
Mar 10 12:54:59.640: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 149.129.249.160(41984) -> 131.153.39.141(2222), 1 packet

What will be the reason please ? 

[Richard Burts · Accepted solution]

The log messages are interesting. But I do not believe that they relate to the question about HSRP state. Lets take a look at some of these log messages:

- "Mar 10 12:54:39.335: %OSPF-4-INVALIDKEY: Key ID 0 received on interface Port-channel1.3006" This demonstrates that Po1.3006 is receiving traffic and that it is attempting to negotiate OSPF neighborship but that the key is not valid. Note that this interface does not appear to be related to the original problem. 

- "Mar 10 12:54:42.917: %FMANFP-6-IPACCESSLOGP: F0/0: fman_fp_image: list OUTSIDE denied tcp 6c9c.ed5c.2793 195.82.147.195(56831) -> 216.75.192.169(3393), 1 packet". This indicates that an access list named OUTSIDE is denying incoming traffic. There are a lot of these messages. So there appears to be a lot of incoming traffic that is being denied. I find it interesting the variety of Public IP addresses in both source address and destination address. And I believe that these messages do not relate to the original problem with HSRP.

[Yuvi1983]

There was the configurations issues on the ACL with logging.

I disable the logging on ACL

[Richard Burts]

Thanks for the update, and for marking the discussion as solved.  This will help other members of the community to identify discussions that are helpful. This community is an excellent place to ask questions and to learn about networking. I hope to see you continue to be active in the community.

[davisbolaosrojas]

This line: 
Mar 10 12:54:39.335: %OSPF-4-INVALIDKEY: Key ID 0 received on interface Port-channel1.3006
It seems that there is a authentication mismatch on OSFP

This message indicates that an ACL (access control list) named "OUTSIDE" blocked a TCP packet from 195.82.147.195 to 216.75.192.169 on port 3393.

There are multiple similar lines with different IP addresses and blocked ports.

[sasanka1912]

@Yuvi1983 can you please run debug standby and check if you have any logs for that as above seems not related .

please paste the logs in the chat if you can gives better view of the issue.