OSPF neighbors down and up only en Catalyst 3650

[David Serarols]

Hello everybody,

We have a very strange issue in a Catalyst OSPF network. In this network we have 3560, 3750 and some 3650.

Until now, network without 3650 worked fine, but with new 3650 these log appeared:

Mar 16 21:52:03.994 CET: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.254.43 on GigabitEthernet1/1/3 from FULL to DOWN, Neighbor Down: Dead timer expired
Mar 16 21:52:04.073 CET: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.254.52 on GigabitEthernet1/1/2 from FULL to DOWN, Neighbor Down: Dead timer expired
Mar 16 21:52:04.142 CET: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.254.53 on GigabitEthernet1/1/1 from FULL to DOWN, Neighbor Down: Dead timer expired
Mar 16 21:52:04.915 CET: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.254.43 on GigabitEthernet1/1/3 from LOADING to FULL, Loading Done
Mar 16 21:52:04.915 CET: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.254.52 on GigabitEthernet1/1/2 from LOADING to FULL, Loading Done
Mar 16 21:52:04.915 CET: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.254.53 on GigabitEthernet1/1/1 from LOADING to FULL, Loading Done

This log appeared only related to 3650, and never in 3750 or 3560.

We verifyed MTU is the same in all Catalyst (1500).

I attach an extract of interface configuration:

interface GigabitEthernet1/2
 description L3 link AES-4.2 Ge0/2
 no switchport
 dampening
 ip address 192.168.252.218 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip pim query-interval 10
 ip pim sparse-mode
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 7 ***supressed
 ip ospf network point-to-point
 ip ospf dead-interval minimal hello-multiplier 4
 carrier-delay msec 0
 srr-queue bandwidth share 1 70 25 5
 priority-queue out 
 mls qos trust dscp
!

interface GigabitEthernet0/2
 description L3 link AES-4.1 Ge1/2
 no switchport
 dampening
 ip address 192.168.252.217 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip pim query-interval 10
 ip pim sparse-mode
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 7 ***supressed
 ip ospf network point-to-point
 ip ospf dead-interval minimal hello-multiplier 4
 carrier-delay msec 0
 srr-queue bandwidth share 1 70 25 5
 priority-queue out 
 mls qos trust dscp
!

3650 runs IOS XE and some commands are different and I am not familiar with. 

Anyone knows? Thank you in advance.

David

[Cisco Freak]

Hi David,

How frequent the OSPF neighborship is tearing down? Did it happen one time only or repeated?

CF

[David Serarols]

Hi,

It happens twice or three times per day.

Regards,

David

[Cisco Freak]

Did you check the CPU load on the device. Please share the below outputs:

sh processes cpu history

sh logging

sh processes cpu sorted | ex 0.00

sh memory

CF

[David Serarols]

Thank you very much for your comments.

In fact, switches are not still in production, so traffic amount or CPU would not be a problem.

I carry on investigating and I will post what I find.

Regards,

David

[Bilal Nawaz]

I experienced the exact same problem a couple of years ago with 3750x. OSPF neigborships flapping, it was due to CPU hitting maximum. Further investigation we found it was multicast traffic (some reasonable volumes) i changed the code to 12.2(55) it did improve. In the end this site expanded and put in C6K's and never saw that problem again.

[Martin van Loggerenberg]

Hi David,

I am having the same issue. Did you get this resolved?

Thanks

Martin

[David Serarols]

Hi Martin,

Sorry for not having updated my question. Fortunately we solved this issue.

Have you a fast hello command in the interface configuration?

ip pim query-interval 10
ip pim sparse-mode
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 7 xxxxxxxxxxxxxxxxxxxxx
ip ospf network point-to-point
ip ospf dead-interval minimal hello-multiplier 4

We discovered in these IOS you can not use "ip ospf dead-interval minimal hello-multiplier 4". Please, try without this command and tell us.

Regards,

David

[Vasilii Mikhailovskii]

Hello.

Just for anybody looking to the thread for resolution.

OSPF fast-hellos (sub-second) feature is not supported on 3650/3750[X]/3850 platforms (effective date Jan 2017).

The best timers are 1/3, that even may be affected by CSCuz98375.

CSCuz98375 first will be fixed in 3.7.5E

Please also note CSCvc44763 - first fixed in 3.6.7.

May not be available in 3.7.6 (due to EOL note).

Looks like Polaris (16.3.x and 16.6.x) is not affected.

[vamikhai]

removed