%OSPF-5-ADJCHG Dead Timer Expired

edhunter · ‎08-08-2022

Hi,

I have 4 OSPF adj on one ISR Router with 2 separate linux boxes running quagga.

I am getting the following messages in log repeatedly

%OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from FULL to DOWN, Neighbor Down: Dead timer expired
%OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from LOADING to FULL, Loading Done
%OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from FULL to DOWN, Neighbor Down: Dead timer expired
%OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from LOADING to FULL, Loading Done
%OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from FULL to DOWN, Neighbor Down: Dead timer expired
%OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from LOADING to FULL, Loading Done
%OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from FULL to DOWN, Neighbor Down: Dead timer expired

Here is config, show ip ospf interface output and ospf debug

interface Tunnel11
 ip address 192.168.228.2 255.255.255.252
 ip mtu 1402
 ip ospf message-digest-key 2 md5 
 ip ospf hello-interval 1
 ip ospf cost 50
 tunnel source 172.24.0.242
 tunnel destination X.X.X.X
 tunnel ttl 64
!
interface Tunnel12
 ip address 192.168.228.6 255.255.255.252
 ip mtu 1402
 ip ospf message-digest-key 2 md5 
 ip ospf hello-interval 2
 ip ospf cost 150
 tunnel source 172.24.0.242
 tunnel destination Y.Y.Y.Y
 tunnel ttl 64
!
interface Tunnel13
 ip address 192.168.228.10 255.255.255.252
 ip mtu 1402
 ip ospf message-digest-key 2 md5 
 ip ospf hello-interval 2
 ip ospf cost 60
 tunnel source 172.24.0.234
 tunnel destination X.X.X.X
 tunnel ttl 64
!
interface Tunnel14
 ip address 192.168.228.14 255.255.255.252
 ip mtu 1402
 ip ospf message-digest-key 2 md5 
 ip ospf hello-interval 2
 ip ospf cost 160
 tunnel source 172.24.0.234
 tunnel destination Y.Y.Y.Y
 tunnel ttl 64

router ospf 1
 router-id 210.0.0.0
 area 0.0.0.0 authentication message-digest
 area 0.0.0.0 range 192.168.64.0 255.255.192.0 cost 65535
 area 0.0.0.0 range 192.168.128.0 255.255.128.0 cost 65535


 no passive-interface Tunnel11
 no passive-interface Tunnel12
 no passive-interface Tunnel13
 no passive-interface Tunnel14

ISR show ip ospf interface output

Tunnel14 is up, line protocol is up 
  Internet Address 192.168.228.14/30, Area 0.0.0.0, Attached via Network Statement
  Process ID 1, Router ID 210.0.0.0, Network Type POINT_TO_POINT, Cost: 160
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           160       no          no            Base
  Transmit Delay is 1 sec, State POINT_TO_POINT
  Timer intervals configured, Hello 2, Dead 8, Wait 8, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:00
  Supports Link-local Signaling (LLS)
  Cisco NSF helper support enabled
  IETF NSF helper support enabled
  Index 6/7, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 1, maximum is 2
  Last flood scan time is 0 msec, maximum is 4 msec
  Neighbor Count is 1, Adjacent neighbor count is 0 
  Suppress hello for 0 neighbor(s)
  Message digest authentication enabled
    Youngest key id is 2
Tunnel13 is up, line protocol is up 
  Internet Address 192.168.228.10/30, Area 0.0.0.0, Attached via Network Statement
  Process ID 1, Router ID 210.0.0.0, Network Type POINT_TO_POINT, Cost: 60
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           60        no          no            Base
  Transmit Delay is 1 sec, State POINT_TO_POINT
  Timer intervals configured, Hello 2, Dead 8, Wait 8, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:01
  Supports Link-local Signaling (LLS)
  Cisco NSF helper support enabled
  IETF NSF helper support enabled
  Index 5/6, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 1, maximum is 19
  Last flood scan time is 0 msec, maximum is 4 msec
  Neighbor Count is 1, Adjacent neighbor count is 0 
  Suppress hello for 0 neighbor(s)
  Message digest authentication enabled
    Youngest key id is 2
Tunnel12 is up, line protocol is up 
  Internet Address 192.168.228.6/30, Area 0.0.0.0, Attached via Network Statement
  Process ID 1, Router ID 210.0.0.0, Network Type POINT_TO_POINT, Cost: 150
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           150       no          no            Base
  Transmit Delay is 1 sec, State POINT_TO_POINT
  Timer intervals configured, Hello 2, Dead 8, Wait 8, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:01
  Supports Link-local Signaling (LLS)
  Cisco NSF helper support enabled
  IETF NSF helper support enabled
  Index 4/5, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 1, maximum is 19
  Last flood scan time is 0 msec, maximum is 4 msec
  Neighbor Count is 1, Adjacent neighbor count is 1 
    Adjacent with neighbor 241.0.0.0
  Suppress hello for 0 neighbor(s)
  Message digest authentication enabled
    Youngest key id is 2
Tunnel11 is up, line protocol is up 
  Internet Address 192.168.228.2/30, Area 0.0.0.0, Attached via Network Statement
  Process ID 1, Router ID 210.0.0.0, Network Type POINT_TO_POINT, Cost: 50
  Topology-MTID    Cost    Disabled    Shutdown      Topology Name
        0           50        no          no            Base
  Transmit Delay is 1 sec, State POINT_TO_POINT
  Timer intervals configured, Hello 1, Dead 4, Wait 4, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:00
  Supports Link-local Signaling (LLS)
  Cisco NSF helper support enabled
  IETF NSF helper support enabled
  Index 3/4, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 1, maximum is 25
  Last flood scan time is 0 msec, maximum is 4 msec
  Neighbor Count is 1, Adjacent neighbor count is 1 
    Adjacent with neighbor 242.0.0.0
  Suppress hello for 0 neighbor(s)
  Message digest authentication enabled
    Youngest key id is 2

Quagga show ip ospf interface output

tun11 is up
  ifindex 57, MTU 1402 bytes, BW 0 Kbit <UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>
  Internet Address 192.168.228.1/32, Peer 192.168.228.2, Area 0.0.0.0
  MTU mismatch detection:enabled
  Router ID 242.0.0.0, Network Type POINTOPOINT, Cost: 50
  Transmit Delay is 1 sec, State Point-To-Point, Priority 1
  No designated router on this network
  No backup designated router on this network
  Multicast group memberships: OSPFAllRouters
  Timer intervals configured, Hello 1s, Dead 4s, Wait 4s, Retransmit 5
    Hello due in 0.711s
  Neighbor Count is 1, Adjacent neighbor count is 1
tun13 is up
  ifindex 58, MTU 1402 bytes, BW 0 Kbit <UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>
  Internet Address 192.168.228.9/32, Peer 192.168.228.10, Area 0.0.0.0
  MTU mismatch detection:enabled
  Router ID 242.0.0.0, Network Type POINTOPOINT, Cost: 60
  Transmit Delay is 1 sec, State Point-To-Point, Priority 1
  No designated router on this network
  No backup designated router on this network
  Multicast group memberships: OSPFAllRouters
  Timer intervals configured, Hello 2s, Dead 8s, Wait 8s, Retransmit 5
    Hello due in 0.233s
  Neighbor Count is 0, Adjacent neighbor count is 0






tun12 is up
  ifindex 57, MTU 1402 bytes, BW 0 Kbit <UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>
  Internet Address 192.168.228.5/32, Peer 192.168.228.6, Area 0.0.0.0
  MTU mismatch detection:enabled
  Router ID 241.0.0.0, Network Type POINTOPOINT, Cost: 150
  Transmit Delay is 1 sec, State Point-To-Point, Priority 1
  No designated router on this network
  No backup designated router on this network
  Multicast group memberships: OSPFAllRouters
  Timer intervals configured, Hello 2s, Dead 8s, Wait 8s, Retransmit 5
    Hello due in 0.178s
  Neighbor Count is 1, Adjacent neighbor count is 1
tun14 is up
  ifindex 58, MTU 1402 bytes, BW 0 Kbit <UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>
  Internet Address 192.168.228.13/32, Peer 192.168.228.14, Area 0.0.0.0
  MTU mismatch detection:enabled
  Router ID 241.0.0.0, Network Type POINTOPOINT, Cost: 160
  Transmit Delay is 1 sec, State Point-To-Point, Priority 1
  No designated router on this network
  No backup designated router on this network
  Multicast group memberships: OSPFAllRouters
  Timer intervals configured, Hello 2s, Dead 8s, Wait 8s, Retransmit 5
    Hello due in 0.178s
  Neighbor Count is 0, Adjacent neighbor count is 0

OSPF debug

Aug  8 08:17:33.820: OSPF-1 ADJ   Tu13: Send with youngest Key 2
Aug  8 08:17:34.104: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:34.492: OSPF-1 ADJ   Tu14: Send with youngest Key 2
Aug  8 08:17:34.600: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:34.800: OSPF-1 ADJ   Tu12: Send with youngest Key 2
Aug  8 08:17:35.060: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:35.520: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:35.816: OSPF-1 ADJ   Tu13: Send with youngest Key 2
Aug  8 08:17:35.976: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:36.288: OSPF-1 ADJ   Tu11: 242.0.0.0 address 192.168.228.1 is dead
Aug  8 08:17:36.288: OSPF-1 ADJ   Tu11: 242.0.0.0 address 192.168.228.1 is dead, state DOWN
Aug  8 08:17:36.288: %OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from FULL to DOWN, Neighbor Down: Dead timer expired
Aug  8 08:17:36.288: OSPF-1 ADJ   Tu11: 2 Way Communication to 242.0.0.0, state 2WAY
Aug  8 08:17:36.288: OSPF-1 ADJ   Tu11: Nbr 242.0.0.0: Prepare dbase exchange
Aug  8 08:17:36.288: OSPF-1 ADJ   Tu11: Send DBD to 242.0.0.0 seq 0x1FCC opt 0x52 flag 0x7 len 32
Aug  8 08:17:36.288: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:36.288: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:36.360: OSPF-1 ADJ   Tu11: Rcv DBD from 242.0.0.0 seq 0x62F0C2C5 opt 0x2 flag 0x7 len 32  mtu 1402 state EXSTART
Aug  8 08:17:36.360: OSPF-1 ADJ   Tu11: NBR Negotiation Done. We are the SLAVE
Aug  8 08:17:36.360: OSPF-1 ADJ   Tu11: Nbr 242.0.0.0: Summary list built, size 62
Aug  8 08:17:36.360: OSPF-1 ADJ   Tu11: Send DBD to 242.0.0.0 seq 0x62F0C2C5 opt 0x52 flag 0x2 len 1272
Aug  8 08:17:36.360: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:36.360: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:36.484: OSPF-1 ADJ   Tu14: Send with youngest Key 2
Aug  8 08:17:36.504: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:36.624: OSPF-1 ADJ   Tu12: Send with youngest Key 2
Aug  8 08:17:36.788: OSPF-1 ADJ   Tu12: Send with youngest Key 2
Aug  8 08:17:36.788: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:36.856: OSPF-1 ADJ   Tu11: Rcv LS UPD from 242.0.0.0 length 136 LSA count 1
Aug  8 08:17:36.920: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:37.464: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:37.816: OSPF-1 ADJ   Tu13: Send with youngest Key 2
Aug  8 08:17:37.852: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:38.408: OSPF-1 ADJ   Tu14: Send with youngest Key 2
Aug  8 08:17:38.460: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:38.616: OSPF-1 ADJ   Tu12: Send with youngest Key 2
Aug  8 08:17:38.844: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:38.860: OSPF-1 ADJ   Tu12: Send with youngest Key 2
Aug  8 08:17:39.432: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:39.772: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:39.812: OSPF-1 ADJ   Tu13: Send with youngest Key 2
Aug  8 08:17:40.292: OSPF-1 ADJ   Tu14: Send with youngest Key 2
Aug  8 08:17:40.412: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:40.608: OSPF-1 ADJ   Tu12: Send with youngest Key 2
Aug  8 08:17:40.744: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:41.104: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:41.360: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:41.364: OSPF-1 ADJ   Tu11: Rcv DBD from 242.0.0.0 seq 0x62F0C2C6 opt 0x2 flag 0x1 len 52  mtu 1402 state EXCHANGE
Aug  8 08:17:41.364: OSPF-1 ADJ   Tu11: Exchange Done with 242.0.0.0
Aug  8 08:17:41.364: OSPF-1 ADJ   Tu11: Synchronized with 242.0.0.0, state FULL
Aug  8 08:17:41.364: %OSPF-5-ADJCHG: Process 1, Nbr 242.0.0.0 on Tunnel11 from LOADING to FULL, Loading Done
Aug  8 08:17:41.364: OSPF-1 ADJ   Tu11: Send DBD to 242.0.0.0 seq 0x62F0C2C6 opt 0x52 flag 0x0 len 32
Aug  8 08:17:41.364: OSPF-1 ADJ   Tu11: Send with youngest Key 2
Aug  8 08:17:41.396: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:41.404: OSPF-1 ADJ   Gi2/0: Send with youngest Key 3
Aug  8 08:17:41.436: OSPF-1 ADJ   Tu11: Send with youngest Key 2

Since timers are configured correctly on both sides, what seems to be the problem? Where else should i look? Do i need to increase timers? Timers were intentionally lowered for convergence but if i need to i can increase them.

Thank you!

Meddane · ‎08-11-2022

Make sure the timers are the same in the peer connected to tunnel 11 interface.

edhunter · ‎08-11-2022

They are the same. Hence why i am asking

  Timer intervals configured, Hello 1s, Dead 4s, Wait 4s, Retransmit 5

Timer intervals configured, Hello 1, Dead 4, Wait 4, Retransmit 5

MHM Cisco World · ‎08-11-2022

OK,
you config multi P2P OSPF interface but same tunnel source, the OSPF keeplive receive in source interface for which tunnel,
so you need I think tunnel-Key
tunnel-key used to make source interface send packet to exact tunnel.

Joseph W. Doherty · ‎08-11-2022

Only tunnel11? If so, I notice it appears to be the only tunnel using a hello interval of 1 while all the others are using a hello interval of 2. Have you tried also using a hello interval of 2 for it, to see if that makes a difference?

How "busy" is tunnel11's physical interfaces, on both sides? Either side using QoS on physical interfaces to insure priority dequeuing for OSPF hello packets?

edhunter · ‎08-11-2022

I do get for tunnel 12 too (the two active ones, i have slas that failover to the other 2 13,14) but less frequently.

Interface is pretty busy, what should i get as a metric? They are my main site-to-site VPNs so they are quite busy.

No QoS configured on any interfaces, either side.

paul driver · ‎08-11-2022

Hello

Try not using authentication at all and then if ospf becomes stable use interface authentication instead of area, also suggest append ospf to the interface instead of summarizing the networks, Lasty what does the debug ip ospf adjacency show?

router ospf 1
log-adjacency-changes
no area 0.0.0.0 authentication message-digest
no area 0.0.0.0 range 192.168.64.0 255.255.192.0 cost 65535
no area 0.0.0.0 range 192.168.128.0 255.255.128.0 cost 65535

interface ran Tunnel11-14
ip ospf 1 area 0
ip ospf authentication message-digest
ip ospf message-digest-key 2 md5 <password>

debug ip ospf adjacency

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul