OSPF Neighbour Down

[asadgulzar90]

I have configured OSPF and GRE Tunnel 50 in HO & branch routers. All routers were working fine but suddenly from last month, one branch router starts giving below error and i lost connectivity. 

%OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from FULL to DOWN, Neighbor Down: Dead timer expired .

then i have to shut reshut int vlan 50 & physical port to amke link up again. Problem comes back again after few hours or a day.

[balaji.bandi]

Since you have an OSPF interface configured on Tunnel, i suspect when the tunnel went down, the OSPF process terminated with a neighbour. investigate with Tunnel status and see any abnormal in the logs? both the side, since your logs only from one side.

[Georg Pauwen]

Hello,

is this a DMVPN setup ? What kind of tunnel do you have configured, just a GRE tunnel, or an IPSec/GRE tunnel ? And what interface do you need to bounce ? Is the corresponding Internet link going down as well ? Is this branch configured like all other branches, and is the hardware and IOS identical to what you have at the other branches ?

In short, we need (a whole lot) more information...

[asadgulzar90]

yes it is dmvpn tunnel with ipsec/gre. int vlan 50 is configured for tunnel 50. all branches are configured same way. this branch was also configured 9 months ago, but problems started almost a month ago

[pigallo]

Your message is not tunnel related issue but control-plane related issue.
OSPF dead timer expiration caused adjacency fault in your case. You have to understand why that happened.
This doesn't mean it's necessary a tunnel problem as in GRE tunneling is stateless. It means GRE doesn't test connectivity to bring up the tunnel. As long you have valid route in the rib for destination, tunnel goes always up unless you configure keepalives.
If in your HO you have kind of hub /spoke setup (P2MP on hub and P2P on spokes) and you use single tunnel to connect to all branch sites, then ,whatever the issue was, all sites would probably be impacted if the problem was caused by hub site.
As in your case the issue is related to a single branch i would first investigate remotely on that branch to see what's happening on the network:
For example, if you have latency on that link which is caused by peak of traffic which impacts the tunneled GRE ospf packets that would be a problem. It could be a QOS issue too if control-plane traffic is not well prioritized.
Or probably you have set too aggressive timer for that link which should be adjusted.

Regards

[asadgulzar90]

i did not find any delay in traffic, also please note that sometimes it goes off and come up again automatically .
Few minutes ago, it went down multiple times and came up by itself.. below are the logs from branch router:

Oct 27 12:45:46: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1, changed state to down
Oct 27 12:45:47: %LINK-3-UPDOWN: Interface FastEthernet1, changed state to up
Oct 27 12:45:48: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1, changed state to up
Oct 27 12:46:24: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from FULL to DOWN, Neighbor Down: Dead timer expired
Oct 27 12:46:30: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from LOADING to FULL, Loading Done
Oct 27 13:06:14: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1, changed state to down
Oct 27 13:06:15: %LINK-3-UPDOWN: Interface FastEthernet1, changed state to up
Oct 27 13:06:16: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1, changed state to up
Oct 27 13:06:48: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from FULL to DOWN, Neighbor Down: Dead timer expired
Oct 27 13:06:52: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from LOADING to FULL, Loading Done
Oct 27 13:42:01: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from FULL to DOWN, Neighbor Down: Dead timer expired
Oct 27 16:22:48: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from LOADING to FULL, Loading Done
Oct 27 17:14:50: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from FULL to DOWN, Neighbor Down: Dead timer expired
Oct 27 17:16:31: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from LOADING to FULL, Loading Done
Oct 27 17:17:48: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from FULL to DOWN, Neighbor Down: Dead timer expired
Oct 27 17:20:13: %OSPF-5-ADJCHG: Process 3, Nbr 172.16.50.2 on Tunnel50 from LOADING to FULL, Loading Done

[balaji.bandi]

Since we are not aware of your setup you need to give us more information, and some topology, looking at the logs as mentioned. some of the interface gone down due to some reason, OSPF always relays on your underlay infrastructure to form and neighbourship.

can we know more information, how your network connected

Do you see some information here this interface gone down the same time? what is the interface? and where it is connected?

Oct 27 12:45:46: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1, changed state to down

[asadgulzar90]

Each branch has 2 routers, one is connected to Mpls link and other is connected to Internet link.

This Router at branch is directly connected to MPLS link at port Fastethernet1, then it is connected to LAN Switch.
Internet router is connected to Firewall then from firewall it is connected to same LAN switch.

172.16.50.2 is the LAN port IP of HO router. 

[Georg Pauwen]

Hello,

is this a phase 2 or phase 3 DMVPN ? Is your OSPF configured as 'broadcast' network type?

[asadgulzar90]

yes it is DMVPN 3 and ospf is set to broadcast

[paul driver]

Hello

Can you please confirm the MTU values on the tunnel interfaces and also initiate a debug of the ospf adjacency and post the output.
sh ip interface tunnel x | in MTU

debug ip ospf adj

[asadgulzar90]

Dear All, Thanks for your replies, I have changed the router port 2 days ago, and it is working fine since then,, seemed there was some hard ware issue.

[balaji.bandi]

just curious to know - is this one? interface FastEthernet1 was going up and down?