04-10-2015 02:16 PM - edited 03-07-2019 11:29 PM
I have a system where the Main location has multiple branches . All the branches have their own MetroEthernet circuit on a 3650 that comes back to a single interface at the Main location's 4500. This is why it is configured as a point-to multipoint. In addition to the PtoMp, we have to seperate public and staff traffic so we use VRF lite. The Metro E interface at each location is configured as a dot1q trunk port with a L3 SVI for each VRF instance. There is a seperate OSPF instance for each VRF. These have been running fine since we put them in last fall but about a month ago one branch started filling the logs with this:
%OSPF-5-ADJCHG: Process 2, Nbr 2.2.2.1 on Vlan2 from DOWN to DOWN, Neighbor Down: Ignore timer expired
%OSPF-5-ADJCHG: Process 2, Nbr 2.2.2.1 on Vlan2 from EXSTART to DOWN, Neighbor Down: Too many retransmissions
What's strange is that when I show OSPF neighbors for that VRF all the other branches show as having a Full adjacency except for Main . The EXSTART error in every bit of documentation I can find indicates a MTU error but the base OSPF instance isn't having this problem and it's on the same physical interface. I've looked for a bug and don't see any. I rebooted the switch in question and it didn't solve the problem. Today, another branch rebooted due to a power outage and now I'm getting this error on it as well.
Please help, not sure what's causing this.
Thanks ,
Bonnushka
GigabitEthernet1/0/1 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 58f3.9c0b.9e01 (bia 58f3.9c0b.9e01)
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
2.2.2.1 0 EXSTART/ - 00:01:59 2.2.2.1 Vlan2
2.2.2.3 0 FULL/ - 00:01:36 2.2.2.3 Vlan2
2.2.2.13 0 FULL/ - 00:01:34 2.2.2.13 Vlan2
2.2.2.4 0 FULL/ - 00:01:45 2.2.2.4 Vlan2
2.2.2.20 0 FULL/ - 00:01:39 2.2.2.20 Vlan2
1.1.1.1 0 FULL/ - 00:01:42 1.1.1.1 Vlan10
1.1.1.3 0 FULL/ - 00:01:54 1.1.1.3 Vlan10
1.1.1.13 0 FULL/ - 00:01:42 1.1.1.13 Vlan10
1.1.1.4 0 FULL/ - 00:01:54 1.1.1.4 Vlan10
1.1.1.16 0 FULL/ - 00:01:57 1.1.1.16 Vlan10
EX:
Main Location Router:
int vlan 10
ip add 1.1.1.1 255.255.255.0
ip ospf point-to-multipoint
int vlan 20
ip vrf forwarding Public
ip add 2.2.2.1 255.255.255.0
ip ospf point-to-multipoint
Branches:
interface GigabitEthernet1/0/1
switchport trunk allowed vlan 10,20
switchport mode trunk
speed 100
duplex full
int vlan 10
ip add 1.1.1.2(through 20) 255.255.255.0
ip ospf network point-to-multipoint
int vlan 20
ip vrf forwarding Public
ip add 2.2.2.2(through 20) 255.255.255.0
ip ospf network point-to-multipoint
04-10-2015 05:58 PM
Hi,
If it's allowed to run a debug on router, please try below steps on branch router:
In configuration mode:
- enable monitor logging at debug level
- set logging rate to 1
In global mode:
- command: debug ip ospf 2 events
- command: debug ip ospf 2 packets
After that, please post the output here.
Regards,
Minh
04-13-2015 08:06 AM
Apr 13 09:41:36: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.6 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:38: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:41:39: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.20 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:42: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:41:43: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.1 aid:0.0.0.0 chk:9318 aut:0 auk: from Vlan20
Apr 13 09:41:43: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.10 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:45: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.12 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:46: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.19 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:47: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:41:48: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.4 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:50: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.11 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:51: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.15 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:52: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:41:52: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.5 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:52: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.8 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:55: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.13 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:41:57: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:41:57: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.18 aid:0.0.0.0 chk:92D8 aut:0 auk: from Vlan20
Apr 13 09:42:00: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.2 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:01: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:42:03: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.21 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:03: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.255.1 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:04: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.7 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:06: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:42:06: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.20 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:11: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:42:12: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.10 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:12: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.1 aid:0.0.0.0 chk:9318 aut:0 auk: from Vlan20
Apr 13 09:42:14: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.19 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:15: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.12 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:15: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.235.1 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:16: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.4 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:16: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:42:18: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.11 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:20: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.5 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:20: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.15 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:21: OSPF-2 PAK : rcv. v:2 t:2 l:32 rid:2.2.2.1 aid:0.0.0.0 chk:7E6 aut:0 auk: from Vlan20
Apr 13 09:42:22: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.8 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
Apr 13 09:42:24: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.18 aid:0.0.0.0 chk:92D8 aut:0 auk: from Vlan20
Apr 13 09:42:25: OSPF-2 PAK : rcv. v:2 t:1 l:128 rid:2.2.2.13 aid:0.0.0.0 chk:9398 aut:0 auk: from Vlan20
04-13-2015 04:32 PM
Hi,
Thanks. Please post output of: debug ip ospf adj.
Regards,
Minh
04-14-2015 06:30 AM
It won't do any good now, after 3 weeks the stupid thing cleared up last night all on their own. I rebooted it for a second time yesterday morning but the OSPF flapping didn't clear up until almost 10 hours later. When the problem was occurring the router would should Full/- adjacency with all the branches but -/- with the main router.
04-17-2015 07:48 AM
copy the database show ip os database , and try compare when you have that behaviour again
04-11-2015 07:48 AM
Hi,
I had this type of behaviour before, using Layer 2 connection across providers and root cause was BPDUs (using different VID) of Spanning-tree.
So any error messages of Spanning-tree Inconsistent? Try use bpdu-filter on Main Router.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide