Re: eigrp and GRE tunnels

rcohenbci · ‎09-06-2013

I've got 2 3750x with IP Services

switch-A(GRE Tunnel delay 120)---------bunch of stuff----------(GRE Tunnel delay 120)switch-B

On the same switches

switch-A(vlan interface)---------------ethernet------------------(vlan interface)switch-B

One eigrp process on each switch same AS

The vlan and gre have unique /30 subnets and these are included in the eigrp process (along with other subnets local to each switch)

sho eigrp top shows that both paths are seen.

Everything works when both connections are up, and when only the vlan/ethernet connection is up, but if I lose the vlan connection it doesn't redirect over the gre, which is of course the whole reason for the gre being there. Routes are lost, eigrp doesn't see the neighbor. When its working , the topology showing both paths "should" mean that eigrp is going over the tunnel, and the tunnel does pass traffic.

Any ideas are more than welcome. This shouldn't be a problem but I just can't see what's wrong.

router eigrp 42

network <tunnel subnet>

network <vlan subnet>

network <local unique subnet>

eigrp router-id <local unique int IP> (added this when it was suggested it could be the problem, didn't change anything)

amabdelh · ‎09-06-2013

Randall

can you post the show ip route eigrp from both switches? I would like to see both routes in the table

rcohenbci · ‎09-07-2013

The eigrp topology shows two paths (potential routes). Only one route is selected by eigrp, and that one is shown and labeled as an eigrp route.

Additional info: show eigrp events says duplicate router ID and ignores the secondary path. This makes sense to me since it's getting eigrp on two connections from the same router, and the router IDs are definitely different. I would expect this would not happen if there were additional routers in between.

ALIAOF_ · ‎09-07-2013

So you have the same VLAN on both switches like:

Switch A: VLAN 10 lets say with an IP of 192.168.1.0/24

Switch B: VLAN 10 192.168.1.0/24

???

Peter Koltl · ‎09-07-2013

Is the tunnel destination by any chance routed via the VLAN interface?

Post your config and routing table.

rcohenbci · ‎09-07-2013

Switch-A

VLAN 1 192.168.1.0/24 with 6 access ports assigned.

VLAN 2 129.168.2.0/24 with 6 access ports assigned.

Both VLAN interface addresses are .1 in their subnet.

VLAN 10 10.10.10.1/30 a single access port which connects to switch B over a QnQ Ethernet circuit.

VLAN 20 publicIP single access port connected to firewall and then Internet.

Tunnel-1 10.10.20.1/30 connected to switch B across the Internet.

Switch-B

VLAN 1 192.168.11.0/24 six access ports assigned.

VLAN 2 192.168.12.0/24 six access ports assigned.

VLAN 10 10.10.10.2/30 One access port, QnQ to switch A.

VLAN 30 publicIP single access port to firewall then to Internet.

Tunnel-1 10.10.20.2/30 connected to switch A across Internet.

I'm posting from my iPhone so the actual config will have to wait.

I've done eigrp countless times but usually with more routers involved and more routers between potential paths. Could this have something to do with there being only two routers?

dslice · ‎09-08-2013

Without routing and topology table output, it's going to be darn difficult to help you. With the Vlan interface down, what is the expected path between tunnel endpoints?

Sent from Cisco Technical Support iPad App

rcohenbci · ‎09-09-2013

Below is the releavant config and output for "sho ip eigrp top". The "sho ip rout" on both switches shows everything correctly when its working, with both paths up. There are no duplicated routes and wether it's L C or D is correct. When the primary path goes down only the L and C routes are shown and only the local networks are shown in the topology.

You'll have to take some of this on faith since I'm not going to post anything that hasn't been sanatized for public consumption. For now let's assume that after 25 years I've got some idea what I'm doing and I've had at least 6 others look at the configs and haven't seen anything they'd call 'wrong"

"With the Vlan interface down, what is the expected path between tunnel endpoints?" On sw-A the tunnel goes over vlan20 which has a public IP, is assigned to only one access port, and connects to a firewall running in router mode. The tunnel then travels accross the Internet finally going through another firewall, also in router mode, and then to the single access port assigned to vlan30 on sw-B which has a piblic IP.

Switch-A

router eigrp 42

network 10.10.10.0 0.0.0.3

network 10.10.20.0 0.0.0.3

network 192.168.1.0 0.0.0.255

network 192.168.2.0 0.0.0.255

network 222.222.222.192 0.0.0.63

eigrp router-id 222.222.222.193

EIGRP-IPv4 Topology Table for AS(42)/ID(222.222.222.193)

Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,

r - reply Status, s - sia Status

P 222.222.222.192/26, 1 successors, FD is 2816

via Connected, Vlan30

P 111.111.111.192/26, 1 successors, FD is 3072

via 10.10.10.1 (3072/2816), Vlan10

via 10.10.20.1 (286976/2816), Tunnel1

P 192.168.11.0/24, 1 successors, FD is 3072

via 10.10.10.2 (3072/2816), Vlan10

via 10.10.20.2 (286976/2816), Tunnel1

P 192.168.12.0/24, 1 successors, FD is 3072

via 10.10.10.2 (3072/2816), Vlan10

via 10.10.20.2 (286976/2816), Tunnel1

P 10.10.20.0/30, 1 successors, FD is 286720

via Connected, Tunnel1

P 10.10.10.0/30, 1 successors, FD is 2816

via Connected, Vlan10

P 192.168.1.0/24, 1 successors, FD is 2816

via Connected, Vlan1

P 192.168.2.0/24, 1 successors, FD is 2816

via Connected, Vlan2

Switch-B

router eigrp 42

network 10.10.10.0 0.0.0.3

network 10.10.20.0 0.0.0.3

network 192.168.11.0 0.0.0.255

network 192.168.12.0 0.0.0.255

network 111.111.111.192 0.0.0.63

eigrp router-id 111.111.111.193

EIGRP-IPv4 Topology Table for AS(42)/ID(111.111.111.193)

Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,

r - reply Status, s - sia Status

P 111.111.111.192/26, 1 successors, FD is 2816

via Connected, Vlan30

P 222.222.222.192/26, 1 successors, FD is 3072

via 10.10.10.1 (3072/2816), Vlan10

via 10.10.20.1 (286976/2816), Tunnel1

P 192.168.1.0/24, 1 successors, FD is 3072

via 10.10.10.1 (3072/2816), Vlan10

via 10.10.20.1 (286976/2816), Tunnel1

P 192.168.2.0/24, 1 successors, FD is 3072

via 10.10.10.1 (3072/2816), Vlan10

via 10.10.20.1 (286976/2816), Tunnel1

P 10.10.20.0/30, 1 successors, FD is 286720

via Connected, Tunnel1

P 10.10.10.0/30, 1 successors, FD is 2816

via Connected, Vlan10

P 192.168.11.0/24, 1 successors, FD is 2816

via Connected, Vlan1

P 192.168.12.0/24, 1 successors, FD is 2816

via Connected, Vlan2

dslice · ‎09-09-2013

This still doen't provide any output pertaining to the routing for the tunnel endpoints with the vlan down. I'm guessing that with the vlan down, the tunnel itself is down and EIGRP is unable to form peers. Can you ping the tunnel source and destination on both switches with the VLAN down?

BTW, please don't feel insulted when folks ask for more data in order to make a guess. My ability to give you a good answer is directly based on my understanding of the situation, and without real data it's impossible for me to know what is going on. I appreciate your 25 years of experience and am not trying to disparage your capabilities. I just don't know how to troubleshoot without information.

(And just so you know, I've been troubleshooting networks now for 35 years and have spent 18 years at Cisco including 11 years as an EIGRP developer. My questions are not intended to be insulting or frivolous.)

rcohenbci · ‎09-09-2013

I'm really glad to hear about how long you've been doing this, finally someone you knows the difference between an interface and a port (despite how they are using interchangably now days).

Since all interfaces on these switches are vlans, I'm assuming you're reffering to vlan10 which connect to either end of the carrier provided QnQ circuit. Whenever a problem has occured vlan10 stays up but no traffic is passed (carrier issue someplace), and eigrp look just as it would if was receiving nothing. BUT, the gre is still up and passing traffic. I've confirmed this with both a ping, as well as, traffic that always goes over the gre (directed via route-map) continues uninterupted.

At first I thought eigrp wasn't going over the gre tunnel, and the results certainly look like this could be the problem, but the topology does show two potential paths;

P 192.168.1.0/24, 1 successors, FD is 3072

via 10.10.10.1 (3072/2816), Vlan10

via 10.10.20.1 (286976/2816), Tunnel1

Is there a better, more accurate, way to determine if eigrp is going over the gre? I know when the primary path is out of service I lose all my eigrp, so again it looks like eigrp isn't going over the gre....but the topology says....

BTW I'm absolutly certain the gre is up, I was able to add static routes when the primary path was down and all the traffic went over the gre just fine.

Hmmm...does the public interface the gre goes over need to be in the eigrp config as well? If so I'd have to make sure that both ends only allowed/sent eigrp to the other end and not to the internet in general. What do yoiu think?

dslice · ‎09-09-2013

When VLAN 10 goes down or is not passing traffic, do you see the peer go down through VLAN 10 and the peer stay up through the tunnel interface?

As for how to tell if EIGRP going across the tunnel, you can do "debug eigrp packet" (or some subset of EIGRP packets, such as HELLOs, etc.) to see if communications are continuing properly with VLAN10 down. Also my question above about the peer relationship will tell you if at least the multicast hello packets are being exchanged.

As for your last question about the public interface needing to be in EIGRP, the answer should be no. Whatever technology is used to advertise the tunnel endpoints so the tunnel can be established is independent of the EIGRP peer relationship and routing updates sent across the tunnel.

I also realized that we may be having a communications problem ourselves. Just for clarity, when I say tunnel endpoints, I'm not talking about the IP addresses assigned to the tunnel interface. The tunnel itself has a source and destination address for the tunnel creation, then the defined tunnel IP addresses are applied to the tunnel interface created and used for EIGRP to form peers and send updates.

Peter Koltl · ‎09-09-2013

Please post some EIGRP debugs taken when primary path goes down

I would check the GRE transfer for large packets too, there might be an MTU issue when HELLOs get through but large packets not. Please post your tunnel interface config. What is the achievable MTU via the path? Do you also have IPsec encapsulation involved?

You mention a route-map that directs traffic to tunnel. Let's see that config part too.

I'm wondering whether we should boycott further assisting without seeing the anonymized config and the routing table and the logs.

rcohenbci · ‎09-13-2013

I'm working on getting the debugs with the primary path down, not that easy since it's in constant use and it's not really a situation where I can just take it down.

Peter, no one has any obligation to help anyone else. This is a community site and personally I feel the very idea of "boycott"ing anyone for any reason, other than being verbally abusive, is contrary to the community idea. Please note that this is what "I feel", I never pretend to know what others are thinking or their intent. Don't let the small numbers of my posts on this site fool you, I've been helping others (peers and new) since before there was a web, anyone remember FidoNet.

Anonymizing an 800+ line config while being sure it's still consistent isn't fun. I'm trying to post everything relavent without causing confusion, or jepordizing security. And I think I'm doing pretty good given that 90% or more of the config is irrelavant, still I'll be posting more.

I think Donald is right, in that I really can't confirm that there are eigrp HELLOs getting through on the secondary when the primary is down, although I can't see why this would happen, I also can't prove it's not happening. I'm assuming the HELLOs are getting through on the secondary when the primary is up only because the topology shows two paths. Is it possible that it's learning of the second path via the eigrp exchange on the primary path, I would think not, but it would explain what I'm seeing.

I kind of dought there is an MTU issue. The HELLOs are either not going through or are being ignored. Keep in mind that by adding a static route, traffic over the GRE works fine. This would have no affect on frame size. The tunnel is also set with an "ip mtu 1400".

I'm hoping to grab some time Monday night to get the debugs.

rcohenbci · ‎09-13-2013

Just FYI, this is with both paths up

debug eigrp packet hello from switch-A

Sep 13 18:05:11.625: EIGRP: Sending HELLO on Vlan10

Sep 13 18:05:11.625: AS 42, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0

Sep 13 18:05:12.682: EIGRP: Sending HELLO on Tunnel1

Sep 13 18:05:12.682: AS 42, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0

Sep 13 18:05:13.613: EIGRP: Received HELLO on Tunnel1 nbr 10.10.20.1

Sep 13 18:05:13.613: AS 42, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0

Sep 13 18:05:14.728: EIGRP: Received HELLO on Vlan10 nbr 10.10.10.1

Sep 13 18:05:14.728: AS 42, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0

debug eigrp packet hello from switch-B

Sep 13 18:05:13.612: EIGRP: Sending HELLO on Tunnel1

Sep 13 18:05:13.612: AS 42, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0

Sep 13 18:05:14.728: EIGRP: Sending HELLO on Vlan10

Sep 13 18:05:14.728: AS 42, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0

Sep 13 18:05:16.372: EIGRP: Received HELLO on Vlan10 nbr 10.10.10.2

Sep 13 18:05:16.372: AS 42, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0

Sep 13 18:05:17.337: EIGRP: Received HELLO on Tunnel1 nbr 10.10.20.2

Sep 13 18:05:17.337: AS 42, Flags 0x0:(NULL), Seq 0/0 interfaceQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0

dslice · ‎09-13-2013

Traceroute from tunnel end-point to tunnel end-point to see how the tunnel travels