Solved: Re: Frame Relay - hubs can not ping eachother

arfett111 · ‎11-06-2010

I have a hub-and-spoke multipoint frame relay network configured with EIGRP and all routers receive routes from eachother. If I ping or traceroute the IP address of a loopback on spoke R2 from spoke R3 it goes out to the hub R1, but never gets any further. R1 can ping loopbacks on R2 and R3. R2 and R3 can ping loopbacks on R1. R2 and R3 received routes via EIGRP for the loopbacks via R1 (172.16.124.1) Can anyone tell me why the hubs can not ping eachother?

Here is an image of the setup in GNS3

R1

interface Loopback1
ip address 10.1.1.1 255.255.255.0
!
interface Loopback2
ip address 10.1.2.1 255.255.255.0
!
interface Loopback3
ip address 10.1.3.1 255.255.255.0
!
interface Serial0/0
bandwidth 256
no ip address
encapsulation frame-relay
clock rate 2000000
!
interface Serial0/0.1 multipoint
ip address 172.16.124.1 255.255.255.248
no ip split-horizon eigrp 25
frame-relay map ip 172.16.124.3 103 broadcast
frame-relay map ip 172.16.124.2 102 broadcast
!
router eigrp 25
network 10.1.0.0 0.0.255.255
network 172.16.0.0
no auto-summary

R2

interface Loopback1
ip address 10.2.1.1 255.255.255.0
!
interface Loopback2
ip address 10.2.2.1 255.255.255.0
!
interface Loopback3
ip address 10.2.3.1 255.255.255.0
!
interface Serial0/0
bandwidth 256
no ip address
encapsulation frame-relay
clock rate 2000000
!
interface Serial0/0.1 multipoint
ip address 172.16.124.2 255.255.255.248
frame-relay map ip 172.16.124.1 201 broadcast
!
interface Serial0/1
no ip address
shutdown
clock rate 2000000
!
router eigrp 25
network 10.2.0.0 0.0.255.255
network 172.16.0.0
no auto-summary

R2#show ip route
     172.16.0.0/29 is subnetted, 1 subnets
C       172.16.124.0 is directly connected, Serial0/0.1
     10.0.0.0/24 is subnetted, 9 subnets
D       10.3.1.0 [90/2809856] via 172.16.124.1, 00:59:48, Serial0/0.1
D       10.1.3.0 [90/2297856] via 172.16.124.1, 01:04:26, Serial0/0.1
C       10.2.1.0 is directly connected, Loopback1
D       10.1.2.0 [90/2297856] via 172.16.124.1, 01:04:26, Serial0/0.1
D       10.3.3.0 [90/2809856] via 172.16.124.1, 00:59:48, Serial0/0.1
C       10.2.2.0 is directly connected, Loopback2
D       10.1.1.0 [90/2297856] via 172.16.124.1, 01:04:26, Serial0/0.1
D       10.3.2.0 [90/2809856] via 172.16.124.1, 00:59:48, Serial0/0.1
C       10.2.3.0 is directly connected, Loopback3

R3

interface Loopback1
ip address 10.3.1.1 255.255.255.0
!
interface Loopback2
ip address 10.3.2.1 255.255.255.0
!
interface Loopback3
ip address 10.3.3.1 255.255.255.0
!
interface Serial0/0
bandwidth 256
no ip address
encapsulation frame-relay
clock rate 2000000
!
interface Serial0/0.1 multipoint
ip address 172.16.124.3 255.255.255.248
frame-relay map ip 172.16.124.1 301 broadcast
!
interface Serial0/1
no ip address
shutdown
clock rate 2000000
!
router eigrp 25
network 10.3.0.0 0.0.255.255
network 172.16.0.0
no auto-summary

R3#show ip route
     172.16.0.0/29 is subnetted, 1 subnets
C       172.16.124.0 is directly connected, Serial0/0.1
     10.0.0.0/24 is subnetted, 9 subnets
C       10.3.1.0 is directly connected, Loopback1
D       10.1.3.0 [90/2297856] via 172.16.124.1, 01:05:02, Serial0/0.1
D       10.2.1.0 [90/2809856] via 172.16.124.1, 01:00:58, Serial0/0.1
D       10.1.2.0 [90/2297856] via 172.16.124.1, 01:05:02, Serial0/0.1
C       10.3.3.0 is directly connected, Loopback3
D       10.2.2.0 [90/2809856] via 172.16.124.1, 01:00:58, Serial0/0.1
D       10.1.1.0 [90/2297856] via 172.16.124.1, 01:05:02, Serial0/0.1
C       10.3.2.0 is directly connected, Loopback2
D       10.2.3.0 [90/2809856] via 172.16.124.1, 01:00:58, Serial0/0.1

Peter Paluch · ‎11-06-2010

Hello,

As I indicated in my previous post, the problem here is caused by the selection of the source IP address by the spoke router. Without modifying it explicitly, it selects the IP address of the outgoing interface as the source for its ping packets. The ensuing reachability problem is natural and is typical for partially-meshed multiaccess networks: the IP thinks that just because two machines are in the same IP subnet they must be able to communicate directly - and that does not generally hold in partially-meshed networks like yours.

The OSPF in point-to-multipoint network mode works this around by adding the individual router's IP addresses with the /32 mask into the routing table and treating them just like any other reachable subnet, resulting in all other spoke's IP addresses being reachable, according to the routing table, via the hub router. The OSPF thus helps to solve this situation without actually having to modify the IP/DLCI mappings.

With EIGRP, I do not know of any such additional support, and you have to add the mappings manually. Certainly, in a large hub&spoke topologies, this can be tedious to do.

I am thinking whether the NHRP (Next Hop Resolution Protocol) - a generalized version of ARP - could be used for this, as it allows dynamic spoke address registration and resolution, but I have not tried that yet. I may do that in a few days.

Best regards,

Peter

View solution in original post

Peter Paluch · ‎11-06-2010

Hello,

The routing tables look fine... the spoke routes are learned correctly and the next hop is the R1's IP address that should be reachable. Interesting problem.

What I am missing in your configuration are the commands frame-relay interface-dlci used on your subinterfaces. They help the router to sort out the DLCIs among interfaces and subinterfaces. Please add them according to the following example just to be sure that we haven't omited any compulsory Frame Relay configuration commands:

R1:

interface Serial0/0.1 multipoint
 frame-relay interface-dlci 102 ! This command is added
 frame-relay interface-dlci 103 ! This command is added
 ip address 172.16.124.1 255.255.255.248
 no ip split-horizon eigrp 25
 frame-relay map ip 172.16.124.3 103 broadcast
 frame-relay map ip 172.16.124.2 102 broadcast

R2:

interface Serial0/0.1 multipoint
frame-relay interface-dlci 201 ! This command is added
ip address 172.16.124.2 255.255.255.248
frame-relay map ip 172.16.124.1 201 broadcast

R3:

interface Serial0/0.1 multipoint
frame-relay interface-dlci 301 ! This command is added
ip address 172.16.124.3 255.255.255.248
frame-relay map ip 172.16.124.1 301 broadcast

Then try to test the ping again. If it does not work please let us know and include the output of show frame-relay pvc and show frame-relay map from all routers. Thanks!

Best regards,

Peter

arfett111 · ‎11-06-2010

Thanks for the suggestion. Pings are still unsuccessful though. Here is the requested show information.

R1

R1#show frame-relay pvc

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
Local          2            0            0            0
Switched       0            0            0            0
Unused         0            0            0            0

DLCI = 102, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0.1

input pkts 176           output pkts 178          in bytes 12959
out bytes 14036          dropped pkts 0           in pkts dropped 0
out pkts dropped 0                out bytes dropped 0
in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
out BECN pkts 0          in DE pkts 0             out DE pkts 0
out bcast pkts 99        out bcast bytes 6336
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
pvc create time 01:46:03, last time pvc status changed 01:46:03

DLCI = 103, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0.1

input pkts 187           output pkts 183          in bytes 14552
out bytes 14163          dropped pkts 0           in pkts dropped 0
out pkts dropped 0                out bytes dropped 0
in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
out BECN pkts 0          in DE pkts 0             out DE pkts 0
out bcast pkts 99        out bcast bytes 6336
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
pvc create time 01:46:06, last time pvc status changed 01:46:06

R1#show frame-relay map
Serial0/0.1 (up): ip 172.16.124.2 dlci 102(0x66,0x1860), static,
              broadcast,
              CISCO, status defined, active
Serial0/0.1 (up): ip 172.16.124.3 dlci 103(0x67,0x1870), static,
              broadcast,
              CISCO, status defined, active

R2

R2#show frame-relay pvc

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
Local          1            0            0            0
Switched       0            0            0            0
Unused         0            0            0            0

DLCI = 201, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0.1

input pkts 182           output pkts 184          in bytes 14292
out bytes 13671          dropped pkts 0           in pkts dropped 0
out pkts dropped 0                out bytes dropped 0
in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
out BECN pkts 0          in DE pkts 0             out DE pkts 0
out bcast pkts 107       out bcast bytes 6902
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
pvc create time 01:54:49, last time pvc status changed 01:54:00

R2#show frame-relay map
Serial0/0.1 (up): ip 172.16.124.1 dlci 201(0xC9,0x3090), static,
broadcast,
CISCO, status defined, active

R3

R3#show frame-relay pvc

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
Local          1            0            0            0
Switched       0            0            0            0
Unused         0            0            0            0

DLCI = 301, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0.1

input pkts 188           output pkts 196          in bytes 14483
out bytes 15328          dropped pkts 0           in pkts dropped 0
out pkts dropped 0                out bytes dropped 0
in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
out BECN pkts 0          in DE pkts 0             out DE pkts 0
out bcast pkts 108       out bcast bytes 6966
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
pvc create time 01:54:17, last time pvc status changed 01:54:17

R3#show frame-relay map
Serial0/0.1 (up): ip 172.16.124.1 dlci 301(0x12D,0x48D0), static,
broadcast,
CISCO, status defined, active

Peter Paluch · ‎11-06-2010

Hello,

Can you please use the debug ip icmp on all three routers and try to ping the R3 from R2? Make note of the debugging messages on all three routers if there is anything interesting going on. Also, on R2, does it work if you ping the 172.16.124.3 (the R3's Frame Relay IP address)?

Best regards,

Peter

arfett111 · ‎11-06-2010

Peter,

Thanks for working with me. Pinging the serial interface IPs was one of the first things I tried when setting this up and R2 can not ping R3 (172.16.124.3) or the other way around.

After running the debugs it looks like this is happening (both ways):

R2 sends ping to R3

R3 receives ping

R3 sends ping reply

R2 never receives reply

This is the output from R3 when I had R2 pinging 10.3.1.1

*Mar 1 02:22:17.923: ICMP: echo reply sent, src 10.3.1.1, dst 172.16.124.2
R3#
*Mar 1 02:22:19.875: ICMP: echo reply sent, src 10.3.1.1, dst 172.16.124.2
R3#
*Mar 1 02:22:21.839: ICMP: echo reply sent, src 10.3.1.1, dst 172.16.124.2
R3#
*Mar 1 02:22:23.795: ICMP: echo reply sent, src 10.3.1.1, dst 172.16.124.2
R3#
*Mar 1 02:22:25.851: ICMP: echo reply sent, src 10.3.1.1, dst 172.16.124.2

When pinging R3 (172.16.124.3) from R2 then none of the routers show any debugging output.

R1 does not show any debugging output regardless of what you ping from R2/R3.

Peter Paluch · ‎11-06-2010

Hello,

Oh yes, now it's clear! Why has this eluded me so long?!

The problem is the missing IP/DLCI mapping - note that when you pinged the 10.3.1.1 from R2, your ping was sourced from the IP address 172.16.124.2 on R2. From the R3's viewpoint, this IP address is in the directly connected network on the Frame Relay subinterface, so instead of routing it through any particular neighboring router, it simply tries to directly send the reply back (which is a perfectly correct thing to do under IP). Because you did not create the IP/DLCI mapping on R3 that maps the 172.16.124.2 to the DLCI 301, the R3 is unable to send the reply back, resulting in failed pings.

Note that if you pinged the 10.3.1.1 using the source IP address 10.2.1.1, all would work because the ping would be sourced from a different IP address that is not considered to be directly reachable by the R3, and would instead be correctly routed via R1.

This problem can be remedied easily: On R2, add this command to the Frame Relay subinterface:

frame-relay map ip 172.16.124.3 201

and on R3:

frame-relay map ip 172.16.124.2 301

Let me know if this worked.

Best regards,

Peter

arfett111 · ‎11-06-2010

Peter,

Thank you very much. That worked. I just assumed with the routing table showing that subnet on S0/0.1 it would route any traffic out that interface, but I guess you must manually assign the map to all other spokes. This sounds tedious with more routers!

Peter Paluch · ‎11-06-2010

Hello,

As I indicated in my previous post, the problem here is caused by the selection of the source IP address by the spoke router. Without modifying it explicitly, it selects the IP address of the outgoing interface as the source for its ping packets. The ensuing reachability problem is natural and is typical for partially-meshed multiaccess networks: the IP thinks that just because two machines are in the same IP subnet they must be able to communicate directly - and that does not generally hold in partially-meshed networks like yours.

The OSPF in point-to-multipoint network mode works this around by adding the individual router's IP addresses with the /32 mask into the routing table and treating them just like any other reachable subnet, resulting in all other spoke's IP addresses being reachable, according to the routing table, via the hub router. The OSPF thus helps to solve this situation without actually having to modify the IP/DLCI mappings.

With EIGRP, I do not know of any such additional support, and you have to add the mappings manually. Certainly, in a large hub&spoke topologies, this can be tedious to do.

I am thinking whether the NHRP (Next Hop Resolution Protocol) - a generalized version of ARP - could be used for this, as it allows dynamic spoke address registration and resolution, but I have not tried that yet. I may do that in a few days.

Best regards,

Peter