cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5869
Views
20
Helpful
11
Replies

frame relay and OSPF networks

willowklan1
Level 1
Level 1

Hi everyone, 

having some trouble understanding the frame relay and ospf netowrks.

so I set up a hub-and-spoke lab. all the routers are on different networks.

the hub has two sub-interfaces, both multipoint and the spokes are configured as sub-interface point-to point.

connected some pc`s as well and here is my problem:

as far as I`m able to understand, when configuring ospf on such a network it will have a network type of NBMA by default, that means no dynamic adjacency.

so I tried to use the neighbor command, but still didn`t get any adjacency even after that.

now I know I can change the network type to "point-to-multipoint" but I don`t want to. I want to stick with NBMA 'cuz that`s the guy I am (plus packet tracer does not support that feature...)

I know I`m doing something wrong, but that aside - I would like to know some other things:

1) I know that on a multipoint sub-interface, if I don`t use inverse-arp, I have to configure frame relay using the "frame relay map" command, but some people here shut down the inverse-arp and still they use "interface dlci". how come? the interface dlci has to have inverse arp, because there needs to be something that connects the layers together. how do they do that? 

2) what is the whole /32 thingy in point-to-multipoint? when ospf advertise routes it does it in /32. what does it mean? does that have something to do with the broadcast?

3) when you configure a static mapping and add the "broadcast" statement at the end - what does it actually do? because OSPF can not use broadcast anyhow, it`s NBMA, so why do you need to to add that statement? it seems useless. what does it give me?

4) when using "frame relay map" on a sub-interface, it really doesn`t matter anymore if inverse arp is on, right? as far as I know using this command stops inverse arp. am I right?

5) when you say "full mesh" that only means all the routers can speak to all the routers, but that DOES NOT mean they have to be on the same network, right? they can be on different networks, have layer 2 dlci between one another and ospf on layer 3 advertising everything to everyone, and that would still be a full mesh. the "full" and "partial" has nothing do to with layer 3 ip addresses, it`s only about layer 2 connectivity, am I correct?

6) after I did the "neighbors" command and saw it didn`t work I tried to disable it - and lo and behold I got a log saying "interface gone from INIT to DOWN". ok, that means it tried to connect to the other side but didn`t get response, but the other side just did the same... got from INIT to down as well. what does that mean? why didn`t they get the adjacent?

7) last one, swear: in a NBMA network, when a spoke wants to neighbor with the other spoke - which address does he use for that? the address of that other spoke or the address of the hub? because I know NBMA does not make layer 2 connections for the next-hop (unlike point-to-multipoint) so if a spoke wants to know his neighbor he can not address him, he needs to go through the hub so he must neighbor with the hub, isn`t it true? tried both and got nothing though.


I`m so confused about that subject! now I know it is way beyond my ccna, but I don`t really care, I want to understand this subject! saw a couple of vids and just got even more jaded than I was before... :(

here is my very simple frame relay and ospf unworking network. the lines from the running-config are just the relevant ones for the sake of the case in hand. the ISP is configured correctly btw (at least I got that one going!)

=======================
router 1 (hub)
=======================

interface FastEthernet0/0
ip address 192.168.1.1 255.255.255.0
duplex auto
speed auto
!
interface FastEthernet0/1
no ip address
duplex auto
speed auto
shutdown
!
interface Serial0/1/0
no ip address
encapsulation frame-relay
!
interface Serial0/1/0.1 multipoint
ip address 10.0.0.2 255.0.0.0
frame-relay map ip 10.0.0.1 102
clock rate 2000000
!
interface Serial0/1/0.2 multipoint
ip address 172.16.0.2 255.255.0.0
frame-relay map ip 172.16.0.1 103
clock rate 2000000
!
router ospf 1
log-adjacency-changes
network 10.0.0.0 0.255.255.255 area 0
network 192.168.1.0 0.0.0.255 area 0
neighbor 10.0.0.1
neighbor 172.168.0.1


===================
router 2 (spoke)
===================

interface FastEthernet0/0
ip address 192.168.2.1 255.255.255.0
duplex auto
speed auto
!
interface FastEthernet0/1
no ip address
duplex auto
speed auto
shutdown
!
interface Serial0/1/0
no ip address
encapsulation frame-relay
!
interface Serial0/1/0.1 point-to-point
ip address 10.0.0.1 255.0.0.0
frame-relay interface-dlci 201
clock rate 2000000
!
router ospf 1
log-adjacency-changes
network 10.0.0.0 0.0.0.255 area 0
network 192.168.2.0 0.0.0.255 area 0
neighbor 10.0.0.2
neighbor 172.16.0.2
neighbor 172.16.0.1


====================
router 3 (spoke)
====================

interface FastEthernet0/0
ip address 192.168.3.1 255.255.255.0
duplex auto
speed auto
!
interface FastEthernet0/1
no ip address
duplex auto
speed auto
shutdown
!
interface Serial0/1/0
no ip address
encapsulation frame-relay
!
interface Serial0/1/0.1 point-to-point
ip address 172.16.0.1 255.255.0.0
frame-relay interface-dlci 301
clock rate 2000000
!
router ospf 1
log-adjacency-changes
network 172.16.0.0 0.0.255.255 area 0
network 192.168.3.0 0.0.0.255 area 0
neighbor 172.16.0.2
neighbor 10.0.0.2
neighbor 10.0.0.1

=========================================

those are a bunch of questions so thatks alot in advance

willow

2 Accepted Solutions

Accepted Solutions

Peter Paluch
Cisco Employee
Cisco Employee

Hi Willow,

First of all, kudos to your interest in going this deep and wanting to understand things so meticulously!

Your configuration appears to be correct for the most part. I assume that you can ping from your hub to both of your spokes, as that proves that the basic IP connectivity over Frame Relay is working.

Most probably, your OSPF is not coming up because of OSPF network type mismatch. Your hub uses multipoint subinterfaces, and OSPF defaults to the NBMA network type on those. However, on the spokes, you are using point-to-point subinterfaces and there, OSPF defaults to point-to-point network type. Because these two network types differ in default timers, OSPF routers ignore themselves even if they can hear each other's Hellos. For your topology to come up, you would need to make the network type identical - for example, by configuring ip ospf network non-broadcast on spoke routers and the respective subinterfaces (if they support this command in Packet Tracer).

1) I know that on a multipoint sub-interface, if I don`t use inverse-arp, I have to configure frame relay using the "frame relay map" command, but some people here shut down the inverse-arp and still they use "interface dlci". how come? the interface dlci has to have inverse arp, because there needs to be something that connects the layers together. how do they do that? 

I am not sure if that configuration would be working - but I would need to see a full interface configuration to see exactly what you are talking about. Nonetheless, however, you are saying that "interface dlci has to have inverse arp". I do not agree here. These two things are orthogonal - the frame-relay interface-dlci tells the router about the particular DLCI number you want to "park" under a particular interface or a subinterface, while InverseARP tries to find out who is connected to the other end of this DLCI - but you can substitute the InverseARP with static mappings.

2) what is the whole /32 thingy in point-to-multipoint? when ospf advertise routes it does it in /32. what does it mean? does that have something to do with the broadcast?

Let's explain it in an example. Assume that in your topology, you had only a single multipoint subinterface on the hub configured with 10.0.0.1/24, and both your spokes would also be using a multipoint subinterface or simply a physical interface configured with 10.0.0.11/24 (spoke 1) and 10.0.0.12/24 (spoke 2). All three routers would be in a single IP network 10.0.0.0/24 spanning over the Frame Relay cloud, but the PVCs would only go from hub to spokes, not between spokes.

Now if you wanted to reach one spoke from the other, under normal circumstances, you would need to configure static mappings on both spokes - InverseARP wouldn't help here, as it does not allow you to see beyond the nearest router. So the spoke 10.0.0.11 would need to use frame-relay map ip 10.0.0.12 spoke1-to-hub-DLCI while the spoke 10.0.0.12 would need to use the frame-relay map ip 10.0.0.11 spoke2-to-hub-DLCI command. Only now the spokes would be able to, say, ping each other.

Note that this static mapping is not necessary only for spokes pinging each other, but also for one spoke reaching a network behind the other spoke. Assume that behind spoke 1, there is a LAN with 192.168.11.0/24 network. On spoke 2, the routing entry would be: 192.168.11.0/24 via 10.0.0.11. Now, the next hop 10.0.0.11 is obviously in the directly connected network 10.0.0.0/24 that connects all three routers together over Frame Relay, so spoke 2 would want to resolve this next hop IP address into a Layer2 address, meaning it would be looking for the DLCI going to 10.0.0.11. If you didn't create static mappings, however, spoke 2 would have no clue which DLCI to use to reach the 10.0.0.11, and the network 192.168.11.0/24 behind this next hop would be unreachable from spoke 2.

The OSPF network type point-to-multipoint tries to solve both these problems by having each router advertise its own address on the Frame Relay network with a /32 netmask. At the same time, the point-to-multipoint network type forces OSPF to assume that only those routers that can hear each other's Hellos are directly reachable, and it starts ignoring the usual notion of "one common IP network - everyone can talk directly to everyone else". This helps OSPF to see the connectivity over a Frame Relay network as it truly exists. This has a profound impact on how the reachability turns out:

  • Spoke 1 would learn about 10.0.0.12/32 (spoke 2) from the hub, and the next hop would be the hub itself, that is, 10.0.0.1. Note that the reachability from spoke 1 to the 10.0.0.1 on the hub is automatic thanks to InverseARP. So without performing any static mappings, spoke 1 now knows thanks to the routing table contents that to reach 10.0.0.12, it has to go through 10.0.0.1 over the already known DLCI.
  • Spoke 2 would learn about 10.0.0.11/32 (spoke 1) from the hub, and the next hop would be the hub itself, that is 10.0.0.1. By the same logic as before, without any static mappings, spoke 2 now knows that to reach 10.0.0.11, it has to go through 10.0.0.1 over the already known DLCI.

So by introducing the point-to-multipoint OSPF network type, OSPF has made all the tedious work automatically for you, so that you don't need to do the static spoke-to-other-spokes mappings and things just work.

3) when you configure a static mapping and add the "broadcast" statement at the end - what does it actually do? because OSPF can not use broadcast anyhow, it`s NBMA, so why do you need to to add that statement? it seems useless. what does it give me?

The broadcast keyword after a static mapping is a flag that tells router to make a copy of every multicast and broadcast packet that is being sent out of this particular interface, and send it over the DLCI in this mapping. Conversely, multicasts and broadcasts will not be replicated over DLCIs in those mappings that lack the broadcast keyword. Mappings created by InverseARP always have the broadcast flag - this cannot be modified.

If you are running OSPF in NBMA mode, then indeed, the broadcast keyword is useless for OSPF. However, there may be other multicast/broadcast-based applications besides OSPF running over that interface, and these would no longer work. So you have to carefully judge whether there is truly no other protocol running between your routers that resorts to broadcasts and multicasts. If there is none, you can omit the broadcast flag from static mappings.

4) when using "frame relay map" on a sub-interface, it really doesn`t matter anymore if inverse arp is on, right? as far as I know using this command stops inverse arp. am I right?

When frame-relay map is used with a particular DLCI, the router will stop sending out InverseARP queries through that particular DLCI. However, if it receives a query from the other end, it will still respond with an InverseARP reply. So configuring a static mapping does not prevent a router from responding to InverseARP queries over that DLCI, it just stops it from sending queries itself.

The no frame-relay inverse-arp without further arguments affects all DLCIs at once; however, the result is similar: The router won't send out InverseARP requests but it will still respond to incoming requests.

Some textbooks and labs seem to be very focused on deactivating the InverseARP. I have never understood why they do this. InverseARP is here to help you and ease your life. I would not personally deactivate InverseARP unless I wanted to have almost paranoid control over what mappings are created and where they lead.

5) when you say "full mesh" that only means all the routers can speak to all the routers, but that DOES NOT mean they have to be on the same network, right? they can be on different networks, have layer 2 dlci between one another and ospf on layer 3 advertising everything to everyone, and that would still be a full mesh. the "full" and "partial" has nothing do to with layer 3 ip addresses, it`s only about layer 2 connectivity, am I correct?

Not entirely. Terms "full mesh" and "partial mesh" with respect to Frame Relay indeed refer to Layer2 PVCs between routers, but the context they are mentioned in is always related to a particular IP network. In other words, whenever I have routers connected over a Frame Relay cloud using a particular IP network, my question is: Do I have a full mesh of PVCs between these routers in this particular IP network, or is there only a partial mesh? This affects how the routing protocols and routing should operate within this network - whether we can assume that each router can talk to any other router in this network directly, or whether we have to use non-intuitive extra hops within this single IP network, like in the example with the OSPF point-to-mutipoint network.

We do not extend the notion of full/partial mesh beyond the scope of one single IP network, because to reach one network from another, you need routing no matter what. The question of full/partial mesh impacts whether you actually need to do routing even within a single IP network to provide full reachability (or better said, to substitute for a lacking reachability on Layer2).

6) after I did the "neighbors" command and saw it didn`t work I tried to disable it - and lo and behold I got a log saying "interface gone from INIT to DOWN". ok, that means it tried to connect to the other side but didn`t get response, but the other side just did the same... got from INIT to down as well. what does that mean? why didn`t they get the adjacent?

With real devices, this should have not been the case. I assume that this is a particular Packet Tracer peculiarity where it does not behave like a real OSPF implementation would. A static neighbor starts in the DOWN state, and for that neighbor to move to INIT state, at least one acceptable Hello would need to be received from it. However, because you have a network type mismatch between your spokes and the hub, the default timers differ between your hub and spoke, and that is why the Hello packets are not mutually acceptable at all. Therefore, on real routers, the static neighbors would not even move to the INIT state at all.

7) last one, swear: in a NBMA network, when a spoke wants to neighbor with the other spoke - which address does he use for that? the address of that other spoke or the address of the hub? because I know NBMA does not make layer 2 connections for the next-hop (unlike point-to-multipoint) so if a spoke wants to know his neighbor he can not address him, he needs to go through the hub so he must neighbor with the hub, isn`t it true? tried both and got nothing though.

As far as OSPF is concerned (and the same would be also for RIP and EIGRP), whenever you define static neighbors, you define them ONLY for those neighbors that are directly reachable. This means that in your network, a hub would define both spokes as static neighbors, but both spokes would only refer to the hub as their only static neighbor. It is incorrect to configure spokes for mutual neighborship. This is because in the better case, this neighborship is not necessary at all (the hub will propagate the learned information further), and in worse case, it would break the OSPF's capability of representing the true underlying connectivity that exists or does not exist in the network.

Feel welcome to ask further!

Best regards,
Peter

View solution in original post

Hi Willow,

Sorry for responding somewhat late.

when you got a hybrid network [ ... cut ... ] what happens with OSPF? on the physical or multipoint it would default to NBMA and on the p2p it would default to p2p.

Yes, that is correct.

can I just configure all of the frame relay to "multipoint" (even on the p2p parts of the network) and than choose OSPF network to be NBMA or point-to-multipoint?

First of all, I want to emphasize that the way you decide to sort all your PVCs under your interfaces is entirely up to you and your choice. Frame Relay will not force you into any particular solution. Whether you decide to create one p2p subinterface for each individual PVC, or whether you leave all PVCs under the physical interface and avoid creating subinterfaces completely, or whether you create a couple of subinterfaces and sort subgroups of PVCs under these subinterfaces like I did in my previous example - this is entirely your choice. Obviously, this will impact how many IP networks you have (as each interface on a router needs to have its own IP network) and whether you need to tweak the connectivity inside the network (if this network is not built over a full mesh of PVCs).

Now, if you decide to use the NBMA OSPF network type on a particular interface then there is a strong requirement I haven't mentioned yet: In the network on this interface, at least one router must have a direct PVC to each other router in the same network, and this router must be configured to become the DR. Routers that do not have direct PVCs to all other routers must be configured so that they will not take part in DR/BDR elections at all (meaning their OSPF priority must be set to 0). This is because otherwise, these routers would engage in DR/BDR elections, each choosing only from among neighbors they hear, and the elections would end up with possibly multiple DRs and BDRs on the segment.

Note that in my network, there is no router having PVCs to all other routers. If I wanted to configure the network with physical interfaces only, without subinterfaces, then I must not be using OSPF NBMA network type - otherwise some things could get very wrong. The only choice I have in the scenario with physical interfaces is the point-to-multipoint network type.

On the other hand, if I implemented the solution with the subgroups of PVCs that constitute full mesh, and having separate subinterfaces for those subgroups (R1/R2/R3, R4/R5/R6, R1/R5, R3/R4) then I could run either NBMA or point-to-multipoint over those. Point-to-multipoint will always work. NBMA will work properly only if the underlying L2 infrastructure is a full mesh, otherwise, it requires tweaking, and what it absolutely requires and what cannot be worked around by tweaking is at least one router having PVCs to all other routers in this network.

should I stick to the defaults and not change the OSPF network-type at all (not likely, I guess)?

The defaults often are not optimal. As you have correctly mentioned, on physical FR interfaces and multipoint suinterfaces, OSPF defaults to NBMA. However, the NBMA still assumes that there is a full mesh of PVCs to every other router in the IP network attached to this interface. NBMA will always require manual bootstrapping - at least the neighbors have to be configured manually. In addition, if the network is not fully meshed then all routers that don't have PVCs to all other routers must be configured with OSPF priority of 0 to avoid engaging into DR/BDR elections. Finally, spoke-to-spoke IP/DLCI mappings still need to be configured.

Point-to-multipoint network type is actually ideal for Frame Relay because it has none of the assumptions above. It does not elect DR/BDR so no hassle with that; it does not require a router having PVCs to every one in the network; rather than the IP network itself, it advertises the /32 addresses of each member router in this network, allowing the routers to learn paths to each other without the need to do manual IP/DLCI mappings. If having a Frame Relay network and in need of running OSPF over a physical or a multipoint Frame Relay interface, point-to-multipoint would be my personal preferred choice. This is, however, outside the CCNA scope.

There is an option of creating a pair of point-to-point subinterfaces for each PVC, one on each endpoint router of this PVC. As you have mentioned, this will consume more IP addresses than the multipoint approaches. However, at the same time, this is one of the easiest and most straightforward ways of running OSPF over Frame Relay: again, no DR/BDR, no full-mesh requirement (two interconnected routers by themselves constitute a full mesh), no need to create static IP/DLCI mappings, not even a need to configure OSPF network type statically as the point-to-point is the default on p2p subinterfaces. This approach, while using many subinterfaces and many small subnets, is the easiest to run, understand, and troubleshoot OSPF over Frame Relay.

It is actually funny that the NBMA network type, while often claimed as the network type of choice for Frame Relay networks, is not really that suitable for them. In reality, the NBMA network type was created for ATM networks where a router could ask the network to set up a virtual circuit to any other router it wanted to, so the full mesh was guaranteed. However, for Frame Relay networks with static PVCs, NBMA really isn't such a jewel. In CCNA, they don't go this deep and they usually teach you to run NBMA for strict hub-and-spoke networks with the spokes having their priority set to 0, meeting all requirements I've mentioned above (the hub has PVCs to all spokes so the hub, and the hub only, must be the DR), but you already know how much manual work is required to have this up and running.

Okay, so, to sum it up: If asked for CCNA purposes, either go for point-to-point subinterfaces for each PVC (the easiest way of doing things, anyway), or - if it is either a full mesh between all routers, or a strict hub-and-spoke - you may use the NBMA network type, provided that the spokes must be configured with the priority of 0. In real life, you would use either point-to-point subinterfaces for each PVC, or point-to-multipoint for multipoint interfaces, or even a specific combination of point-to-multipoint on the hub, and point-to-point on spokes (advanced CCNP topic).

about the split horizon - if I recall correctly from RIP over frame relay, that problem only happens when configuring the physical interface directly and the advice was not to turn off that feature but rather work only with sub-interfaces, all the time, in any case. is that correct? 

In RIP, the split horizon is a useful technique against routing loops, so deactivating it in RIP removed one of mechanisms that tried to prevent routing loops from occurring - that's why it was not that much recommended. EIGRP, on the other hand, has its own internal mechanisms to prevent routing loops, and split horizon is not really helpful to EIGRP as far as route loop prevention is concerned, so there is no risk of introducing routing loops in EIGRP by deactivating split horizon.

However, the comment about using subinterfaces instead of physical interfaces as a substitute for manipulating split horizon is not complete. What you probably meant to say is that if you configured your network using point-to-point subinterfaces then you wouldn't need to care about split horizon. That would be true. However, using multipoint subinterface wouldn't solve anything, as they behave in this respect in the same way as physical interfaces do.

Wow, yet another long and difficult post for you to read ;)

Best regards,
Peter

View solution in original post

11 Replies 11

Peter Paluch
Cisco Employee
Cisco Employee

Hi Willow,

First of all, kudos to your interest in going this deep and wanting to understand things so meticulously!

Your configuration appears to be correct for the most part. I assume that you can ping from your hub to both of your spokes, as that proves that the basic IP connectivity over Frame Relay is working.

Most probably, your OSPF is not coming up because of OSPF network type mismatch. Your hub uses multipoint subinterfaces, and OSPF defaults to the NBMA network type on those. However, on the spokes, you are using point-to-point subinterfaces and there, OSPF defaults to point-to-point network type. Because these two network types differ in default timers, OSPF routers ignore themselves even if they can hear each other's Hellos. For your topology to come up, you would need to make the network type identical - for example, by configuring ip ospf network non-broadcast on spoke routers and the respective subinterfaces (if they support this command in Packet Tracer).

1) I know that on a multipoint sub-interface, if I don`t use inverse-arp, I have to configure frame relay using the "frame relay map" command, but some people here shut down the inverse-arp and still they use "interface dlci". how come? the interface dlci has to have inverse arp, because there needs to be something that connects the layers together. how do they do that? 

I am not sure if that configuration would be working - but I would need to see a full interface configuration to see exactly what you are talking about. Nonetheless, however, you are saying that "interface dlci has to have inverse arp". I do not agree here. These two things are orthogonal - the frame-relay interface-dlci tells the router about the particular DLCI number you want to "park" under a particular interface or a subinterface, while InverseARP tries to find out who is connected to the other end of this DLCI - but you can substitute the InverseARP with static mappings.

2) what is the whole /32 thingy in point-to-multipoint? when ospf advertise routes it does it in /32. what does it mean? does that have something to do with the broadcast?

Let's explain it in an example. Assume that in your topology, you had only a single multipoint subinterface on the hub configured with 10.0.0.1/24, and both your spokes would also be using a multipoint subinterface or simply a physical interface configured with 10.0.0.11/24 (spoke 1) and 10.0.0.12/24 (spoke 2). All three routers would be in a single IP network 10.0.0.0/24 spanning over the Frame Relay cloud, but the PVCs would only go from hub to spokes, not between spokes.

Now if you wanted to reach one spoke from the other, under normal circumstances, you would need to configure static mappings on both spokes - InverseARP wouldn't help here, as it does not allow you to see beyond the nearest router. So the spoke 10.0.0.11 would need to use frame-relay map ip 10.0.0.12 spoke1-to-hub-DLCI while the spoke 10.0.0.12 would need to use the frame-relay map ip 10.0.0.11 spoke2-to-hub-DLCI command. Only now the spokes would be able to, say, ping each other.

Note that this static mapping is not necessary only for spokes pinging each other, but also for one spoke reaching a network behind the other spoke. Assume that behind spoke 1, there is a LAN with 192.168.11.0/24 network. On spoke 2, the routing entry would be: 192.168.11.0/24 via 10.0.0.11. Now, the next hop 10.0.0.11 is obviously in the directly connected network 10.0.0.0/24 that connects all three routers together over Frame Relay, so spoke 2 would want to resolve this next hop IP address into a Layer2 address, meaning it would be looking for the DLCI going to 10.0.0.11. If you didn't create static mappings, however, spoke 2 would have no clue which DLCI to use to reach the 10.0.0.11, and the network 192.168.11.0/24 behind this next hop would be unreachable from spoke 2.

The OSPF network type point-to-multipoint tries to solve both these problems by having each router advertise its own address on the Frame Relay network with a /32 netmask. At the same time, the point-to-multipoint network type forces OSPF to assume that only those routers that can hear each other's Hellos are directly reachable, and it starts ignoring the usual notion of "one common IP network - everyone can talk directly to everyone else". This helps OSPF to see the connectivity over a Frame Relay network as it truly exists. This has a profound impact on how the reachability turns out:

  • Spoke 1 would learn about 10.0.0.12/32 (spoke 2) from the hub, and the next hop would be the hub itself, that is, 10.0.0.1. Note that the reachability from spoke 1 to the 10.0.0.1 on the hub is automatic thanks to InverseARP. So without performing any static mappings, spoke 1 now knows thanks to the routing table contents that to reach 10.0.0.12, it has to go through 10.0.0.1 over the already known DLCI.
  • Spoke 2 would learn about 10.0.0.11/32 (spoke 1) from the hub, and the next hop would be the hub itself, that is 10.0.0.1. By the same logic as before, without any static mappings, spoke 2 now knows that to reach 10.0.0.11, it has to go through 10.0.0.1 over the already known DLCI.

So by introducing the point-to-multipoint OSPF network type, OSPF has made all the tedious work automatically for you, so that you don't need to do the static spoke-to-other-spokes mappings and things just work.

3) when you configure a static mapping and add the "broadcast" statement at the end - what does it actually do? because OSPF can not use broadcast anyhow, it`s NBMA, so why do you need to to add that statement? it seems useless. what does it give me?

The broadcast keyword after a static mapping is a flag that tells router to make a copy of every multicast and broadcast packet that is being sent out of this particular interface, and send it over the DLCI in this mapping. Conversely, multicasts and broadcasts will not be replicated over DLCIs in those mappings that lack the broadcast keyword. Mappings created by InverseARP always have the broadcast flag - this cannot be modified.

If you are running OSPF in NBMA mode, then indeed, the broadcast keyword is useless for OSPF. However, there may be other multicast/broadcast-based applications besides OSPF running over that interface, and these would no longer work. So you have to carefully judge whether there is truly no other protocol running between your routers that resorts to broadcasts and multicasts. If there is none, you can omit the broadcast flag from static mappings.

4) when using "frame relay map" on a sub-interface, it really doesn`t matter anymore if inverse arp is on, right? as far as I know using this command stops inverse arp. am I right?

When frame-relay map is used with a particular DLCI, the router will stop sending out InverseARP queries through that particular DLCI. However, if it receives a query from the other end, it will still respond with an InverseARP reply. So configuring a static mapping does not prevent a router from responding to InverseARP queries over that DLCI, it just stops it from sending queries itself.

The no frame-relay inverse-arp without further arguments affects all DLCIs at once; however, the result is similar: The router won't send out InverseARP requests but it will still respond to incoming requests.

Some textbooks and labs seem to be very focused on deactivating the InverseARP. I have never understood why they do this. InverseARP is here to help you and ease your life. I would not personally deactivate InverseARP unless I wanted to have almost paranoid control over what mappings are created and where they lead.

5) when you say "full mesh" that only means all the routers can speak to all the routers, but that DOES NOT mean they have to be on the same network, right? they can be on different networks, have layer 2 dlci between one another and ospf on layer 3 advertising everything to everyone, and that would still be a full mesh. the "full" and "partial" has nothing do to with layer 3 ip addresses, it`s only about layer 2 connectivity, am I correct?

Not entirely. Terms "full mesh" and "partial mesh" with respect to Frame Relay indeed refer to Layer2 PVCs between routers, but the context they are mentioned in is always related to a particular IP network. In other words, whenever I have routers connected over a Frame Relay cloud using a particular IP network, my question is: Do I have a full mesh of PVCs between these routers in this particular IP network, or is there only a partial mesh? This affects how the routing protocols and routing should operate within this network - whether we can assume that each router can talk to any other router in this network directly, or whether we have to use non-intuitive extra hops within this single IP network, like in the example with the OSPF point-to-mutipoint network.

We do not extend the notion of full/partial mesh beyond the scope of one single IP network, because to reach one network from another, you need routing no matter what. The question of full/partial mesh impacts whether you actually need to do routing even within a single IP network to provide full reachability (or better said, to substitute for a lacking reachability on Layer2).

6) after I did the "neighbors" command and saw it didn`t work I tried to disable it - and lo and behold I got a log saying "interface gone from INIT to DOWN". ok, that means it tried to connect to the other side but didn`t get response, but the other side just did the same... got from INIT to down as well. what does that mean? why didn`t they get the adjacent?

With real devices, this should have not been the case. I assume that this is a particular Packet Tracer peculiarity where it does not behave like a real OSPF implementation would. A static neighbor starts in the DOWN state, and for that neighbor to move to INIT state, at least one acceptable Hello would need to be received from it. However, because you have a network type mismatch between your spokes and the hub, the default timers differ between your hub and spoke, and that is why the Hello packets are not mutually acceptable at all. Therefore, on real routers, the static neighbors would not even move to the INIT state at all.

7) last one, swear: in a NBMA network, when a spoke wants to neighbor with the other spoke - which address does he use for that? the address of that other spoke or the address of the hub? because I know NBMA does not make layer 2 connections for the next-hop (unlike point-to-multipoint) so if a spoke wants to know his neighbor he can not address him, he needs to go through the hub so he must neighbor with the hub, isn`t it true? tried both and got nothing though.

As far as OSPF is concerned (and the same would be also for RIP and EIGRP), whenever you define static neighbors, you define them ONLY for those neighbors that are directly reachable. This means that in your network, a hub would define both spokes as static neighbors, but both spokes would only refer to the hub as their only static neighbor. It is incorrect to configure spokes for mutual neighborship. This is because in the better case, this neighborship is not necessary at all (the hub will propagate the learned information further), and in worse case, it would break the OSPF's capability of representing the true underlying connectivity that exists or does not exist in the network.

Feel welcome to ask further!

Best regards,
Peter

Hו Peter,

Thank you for taking the time and helping me out. I can not emphasize enough how grateful I am for your detailed and comprehensive answers.

I need to get cleared about those timers and network-type: why can`t I just adjust the hello and dead intervals between the NBMA and the point-to-point in my topology? why do I need to change the actual OSPF network definition if all the problem is syncing the timers? wouldn`t it be enough? (it wouldn`t, I tried, since packet tracer does not support point-to-multipoint).

I am also sorry to say I`m not quite sure I understood your answer to question #5. in your example you showed a single IP netwrok (10.0.0.0) and it was a partial mesh (the spokes couldn`t talk directly to one another). I didn`t quite understand what the IP has to do with it or what did you mean when saying "We do not extend the notion of full/partial mesh beyond the scope of one single IP network, because to reach one network from another, you need routing no matter what". can you please give me an example like you did with point-to-multipoint network?

also, is there anything important I need to know about EIGRP on frame relay? all the examples I see are regarded to RIP and OSPF. do you know if EIGRP has any special notes I should take? 

thanks again for all your help!

willow

Hi Willow,

You are welcome!

I need to get cleared about those timers and network-type: why can`t I just adjust the hello and dead intervals between the NBMA and the point-to-point in my topology?

This is because OSPF uses different incompatible approaches to handling and representing various network types in its link-state database.

Multi-access networks (both broadcast and nonbroadcast) need to have DR and BDR elected, and the DR is used in the link-state database to represent the network for which it is the DR. Other routers, instead of indicating an adjacency to each other, only indicate the adjacency to the DR representing the entire network. This helps bring down the number of adjacencies in the link-state database.

On point-to-point and point-to-multipoint network types, this is not done, no DR/BDR is elected, and every router indicates its adjacency to every other router it can hear Hellos from. This helps representing the reachability in a single network as it truly exists, but it requires more links to be stored in the link-state database.

If two routers differ in their OSPF network type then each one will be approaching and representing this network differently in its link-state database, and as a result, the information will not be consistent and the routing won't work properly. Usually, this is seen on real-life routers as the routing table remaining empty and not becoming populated with OSPF-learned routes.

Therefore, simply correcting timers isn't enough. Routers have to treat a single network in a single way.

I am also sorry to say I`m not quite sure I understood your answer to question #5. in your example you showed a single IP netwrok (10.0.0.0) and it was a partial mesh (the spokes couldn`t talk directly to one another). I didn`t quite understand what the IP has to do with it or what did you mean when saying "We do not extend the notion of full/partial mesh beyond the scope of one single IP network, because to reach one network from another, you need routing no matter what". can you please give me an example like you did with point-to-multipoint network?

Okay, let's see... Consider the following figure:

To omit unnecessary details, I haven't drawn a huge Frame Relay cloud here, and instead, I have simply drawn a line where a PVC exists. Would you say this is a full mesh? Certainly not, as there are lots of PVCs missing (for example, R2 doesn't have a PVC to either R4, R5 or R6). This is, as far as you can tell, a partial mesh, and if you configure all these routers on their physical interfaces only (meaning that they all are in the same IP network), you will need to deal with all peculiarities of a single IP network over a partially meshed Layer2 topology.

However, what if we decided to take out the sub-portions of this topology that do constitute a full mesh, and had them configured under a specific subinterface? Let's check it out again:

R1/R2/R3 are fully meshed to each other, so let's configure these DLCIs on R1, R2 and R3 on a separate multipoint subinterface, meaning this full mesh will become a standalone IP network, say, 10.0.1.0/24. Similarly, R4/R5/R5 are fully meshed to each other, so let's create a standalone multipoint interface for these DLCIs on R4, R5, and R6, meaning that this full mesh will become another standalone IP network, say, 10.0.2.0/24.

Additionally, for R1/R5 and R3/R4 PVCs, we can create additional point-to-point subinterfaces (no need for multipoint here) which will become two new IP networks, say, 10.0.15.0/24 and 10.0.34.0/24, respectively.

Notice what I have done: I have simply started treating this formerly partially-meshed topology differently - because the way I am going to sort the DLCIs under individual subinterfaces is purely my choice. I have taken out the sub-components that are fully meshed, and created IP networks out of those:

R1/R2/R3: 10.0.1.0/24
R4/R5/R6: 10.0.2.0/24
R1/R5: 10.0.15.0/24
R3/R4: 10.0.34.0/24

Each of these IP networks has been deliberately selected and set up over a fully meshed sub-topology of the entire topology. Now, from the viewpoint of 10.0.1.0/24, do I care that the whole topology is not fully meshed? Or do I even care how whether other IP networks are fully meshed? No, I do not - as far as I am concerned, to leave 10.0.1.0/24 for any other network, I need to go either through R1 or R3 depending on the routing and the metric (and knowing that I have a full mesh in 10.0.1.0/24, I know I can reach R1 or R3 directly, without an extra hop), but what happens then isn't really my problem inside 10.0.1.0/24 - it's the problem of the next hop.

This is what I had on my mind when I said that we do not talk about full/partial mesh outside the scope of a particular IP network. Once again, the issue of full or partial mesh is always discussed with respect to the IP network that runs over this mesh, because it impact the IP operation in this network. Outside this network it doesn't matter - the other networks have their own full or partial mesh and take care about themselves.

also, is there anything important I need to know about EIGRP on frame relay? all the examples I see are regarded to RIP and OSPF. do you know if EIGRP has any special notes I should take? 

A quick list of things to be aware of:

  • If you rely on InverseARP then you do not need to configure static neighbors in EIGRP. Mappings created by InverseARP will have the broadcast flag set, meaning that EIGRP routers will be able to talk over the DLCIs using multicasts. With InverseARP, EIGRP will start working out of the box.
  • Be careful about the Split Horizon rule. On physical Frame Relay interfaces, it is on by default. The hub will learn about all networks behind the spokes but it won't be allowed to advertise them back through the same interface to other spokes. As a result, spokes will not be aware of networks on other spokes. For this, the Split Horizon will need to be deactivated on the hub.
  • EIGRP has the choice of modifying or keeping the next hop of one spoke network as it is advertised to another spoke via hub. In a hub-and-spoke topology, each spoke should learn the networks from other spokes with the next hop set to the hub's IP address - otherwise you'll be forced to create the static mappings manually. The hub shall be configured with ip next-hop-self eigrp process-id on the Frame Relay interface to maintain this (this should be the default, though). You will still need to create the static mappings if you want to ping one spoke from another.

These are some of the things off the top of my mind. There are also issues with the bandwidth setting but they are CCNP-level stuff so you shouldn't be bothered by that too much.

Feel welcome to ask further!

Best regards,
Peter

Hi Peter,

I think I`m getting there... it would still take some lab-set-ups to assure that, but the main parts starts falling to place. 

about the split horizon - if I recall correctly from RIP over frame relay, that problem only happens when configuring the physical interface directly and the advice was not to turn off that feature but rather work only with sub-interfaces, all the time, in any case. is that correct? 

sincerely,

willow

Hi Peter,

been doing labs all weekend and most of it seems to go very well, thanks again. just a quick wrap-up: when you got a hybrid network (i.e. like in your example, where some of the routers share the same network in a full mesh and some are on different networks in a point-to-point sub-interface) - what happens with OSPF? on the physical or multipoint it would default to NBMA and on the p2p it would default to p2p.

can I just configure all of the frame relay to "multipoint" (even on the p2p parts of the network) and than choose OSPF network to be NBMA or point-to-multipoint? should I stick to the defaults and not change the OSPF network-type at all (not likely, I guess)? should I create the whole network via p2p sub-interfaces (would waste וnnecessary addresses by doing that)? 

what is your experienced opinion?

thanks,

willow

Hi Willow,

Sorry for responding somewhat late.

when you got a hybrid network [ ... cut ... ] what happens with OSPF? on the physical or multipoint it would default to NBMA and on the p2p it would default to p2p.

Yes, that is correct.

can I just configure all of the frame relay to "multipoint" (even on the p2p parts of the network) and than choose OSPF network to be NBMA or point-to-multipoint?

First of all, I want to emphasize that the way you decide to sort all your PVCs under your interfaces is entirely up to you and your choice. Frame Relay will not force you into any particular solution. Whether you decide to create one p2p subinterface for each individual PVC, or whether you leave all PVCs under the physical interface and avoid creating subinterfaces completely, or whether you create a couple of subinterfaces and sort subgroups of PVCs under these subinterfaces like I did in my previous example - this is entirely your choice. Obviously, this will impact how many IP networks you have (as each interface on a router needs to have its own IP network) and whether you need to tweak the connectivity inside the network (if this network is not built over a full mesh of PVCs).

Now, if you decide to use the NBMA OSPF network type on a particular interface then there is a strong requirement I haven't mentioned yet: In the network on this interface, at least one router must have a direct PVC to each other router in the same network, and this router must be configured to become the DR. Routers that do not have direct PVCs to all other routers must be configured so that they will not take part in DR/BDR elections at all (meaning their OSPF priority must be set to 0). This is because otherwise, these routers would engage in DR/BDR elections, each choosing only from among neighbors they hear, and the elections would end up with possibly multiple DRs and BDRs on the segment.

Note that in my network, there is no router having PVCs to all other routers. If I wanted to configure the network with physical interfaces only, without subinterfaces, then I must not be using OSPF NBMA network type - otherwise some things could get very wrong. The only choice I have in the scenario with physical interfaces is the point-to-multipoint network type.

On the other hand, if I implemented the solution with the subgroups of PVCs that constitute full mesh, and having separate subinterfaces for those subgroups (R1/R2/R3, R4/R5/R6, R1/R5, R3/R4) then I could run either NBMA or point-to-multipoint over those. Point-to-multipoint will always work. NBMA will work properly only if the underlying L2 infrastructure is a full mesh, otherwise, it requires tweaking, and what it absolutely requires and what cannot be worked around by tweaking is at least one router having PVCs to all other routers in this network.

should I stick to the defaults and not change the OSPF network-type at all (not likely, I guess)?

The defaults often are not optimal. As you have correctly mentioned, on physical FR interfaces and multipoint suinterfaces, OSPF defaults to NBMA. However, the NBMA still assumes that there is a full mesh of PVCs to every other router in the IP network attached to this interface. NBMA will always require manual bootstrapping - at least the neighbors have to be configured manually. In addition, if the network is not fully meshed then all routers that don't have PVCs to all other routers must be configured with OSPF priority of 0 to avoid engaging into DR/BDR elections. Finally, spoke-to-spoke IP/DLCI mappings still need to be configured.

Point-to-multipoint network type is actually ideal for Frame Relay because it has none of the assumptions above. It does not elect DR/BDR so no hassle with that; it does not require a router having PVCs to every one in the network; rather than the IP network itself, it advertises the /32 addresses of each member router in this network, allowing the routers to learn paths to each other without the need to do manual IP/DLCI mappings. If having a Frame Relay network and in need of running OSPF over a physical or a multipoint Frame Relay interface, point-to-multipoint would be my personal preferred choice. This is, however, outside the CCNA scope.

There is an option of creating a pair of point-to-point subinterfaces for each PVC, one on each endpoint router of this PVC. As you have mentioned, this will consume more IP addresses than the multipoint approaches. However, at the same time, this is one of the easiest and most straightforward ways of running OSPF over Frame Relay: again, no DR/BDR, no full-mesh requirement (two interconnected routers by themselves constitute a full mesh), no need to create static IP/DLCI mappings, not even a need to configure OSPF network type statically as the point-to-point is the default on p2p subinterfaces. This approach, while using many subinterfaces and many small subnets, is the easiest to run, understand, and troubleshoot OSPF over Frame Relay.

It is actually funny that the NBMA network type, while often claimed as the network type of choice for Frame Relay networks, is not really that suitable for them. In reality, the NBMA network type was created for ATM networks where a router could ask the network to set up a virtual circuit to any other router it wanted to, so the full mesh was guaranteed. However, for Frame Relay networks with static PVCs, NBMA really isn't such a jewel. In CCNA, they don't go this deep and they usually teach you to run NBMA for strict hub-and-spoke networks with the spokes having their priority set to 0, meeting all requirements I've mentioned above (the hub has PVCs to all spokes so the hub, and the hub only, must be the DR), but you already know how much manual work is required to have this up and running.

Okay, so, to sum it up: If asked for CCNA purposes, either go for point-to-point subinterfaces for each PVC (the easiest way of doing things, anyway), or - if it is either a full mesh between all routers, or a strict hub-and-spoke - you may use the NBMA network type, provided that the spokes must be configured with the priority of 0. In real life, you would use either point-to-point subinterfaces for each PVC, or point-to-multipoint for multipoint interfaces, or even a specific combination of point-to-multipoint on the hub, and point-to-point on spokes (advanced CCNP topic).

about the split horizon - if I recall correctly from RIP over frame relay, that problem only happens when configuring the physical interface directly and the advice was not to turn off that feature but rather work only with sub-interfaces, all the time, in any case. is that correct? 

In RIP, the split horizon is a useful technique against routing loops, so deactivating it in RIP removed one of mechanisms that tried to prevent routing loops from occurring - that's why it was not that much recommended. EIGRP, on the other hand, has its own internal mechanisms to prevent routing loops, and split horizon is not really helpful to EIGRP as far as route loop prevention is concerned, so there is no risk of introducing routing loops in EIGRP by deactivating split horizon.

However, the comment about using subinterfaces instead of physical interfaces as a substitute for manipulating split horizon is not complete. What you probably meant to say is that if you configured your network using point-to-point subinterfaces then you wouldn't need to care about split horizon. That would be true. However, using multipoint subinterface wouldn't solve anything, as they behave in this respect in the same way as physical interfaces do.

Wow, yet another long and difficult post for you to read ;)

Best regards,
Peter

Hi Peter,

well, those posts are truly long, but not at all difficult to understand, since you explain everything so clearly. I even got a history lesson and finally understood the origin of NBMA from an ATM point of view... remarkable! btw, all those questions are not regarding ccna, ccnp or any other exam, I really don`t care where do they teach you that stuff, I just want to know it. and now I do thanks to you.

again, thanks alot for time, patience and outstanding knowledge. you have been a great source of help on a tough subject (at least for me). 

willow

Hi Willow,

I am honored and glad to have helped! You are always welcome. Thank you very much for your generous ratings.

Please continue posting your questions on these forums. You have a keen mind and you want to know things deep into their nature - that is awesome. Answering such questions is always a pleasure.

Best regards,
Peter

Hi Peter...

sorry to bother you again, just a quick one this time. you said: 

If you are running OSPF in NBMA mode, then indeed, the broadcast keyword is useless for OSPF (....) So you have to carefully judge whether there is truly no other protocol running between your routers that resorts to broadcasts and multicasts. If there is none, you can omit the broadcast flag from static mappings.

so I set up a lab running NBMA, made all the adjustments (hub is DR and spokes on priority 0, neighbors statically introduced, timers set to 10:40 and "map ip" between the spokes to get a full mesh). after setting OSPF it refused to acknowledge my efforts. only after I used the "broadcast" statement in the "map ip" command the routes appeared, being learned over the network.

now I know NBMA is a non-broadcast by definition, and you also said that "broadcast" in that network is useless, so how come it happened? isn`t the "broadcast" only good for a network that actually supports broadcast, like point-to-multipoint? 

confused again,

willow

Hi Willow,

after setting OSPF it refused to acknowledge my efforts. only after I used the "broadcast" statement in the "map ip" command the routes appeared, being learned over the network.

Was that in Packet Tracer? If so then it is most probably a limitation in its implementation of OSPF. I have just concocted a quick hub-and-spoke network over Frame Relay in my Dynamips/Dynagen - deactivated InverseARP, did all mappings statically, configured OSPF in NBMA network mode with static neighbors defined on the hub, and everything came up as expected.

isn`t the "broadcast" only good for a network that actually supports broadcast, like point-to-multipoint? 

Don't forget that the way we're using the PVCs does not change the underlying nature of the Frame Relay. Frame Relay is a NBMA-type data-link layer technology, period. The broadcast flag only tells the router: "Hey, have some broadcast/multicast packet to send out this interface? Send one copy through this DLCI!" That's all.

It is perhaps noteworthy to mention that the we have "network types" and "OSPF network types". A "network type" is given by the nature of the data-link layer technology, and basically, we have broadcast multi-access (like Ethernet), non-broadcast multi-access (like Frame Relay), and point-to-point (like PPP). You cannot override the "network type" because it is simply the very nature of the data-link layer technology. An "OSPF network type" is what OSPF in particular thinks about the underlying data-link layer technology, how it treats and uses it, perhaps even how it works around its limitations. You can force OSPF to believe that the nature of the  data-link layer is different than what it truly is - sometimes it is legal to do it, sometimes it's not because the "network type" does not change with the "OSPF network type".

I am not sure if this was helpful so again, please feel welcome to ask further.

Best regards,
Peter

Hi Peter,

ya, I tried that "broadcast" command on packet-tracer so as far as I`m concerned it`s a limitations, not a real-life necessity, as you explained. 

thanks alot mate,

willow

Review Cisco Networking for a $25 gift card