MPLS TE FA query

[ulatif]

MPLS TE FA requires bidirectional LSPs but I havent been able to get sufficient reasoning as to why it needs bidir LSPs

(I read that CSPF does some bidir checks before installing it but I need some more insight into this)

Anyone has more detailed knowledge on why FA requires bidir LSPs ?

[ulatif]

Guiseppe & Joe,

Yes agree to an extent that FA is probably looking for equivalent of a direct link A->B and B->A before actively installing it but this then creates loopholes with regards to whether the LSP A->B and B->A needs to be exactly identical i.e. Loopback0 on A to Loopback0 on B and vice versa? or can we use Loopback0 on A to Loop0 on B when going from A->B and then Loopback1 on B to Loop0 on A when going from B->A ?

I ask this because in some of my lab simulations that i have done, i can see that the FA still works even if Loopbacks from A->B and B->A are not identical.... so this still leaves open the question as to what exactly the SW is checking for before installing the FA as a link -

This brings me to my next point with regards to FA in scenarios where we used inter-area TE tunnels.

I'll open a new discussion on that one.

[IVAN PEPELNJAK]

The MPLS TE FA is a point-to-point link between two non-adjacent routers (let's call them A and B) established over unidirectional MPLS LSPs (established through RSVP TE extensions). If you want an SPF routing protocol to treat a point-to-point entity as a link, the adjacency must be reported by both sides of the link (A-to-B and B-to-A), otherwise the SPF algorithm rejects the link when building the topology database (this check prevents, among other things, blackholing due to unidirectional links).

Furthermore, A and B have to be able to forward packets across the point-to-point entity, otherwise you could easily end with a routing loop (remember: you can specify metric on MPLS TE tunnel). The exception is the OSPF virtual circuit, which always costs as much as the underlying physical path (thus preventing the routing loops) and where OSPF takes special precautions to prevent black holes.

If you have an MPLS TE LSP from A to B (but not the reverse LSP), then A can send packets directly to B, but not vice versa, so you could get routing loops in reverse direction (plus you'd fail the check in the SPF algorithm). Thus you need two independent MPLS TE LSPs.

Does this make sense?
Ivan

[ulatif]

Hi Ivan,

I agree with what you are saying and understand that TE FA is defined as type p2p in IGP (e.g. OSPF).

But as we know that TE LSPs are unidirectional and independant of each other e.g. if you have and LSP from A -> B and an LSP from B -> A, these are two seperate TE LSPs independant of each other.

We also know that IGP shorcut (autoroute announce in Cisco) doesnt perform this check but then it also doesnt announce the TE LSP as a link in the IGP but only advertises it as a directly connected route in the head-end routers routing table.

Therefore, when we see the restriction that TE FA needs bidirectional LSPs, one would ask the question, what would break if we didnt have bidrectional TE LSPs - you mentioned that it can lead to loops or black holing ?

TE FA is also not supposed to have IGP peering foirmed over it (e.g. as normally on a p2p link, IGP forms neighbor relationship using Hellos etc) - its only there for SPF to treat it as a potential path to destination.

It would be nice if you could highlight a scenario where (using TE FA) and we break the rule that FA doesnt check for bidrectional LSPs and installs the TE FA even for a unidirectional LSP, in what way does it lead to loops or blackholing ? an example would be nice..

P.S. I am great fan of yours btw ... started my MPLS basics from your book and today definitely attribute my CCIE SP to the knowledge gained from reading your materials.

cheers

Usman

[IVAN PEPELNJAK · Accepted solution]

You should consider two scenarios:

(A) The tail-end LSR advertises point-to-point adjacency to head-end LSR without having an actual transport path. This is plain stupid and it's not hard to build a scenario resulting in a routing loop.

(B) The head-end LSR adverises point-to-point link to the tail-end LSR in its LSA, the tail-end LSR doesn't advertise the adjacency, but other routers still consider the unidirectional adjacency in their SPF algorithm. In OSPF case, this is a clear violation of step 2.b of algorithm described in Section 16.1 of RFC 2328 (page 163 of the RFC).

There are very good reasons why the bidirectional check is in mandated in the SPF algorithm and even if a vendor X might decide it's time to break the RFC rules (and maybe even call that "innovation") the third-party routers (from vendors still believing in following the RFCs) in a mixed-vendor network would ignore such "adjacencies".

Fixed an obvious stupidity: I write about "LSRs" not "LSPs"

[ulatif]

okay I presume when you said "LSP" in the above reply, you meant "LSR" ?

In regards to your statement :

(A) The tail-end LSP advertises point-to-point adjacency to head-end LSP without having an actual transport path. This is plain stupid and it's not hard to build a scenario resulting in a routing loop.

USMAN>> Why would a tail-end LSR advertise p2p adjacency to head-end LSR if it has no TE FA configured ? This is where we started the discussion that head-end has TE FA configured but tail-end has no TE FA configured so head-end is the only device advertising the adjacency to its IGP peers. Pls clarify what you mean by this point. Maybe I have misunderstood.

(B) The head-end LSP adverises point-to-point link to the tail-end in its LSA, the tail-end doesn't advertise the adjacency, but other routers still consider the unidirectional adjacency in their SPF algorithm. In OSPF case, this is a clear violation of step 2.b of algorithm described in Section 16.1 of RFC 2328 (page 163 of the RFC).

USMAN>> You have a point there - agree 100%

I think a snapshot of the actual process of TE FA advertisment into IGP is probably what I need.

From your description, It seems to me that TE FA would be advertised in IGP as a link between RID of head-end and RID of tail-end (even if head-end and tail-end are physically multiple hops away).

And once TE FA gets advertised into IGP, the head-end advertises the adjacency to other IGP peers but if tailend has no adjacency to the head-end its a violation of RFC 2328 with potential risks.

Is this a correct understanding ?

thanks,

Usman

[IVAN PEPELNJAK]

#1: Sure, it's LSR. Fixed.

#2: If you want to stay within the rules of OSPF or IS-IS RFC and have unidirectional LSP in topology database, the tail-end LSR has to lie. As I said, a bad idea, but the only thing you can do unless you violate OSPF rules. It's so clearly stupid that we can be almost sure it's not used by anyone.

#3: Your understanding is correct. To get a more in-depth view, just take 3 routers, configure MPLS TE tunnel and forwarding adjacency between two of them and perform "show ip ospf database router [ self-originate ]".

[ulatif]

"........the tail-end LSR has to lie. As I said, a bad idea, but the only thing you can do unless you violate OSPF rules. It's so clearly stupid that we can be almost sure it's not used by anyone."

Ivan - you wont believe it but we have a vendor (in the top 3 vendors around the world) who is currently doing that and i caught them in the act....

This is why I initiated the discussion..

Sorry cannot name the vendor on the forum.

[ulatif]

Ivan, just so I have a correct understanding of our discussion, if assume we have a special case of one-hop TE tunnel, dont you think the head-end can install TE FA into IGP because the tail-end even though it doesnt have a TE FA back to head-end still has an OSPF adjacency to the head-end via the physical link ?

[IVAN PEPELNJAK]

For a one-hop tunnel, you don't need FA.

[ulatif]

Sorry ... You mean we dont need FA in the opposite direction from tail-end to head-end right ?

Or do you mean we dont need FA at all for one-hop TE ?

because if its the latter, then I have lost you... coz in my understanding, we can still use a higher IGP cost on physical interfaces and have TE FA used for forwarding on one-hop TE.

Anyway, I appreciate your feedback on this issue.

[ulatif]

Ivan - correct me if I am wrong:

I see from the "show ip ospf database router self-originate" that the TE FA gets advertised in the following format into IGP LSDB:

Example:

R2#

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 10.3.3.3
     (Link Data) Router Interface address: 0.0.0.9
      Number of TOS metrics: 0
       TOS 0 Metrics: 1

And if the tail-end (whether or not its physically one-hop away or multiple hops away) has a TE FA configured in the reverse direction, I see:

R3#

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 10.2.2.2
     (Link Data) Router Interface address: 0.0.0.9
      Number of TOS metrics: 0
       TOS 0 Metrics: 1

Normally, a real physical link in OSPF LSDB appears with "Link Data:" either as a network mask or as a meaningful IP address.

It appears however that for a TE Forwarding adjacency (which is not a real physical link but more like virtual link between two peers), the "Link Data:" contains some form of a common identifier in the format of 0.0.0.xxx where both head-end and tail-end share the same identifier.... and this is probably how IGP ensures that there is a two-way adjacency between the peers...

It'll be interesting to know what rules the peers follow when selecting this common 0.0.0.xx identifier for the mutual FA between each other.

[IVAN PEPELNJAK]

Read section 12.4.1.1 of RFC 2328

http://tools.ietf.org/search/rfc2328#page-130

[ulatif]

Thanks for the direction.

So its the MIB-II ifIndex value - this obviously then is a local decision and not mutual.

So I am now clear on the fact that for multi-hop TE FAs, a bidirectional adjacency check is necessary but i am still not quite clear on a one-hop TE coz for a one-hop TE, if head-end has TE FA configured and tail-end has no TE FA confiured, the tail-end still has a p2p adjacency to the head-end via the physical link (assuming physical interfaces are defined as type p2p), so if a vendor is not performing bidirectional TE FA check for one-hop TE, is that still not conforming to 2328 ?

IMO, I think they would/should still be non conforming to 2328 because TE FA is advertised as a seperate adjacency by the head-end (on top of the normal p2p adjacency that exists over physical interfaces) and requires a corresponding seperate reverse adjacency from tail-end to head-end - but then this opens the question of how head-end/tail-end and other IGP peers keep track of which adjacency is relevant to which TE tunnel FA ?? because hypothetically speaking, two routers connected back-to-back can still have many one-hop TE tunnels configured between them (for different purposes etc).

[ulatif]

Ivan - following on from our discussion:

I connected the 3rd party devices to a Cisco router in the following topology:

C3800-------R1----------R2

                  |             |

                  -----R3-----

Router IDs are as follows:

R1 = 192.168.128.101

R2 = 192.168.128.102

R3 = 192.30.102.1

The link IPs are as follows:

R1 <-> R2 = 192.168.168.100/30 (p2p)

R1 <-> R3 = 192.168.168.104/30 (p2p)
R3 <-> R2 = 192.168.168.108/30 (p2p)

I only had one explicit-path based TE tunnel going from R1->R3->R2 so head-end was R1 and tail-end was R2 with FA configured.
There was no reverse TE FA from R2 back to R1.

The C3800 still installed the TE FA half-adjacency into its IGP, see below:

Below is a snapshot of LSDB (self-originated router LSA) from Router-1 (3rd party):

         OSPF Process 1 with Router ID 192.168.128.101
                         Area: 0.0.0.0
                 Link State Database

  Type      : Router
  Ls id     : 192.168.128.101
  Adv rtr   : 192.168.128.101 
  Ls age    : 134
  Len       : 120
  Options   :  E 
  seq#      : 8000037b
  chksum    : 0x87f9
  Link count: 8
     Link ID: 192.168.128.101
     Data   : 255.255.255.255
     Link Type: StubNet     
     Metric : 0
     Link ID: 192.168.128.100
     Data   : 0.0.0.14    
     Link Type: P-2-P       
     Metric : 10
     Link ID: 192.30.102.1
     Data   : 192.168.168.105
     Link Type: P-2-P       
     Metric : 50
     Link ID: 192.168.168.104
     Data   : 255.255.255.252
     Link Type: StubNet     
     Metric : 50
     Link ID: 192.168.128.100
     Data   : 192.168.168.101
     Link Type: P-2-P       
     Metric : 100
     Link ID: 192.168.168.100
     Data   : 255.255.255.252
     Link Type: StubNet     
     Metric : 100
     Link ID: 192.168.192.2
     Data   : 192.168.168.113
     Link Type: P-2-P       
     Metric : 100
     Link ID: 192.168.168.112
     Data   : 255.255.255.252
     Link Type: StubNet     
     Metric : 100

From the C3800, I see the below:

  LS age: 308
  Options: (No TOS-capability, No DC)
  LS Type: Router Links
  Link State ID: 192.168.128.101
  Advertising Router: 192.168.128.101
  LS Seq Number: 8000037B
  Checksum: 0x87F9
  Length: 120
  Number of Links: 8

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 192.168.128.100
     (Link Data) Router Interface address: 0.0.0.14
      Number of TOS metrics: 0
       TOS 0 Metrics: 10

And the C3800 even calculates its routing path via this unidirectional TE FA:

CIS375024T.1#show ip route 192.168.128.100
Routing entry for 192.168.128.100/32
  Known via "ospf 1", distance 110, metric 11, type intra area
  Last update from 192.168.168.113 on Vlan33, 00:07:02 ago
  Routing Descriptor Blocks:
  * 192.168.168.113, from 192.168.128.100, 00:07:02 ago, via Vlan33
      Route metric is 11, traffic share count is 1

The outgoing cost on the C3800 towards R1 was set = 1 and the Loopbacks on 3rd party routers have a cost = 0.

So the question now is why did C3800 install this unidirectional FA into its IGP and even ran SPF based on this ??

(I'll attach a full "show ospf database " from the devices in the above topology.)

I can provide any further outputs if necessary but to me it doesnt quite look like even the Cisco devices are checking for a bidirectional adjacency ?? or maybe i am missing something obvious here.

[Richard Gallagher]

Hey Usman,

Long time no see buddy! OK, so had a look at this and here's the what I read and then what I see......

From what I read this is how is should work:

Let's say we have the following topology:

[A] ---- [B] ---- [C]

- Single hop between tunnel end points - One FA tunnel is sufficient (Tunnel AB) – with direct link meets TWCC (two way connectivity check) requirement
- Multiple hops – At least two FA tunnels in opposite direction (Tunnel AC + Tunnel CA). FA is unidirectional so that two FAs, one from head-end router to the tail-end router and vice versa, are required to pass OSPF TWCC and can be used to route traffic.

But I see in reality is not like this, I lab'd this up today and even for one hop tunnel I need bi-dir tunnels for it to be used by IGP, here is the config from one router, it's a one hop tunnel to it's neighbor:

P1#sh run int e1/0
Building configuration...

Current configuration : 104 bytes
!
interface Ethernet1/0
ip address 192.168.7.1 255.255.255.252
mpls traffic-eng tunnels
mpls ip
end

P1#sh run int tun 1
Building configuration...

Current configuration : 223 bytes
!
interface Tunnel1
ip unnumbered Ethernet1/0
ip ospf cost 2
tunnel mode mpls traffic-eng
tunnel destination 192.168.7.2
tunnel mpls traffic-eng forwarding-adjacency
tunnel mpls traffic-eng path-option 1 dynamic
end

P1#sh mpls traffic-eng forwarding-adjacency 192.168.7.2
  destination 192.168.100.4, area ospf 1  area 0, has 1 tunnels
    Tunnel1     (load balancing metric 0, nexthop 192.168.7.2)
                (flags:  Forward-Adjacency, holdtime 0)
P1#

P1#
P1#sh ip route | inc Tunnel1                          
P1#

But then when I no shut a tunnel pointing back we get the IGP using it:

P1#sh ip route | inc Tunnel1
O     192.168.6.0/24 [110/12] via 0.0.0.0, 00:00:01, Tunnel1
O     192.168.9.0/24 [110/12] via 0.0.0.0, 00:00:01, Tunnel1
O     192.168.10.0/24 [110/12] via 0.0.0.0, 00:00:01, Tunnel1
O     192.168.12.0/24 [110/12] via 0.0.0.0, 00:00:01, Tunnel1
O        192.168.100.4 [110/3] via 0.0.0.0, 00:00:01, Tunnel1
O        192.168.100.6 [110/13] via 0.0.0.0, 00:00:01, Tunnel1
P1#

And for completeness of course for multi-hop tunnels bi-dir tunnels are required for the FA to be used by the IGP