PE->PE MPLS TE, massive configuration works required

[networknoobs]

Hi, I've a customer with 4 primary sites (each primary site has 1 pair of P-routers), then we've 10 remote sites (each remote site has 1 pair of PE routers).

So that makes up to 8 P-Routers and 20 PE-Routers. Things get messy here, we've got a request to configure TE for PE-PE, which effectively there are 20*(20-1)=380 tunnels to be created among all the sites as it is a full-meshed network.

If really we go all the way and configure all the tunnel interface, I've seen some config example whereby we set "ip rsvp bandwidth" based on the actual transmission bandwidth on all the transit link, then reserve the bandwidth using "tunnel mpls traffic-eng bandwidth <allocate-bandwidth>".

Does that means if the P-routers has no sufficient bandwidth then the tunnel would not be established? Or they will use the bandwidth adaptively?

If talking about the dynamic TE tunnel where we let IGP select the path, would that means that even one of the link between P-Routers has been congested, it would not find another path dynamically?

I don't wish to go to an explicitly defined path tunnel as configuration maintenance would be killing the operation team, we're fairly new to the TE and its a big headache as of the moment.

I'd appreciate some suggestion, whether to go for dynamic TE (380 tunnels to be created among the 10 sites) or any other better option, if I've the tunnel with all the reserved bandwidth, how would the QoS would come into the picture, as 1 single router has 19 tunnels to every other routers among the 10 sites, so I'll need to know exactly from SiteA-PE1 to SiteZ-PE1 bandwidth required and for the rest of it?

Sorry if my question not really clear as I've been reading through lots and lots of documents and sometimes that messed up the brain

Would be appreciate to see some suggestion though, thanks

[Giuseppe Larosa]

Hello Creed,

>> Does that means if the P-routers has no sufficient bandwidth then the tunnel would not be established?

yes, in that case the RSVP TE reservation fails.

important: MPLS TE tunnels are unidirectional they subtract resources from the RSVP pool outbound only.

>>>Or they will use the bandwidth adaptively?

you can think of using tunnels with auto-bandwidth.

However, when I performed tests on MPLS TE with auto-bw, it was some years ago, CAC was only done at the admission level: once the tunnel is up you could put whatever traffic rate over it.

>> so I'll need to know exactly from SiteA-PE1 to SiteZ-PE1 bandwidth required and for the rest of it?

No, see below

So in general the tunnel bandwidth can be different then that of the real traffic that will use it.

You can use auto-bw or low BW requirements on the tunnels to have them able to setup.

Each tunnel can have multiple path options and you can pre-configure also some explicit paths if needed and the corrective action will become that of changing the path priority to move traffic quotas.

Setting up MPLS TE is quite complex.

To get full advantage of MPLS TE you need to have SDH/Sonets so to use SDH fast detection reroute capabilities.

You can also think of using IP fast convergence or BFD with your IGP protocol.

But probably the customer has made the choice already.

another important aspect is that you can decide to use MPLS TE only for some critical services mapped over some traffic classes.

>> If talking about the dynamic TE tunnel where we let IGP select the path, would that means that even one of the link between P-Routers has been congested, it would not find another path dynamically?

the IGP you use has TE extensions it should be able to reroute if best path link has ended the RSVP pool (again this can be not the same as user traffic)

>> QoS

it is implemented on the physical interfaces with DiffServ model you will need to mark yor traffic.

there is also now Class Based MPLS TE.

http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/gscbts.html

Hope to help

Giuseppe

[networknoobs]

Hi Giuseppe,

Can I safely says that the QoS is not apply in the tunnel interface, instead it'll still as usual apply at the outgoing physical interface?

Says I've 3 types of traffic classes, and if CBTS is used, I'll need to have 3 tunnels established between PE instead of 1 tunnel. That would make the configuration even more complex.

I just wish I could put one scenario so that it could be easier to understand:

Provided we have 20 devices and have full-meshed TE among all the devices, so PE1 would have a tunnel to PE2 - PE20, that's 19 tunnels in total, so QoS would not apply in each and every tunnel, I'd still apply QoS @ the physical outgoing interface, it means QoS came in before it get routed into the TE tunnel, so for the class-based TE, i might proposed for some services that is bandwidth intensive and have it define an explicit path to route through the secondary P-Router plane which in normal day, would be under utilized.

The QoS is the prime, and the RSVP bandwidth in the TE is irrelevant if I've QoS in my outgoing physical interface right? For example in TE tunnel I just specify 1MB for application_A, but in QoS I allowed 10MB, would the tunnel takes 10MB?

Thanks for the clarification, I'm staying up late till now and hope I can find some easier solution, I've read something about Automesh and now I gotta find some document about that, thanks anyway, you've been real helpful.

[Giuseppe Larosa]

Hello Creed,

>> Can I safely says that the QoS is not apply in the tunnel interface, instead it'll still as usual apply at the outgoing physical interface?

I do agree on this

Class Based traffic selection is a way to put the most critical traffic on a Tunnel with the highest rights in term of priority and so on.

>> The QoS is the prime, and the RSVP bandwidth in the TE is irrelevant if I've QoS in my outgoing physical interface right? For example in TE tunnel I just specify 1MB for application_A, but in QoS I allowed 10MB, would the tunnel takes 10MB?

yes, this is what I meant

Hope to help

Giuseppe

[networknoobs]

Hi Giuseppe,

Thanks for the clarification, since the item1 and item2 are clarified, what I need to concentrate now is to find a way to create automesh tunnels.

What really concern me is, in actual fact I should configure the RSVP bandwidth in TE as exact as possible, else all TE tunnel would be UP and traverse through a congested link in my primary P-Plane.

Let's say what I've configured in the TE tunnel is auto-bw and dynamic, then it'll still act as what IGP bring? just follow the shortest path? If there's 10 dynamic TE with Autobw traversing a P-P link of 100Mbps, if the 100Mbps is congested, would the TE smart enough to find other way to established the TE dynamically so to avoid the congestion?

Thanks

[Giuseppe Larosa]

Helo Creed,

I've reviewed our test report of some years ago.

The Auto-bw feature is able to track the traffic over the tunnels so it will reroute some tunnels when not enough resources are available on the current path.

Probably the priorities and affinities attributes will play a role on deciding what tunnels to reroute.

The tests were made using 4 routers connected in a square with double links between each others.

It was a functional test on sofware based routers like C7200s and C7500s.

An example configuration is the following:

interface Tunnel1

ip unnumbered Loopback0

no ip directed-broadcast

ip route-cache distributed

load-interval 30

tunnel destination 172.16.60.2

tunnel mode mpls traffic-eng

tunnel mpls traffic-eng priority 1 1

tunnel mpls traffic-eng path-option 1 dynamic

>>tunnel mpls traffic-eng auto-bw frequency 360 max-bw 6600 min-bw 3300

end

In global config the sampling frequency was:

mpls traffic-eng auto-bw timers frequency 120

Notice that inside a single TE you can override the timers configured in global configuration and you can specify a minimum and maximum for the auto-bw calculation.

The modified version of the routing protocol (OSPF or IS-IS) is able to recalculate the amount of used RSVP bandwidth on the links.

Scalability requires to use timers that are not too low to avoid to generate too much overhead in the signalling plane

It would be nice to build an auto-mesh of auto-bw tunnels.

Hope to help

Giuseppe

[networknoobs]

Hi Giuseppe,

The auto-bw can reroute tunnels provided we configured the auto-bw correctly with the traffic requirement. That's all right. But with automesh tunnels I can safely says, from 1 PE to another PE, there's only 1 tunnel formed, is that possible for some particular important sites, with the automesh tunnels in place, can I manually create additional tunnels to route certain application via this newly created tunnels between sites?

So it would go like this, all applications go to the tunnel created by automesh, while there's another tunnel I'll have applicationA going through it, I knew CBTS might be able to solve this but I've seen comment saying CBTS and automesh would not work together.

Is that true?

And even with the automesh tunnels created among sites, if we configure in the auto-tunnel for the auto-bw configuration, shall the PE routers went down and back up again, the newly reform automesh tunnels would still retain the configuration? or it'll revert back into default?

Says PE1 has 19 tunnels automatically created, and tunnel 1 is destined to PE2, would it possible if the PE went down and when it came back up again, the tunnel 1 is destined to another PE?

Thanks, you've been great help. I'm still figuring out a way to make the auto-mesh tunnels work in my dynamips, I've opened another thread in this forum see anyone could lend a helping hand.

Thanks again

[Giuseppe Larosa]

Hello Creed,

the automesh feature is not compatible with CBTS and with path protection:

http://www.cisco.com/en/US/docs/ios/mpls/configuration/guide/mp_te_path_prot.html#wp1043332

Do not configure path protection on an automesh tunnel template because the destinations are different and you cannot use the same path option to reach multiple destinations.

the mesh tunnels need to use autoroute announce.

http://www.cisco.com/en/US/docs/ios/mpls/configuration/guide/mp_te_autotun_mesh_ps6017_TSD_Products_Configuration_Guide_Chapter.html

You cannot configure a static route to route traffic over autotunnel mesh group TE tunnels. You should use only the autoroute for tunnel selection.

However, among the restrictions is not present the configuration of manually configured tunnels so you should be able to add them.

There is also another feature that is similat to automesh but provides an automated mesh of backup tunnels

check:

http://www.cisco.com/en/US/docs/ios/mpls/configuration/guide/mp_te_autotunnel_ps6017_TSD_Products_Configuration_Guide_Chapter.html

But in this case the tunnels are not end-to-end but what is needed to provide link and node protection to manually configured tunnels

Automesh is very attractive for a rapid startup of MPLS TE services.

You can add some manually configured TEs from time to time.

Hope to help

Giuseppe

[networknoobs]

well, seems like there's some restriction on the automesh tunnels, good new is I've already got the tunnel up and running, I was thinking to configure two auto-template, so the second auto-template with lower priority would serve as a backup tunnel shall the primary tunnel failed, is that a feasible solution?

And I'm still exploring whether there's a way to have 1 particular type of traffic get routed into my manually configured TE (with explicit path) among few major PE routers, while all the rest of the traffics goes into the default auto-mesh tunnels?

Is it workable solution?

[networknoobs]

Hi, I just had a discussion among team mates, we find auto-mesh tunnels does not really add value as these auto-mesh tunnels would just follow IGP anyway, the obvious advantage I foresee now is the one-hop from PE-PE due to the tunnel interface. (provided I disable the mpls ttl thingy).

If we're just enable "tag-switching ip" on all the interface, we'll have the LSP automatically created anyway, is it any different compared to the auto-mesh tunnels? If I have the LSP in default mode, does it came with FRR and path-protection shall the LSP went down, afaik, the LSP fail-over will depends on the convergence of IGP, but I can further tweak on the BFD and IGP convergence time to make it faster fail-over.

End of the day, I'm thinking for a start, just run the default LSP (which by just enabled tag-switching ip) in all the interfaces, then I'll have the 3 main sites to be created a manual TE tunnels.

If that's the case, can CBTS works, means the TE tunnel will be configured with "mpls traffic-eng exp ", means the traffic that match the EXP will get routed into the tunnel, while rest of the traffics will follow the normal LSP created dynamically.

Hope you understand what I'm trying to convey.

Thanks

[Giuseppe Larosa]

Hello Creed,

>> but I can further tweak on the BFD and IGP convergence time to make it faster fail-over.

This is reasonable.

Instead of using auto-mesh you use MPLS forwarding with LDP LSPs.

Then you add a few of MPLS TE tunnels for specific traffic classes that you want to protect more.

But:

If a given EXP value is not configured on any of the tunnels in the selected set:

-And only one of the tunnels in the selected set is configured as a default, CBTS maps the EXP value onto that tunnel.

-And two or more of the tunnels in the selected set are configured as defaults, CBTS arbitrarily maps the EXP value onto one of these tunnels.

-And no tunnel in the selected set of tunnels is configured as a default, CBTS does not map this EXP value onto any specific tunnel. Instead, CBTS performs CoS-unaware load balancing of that EXP information across all tunnels in the selected set.

from

http://www.cisco.com/en/US/docs/ios/mpls/configuration/guide/mp_te_tun_select_ps6017_TSD_Products_Configuration_Guide_Chapter.html#wp1056401

So you need two tunnels:

one specific for the EXP value desired

one with default settings to carry traffic with all other EXP values

you need N+1 tunnels if N are the traffic classes you want to use.

And all traffic goes over the tunnel bundle nothing is left on LDP LSPs.

But these tunnels can make use of fast reroute.

Hope to help

Giuseppe

[networknoobs]

Hmm, then i might hit some bottleneck, what I planned was single tunnel for my specific application while the rest of the traffics go to LDP signalled LSP.

But if what you said is true, then I have to use automesh tunnels with manual tunnel, so the automesh tunnels are for rest of the traffics and manual tunnel for the specific traffic I wanted to protect.

I've read automesh would not work with SSO or NSF, all of my devices came with redundant route processor, sigh...

Ultimately, I want my specific traffic (EXP5 for exampel) to go thru a tunnel while the rest of the traffics go to the LDP signalled LSP...thanks anyway, gotta read more.

[networknoobs]

Hi Giuseppe,

I've read some documents about the CBTS, and it stated it'll work in conjunction with non-TE path, and in my case, are those LDP signalled LSP.

So I'll have specific tunnels created manually among sites and only match EXP5 and EXP6 for example, so can I safely says that the EXP5 and EXP6 traffics will get routed into the tunnels while the rest of the traffics will follow non-TE path which is the LDP signalled LSP?

If that's the case I might have an initial solution, as the sites that need to be created with tunnels are around 5 sites, which is around 20 tunnels in total, that reduce significant configuration works and ease the maintenance in the future.

I'm still wondering whether I should go for the Automesh as when Cisco stated it does not support SSO and NSF, I'm not sure what it really meant, and whether LDP signalled LSP would have the same constraint?

If I run Automesh, I can enable FRR in the Auto-Template, but is that just it for enabling the FRR? and how about path protection? would any path created pre-maturely just for the fail-over?

Thanks

[Giuseppe Larosa]

Hello Creed,

my understanding is different:

see

Example 1-Default Tunnel Configured

An operator configures the following parameters on tunnels T1 and T2:

•T1: exp = 5, autoroute

•T2: exp = default, autoroute

If T1 and T2 are next-hop interfaces for prefix P, CBTS maps the packets onto the tunnels in this way:

•Packets with onto T1

•Packets with onto T2

http://www.cisco.com/en/US/docs/ios/mpls/configuration/guide/mp_te_tun_select_ps6017_TSD_Products_Configuration_Guide_Chapter.html#wp1056401

So I think that all traffic with a specific destination P or is mapped on a set on tunnels (and you need one with default to avoid evenly load-sharing of unmapped EXP flows) or goes on the LDPs.

However, this allows an approach in which you build protection for 5 sites of 20 with minimal effort.

For those 5 sites all traffic will be on a full mesh of tunnel bundles (at least two tunnels on each direction between POPx and POPy, the second with the default option).

traffic between the other 15 POPs / sites will use LDP LSPs.

Automesh and CBTS are not compatible as per other links reported in the thread and automesh use autoroute announce.

I'm not sure FRR is supported with automesh.

About SSO and NSF:

these features are important and really useful only in topologies without full device redundancy :

I mean if all devices are two:

two PE routers on each site

two P routers for core in main sites

then you can also stay fine without SSO or NSF.

If your core router is only one chassis with double supervisor then SSO and NSF becomes important to provide the desired level of availability in your network.

Hope to help

Giuseppe

[networknoobs]

Hi Giuseppe,

Thanks for the reply, if that's the case, I'd need to create 4 tunnels per direction, so as follows:

1. tunnels for specific traffic

2. protection tunnel for specific traffic

3. tunnels for default traffic

4. protection tunnel for default traffic

is that right, if i just created tunnel for specific traffic and default traffic, how can I enable FRR with path-protection? As far as i understand, path protection need a separate tunnel interface ain't it?

If there's a easier way, would really appreciate you can point it out.

If I have path1 and path2 in the tunnels, is the path2 considered a protection path?

Thanks again