Full Routing Table : label-allocation-mode per-vrf vs per-ce NCS5500

[DanielGutierrez615]

What will be the recommended label-allocation-mode   for a network with three NCS55A2 peering with other ISP getting full routes from the Internet (898752 total router).  The default is label allocation per prefix, but when I peer with another provider. I was getting a lot of these messages: See also attached file.   I changed it to label-allocation-mode per VRF, and the logs when away.  My question is what's the recommended setting in this type of scenario?  My understanding is that the NCS55A2 is running out of local labels and that's why it's can allocate a label per prefix.  I thought about extending the local label range from local-block 15000 15999  to local-block 15000 15999 to local-block 15000 1048575, and that should be enough for all prefixes. Or I can change the label-allocation-mode to per CE, and every prefix will get the same label per vrf  per CE. but I don't  know the recommended mode. Advice?

ipv4_rib[1260]: %L2-GRID-4-BANK_FULL : GRID POOL:GLIF(2), BANK 1 FULL. Max size 178128, Curr RIDs 178127.

16:11:02.869 § LC/0/0/CPU0:Jan 13 16:07:35.873 EST: fib_mgr[184]: %PLATFORM-PLAT_FIB-3-HW_PROG_ERROR : HW Programming failed for table,ipnhgroup,key,0xe303d40000e620,unit,0,dpa_trans_id,1945286,failure-reason,NoResrc
16:11:02.870 § LC/0/0/CPU0:Jan 13 16:07:35.876 EST: fib_mgr[184]: ti queue ext 218 Failed to create queue-ext for object for object:[ref:1 count:1 depth:2 ldi:0x3145005098 ext:0x30e50da1e8 type:Encap-shared flags:[mpls ext, recursive, imp, pd shared] flags2:[] extn_flags:[] intn_flags:[orig-pd-share] timestamp:Jan 13 16:07:35.872]

[Harold Ritter · Accepted solution]

@DanielGutierrez615 ,

"per-vrf" and "per-ce" are generally recommended over "per-prefix", just because of the sheer number of labels required in this scenario. "per-vrf" incurs an additional lookup, which may have a performance impact on the NCS55A2, so I would definitely recommend "per-ce".

You may also refer to this excellent document for more information:

https://xrdocs.io/ncs5500/tutorials/ncs5500-routing-in-vrf/

Regards, 

[smailmilak]

Weird question incoming.

How does the router know what a CE router is in per-ce label allocation mode?

[DanielGutierrez615]

Thank you!!. 

[Harold Ritter]

You are very welcome @DanielGutierrez615 . Have a great day

[gixohe9689]

The "label-allocation-mode" is a configuration command in NCS5500 that determines how MPLS labels are assigned.

In "per-vrf" mode, a unique label is assigned to each VRF (Virtual Routing and Forwarding) instance within the network.

In "per-ce" mode, a unique label is assigned to each Customer Edge (CE) device.

The choice of mode depends on the specific requirements of the network and the services it provides. The "per-vrf" mode is more flexible, as it allows for the creation of multiple VRFs, each with its own label space. The "per-ce" mode is simpler and more scalable, as it reduces the number of labels needed for a given network.

[abhsa]

Totally understood the concept but can we have a more scenario specific example here, are we talking about CE and PE having multiple VRFs among them, where per VRF label allocations can be higher than the per CE based.
In case of CE1--- PE1 --- PE2 ---- CE2, per VRF makes more sense. Can you explain the per CE based scenario and how the PE router will make sure these prefixes belong to a certain CE if we have multiple VRFs among them?

[Harold Ritter]

Hi @abhsa ,

> In case of CE1--- PE1 --- PE2 ---- CE2, per VRF makes more sense.

I am not sure why you say that per-vrf makes more sense in this specific scenario. In fact with only one CE, both modes allocate only one label regardless the number of routes coming from the CE.

With the per-vrf mode, the number of labels will never increase.

With the per-ce mode, the number of labels increases by 1 each time you add an additional CE within the same VRF. If you have multiple VRFs between the PE and the CE, you have one label assigned for each CE for each VRF. In order words, if you have n VRFs between the PE and the CE, you would have n labels allocated by the PE.

Please refer to the following document for more information on the different label allocation modes:

https://xrdocs.io/ncs5500/tutorials/ncs5500-routing-in-vrf/

Regards,

[preshalinp]

Hi @Harold Ritter 

quick question.

if you change a vrf from per-vrf to per-prefix, (ASR9K's)
do you need to delete and recreate the vrf,?
example:
PE1 (per-vrf) - 3 loopbacks with 3 different IPs and 1 static with a null 0 next-hop, this is for testing.

PE2 , see all these prefixes with 1 Label, which is for the vrf on PE1,
PE02#show route vrf VRF-TEST 5.5.5.5/32 detail | i Label
Thu Nov 28 18:48:25.661 SAST
Label: 0x61a5 (24997)
Binding Label: None
PE02#show route vrf VRF-TEST 7.7.7.7/32 detail | i Label
Thu Nov 28 18:48:25.661 SAST
Label: 0x61a5 (24997)
Binding Label: None
PE02#show route vrf VRF-TEST 33.33.33.33/32 detail | i Label
Thu Nov 28 18:48:25.661 SAST
Label: 0x61a5 (24997)
Binding Label: None
PE02#show route vrf VRF-TEST 99.99.99.99/32 detail | i Label
Thu Nov 28 18:48:25.661 SAST
Label: 0x61a5 (24997)

now on PE1, i changed to per-prefix,removed the static route and re-added it, deleted and re-created 1 of the loopbacks, i still see the same label allocation, and the vrf on PE1, still shows vrf mode as per-vrf

PE02#show route vrf VRF-TEST 5.5.5.5/32 detail | i Label
Thu Nov 28 18:56:45.618 SAST
Label: 0x61a5 (24997)
Binding Label: None
PE02#show route vrf VRF-TEST 7.7.7.7/32 detail | i Label
Thu Nov 28 18:56:46.103 SAST
Label: 0x61a5 (24997)
Binding Label: None
PE02#show route vrf VRF-TEST 33.33.33.33/32 detail | i Label
Thu Nov 28 18:56:46.598 SAST
Label: 0x61a5 (24997)
Binding Label: None
PE02#show route vrf VRF-TEST 99.99.99.99/32 detail | i Label
Thu Nov 28 18:56:47.079 SAST
Label: 0x61a5 (24997)

PE01#show mpls forwarding vrf VRF-TEST
Thu Nov 28 19:01:41.268 SAST
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24997 Aggregate VRF-TEST: Per-VRF Aggr[V] \
VRF-TEST 0

config is per-prefix:
router bgp 65000 vrf VRF-TEST address-family ipv4 unicast label mode per-prefix

[MHM Cisco World]

can you make new post please 

thanks

MHM

[Harold Ritter]

Hi @preshalinp ,

"per-prefix" is the default allocation mode.

The reason you are seeing the same label for all 4 prefixes is that these prefixes are all local prefixes inside the same vrf on the PE. If you want to see different labels, you would need to connect a CE to the PE and originate the prefixes from that CE via an IGP or BGP.

You could also add static routes inside the VRF and with a next hop belonging to an interface inside that VRF and redistribute the static routes.

router static

vrf VRF-TEST

address-family ipv4 unicast

<prefix/prefix length> <next hop towards the CE>

router bgp XYZ

vrf VRF-TEST

address-family ipv4 unicast

redistribute static

!

Regards,

[preshalinp]

Hi Harold,

We changed to per-vrf on our PEs, but for this specific solution, i wanted to revert to per-prefix.
as a last test i changed the static route next-hop from null to a random NH that was available on another PE, i was able to see the LFIB support my config, I was concerned that the vrf needed to be recreated,

But we are good now, thank you!

PE01#show mpls forwarding vrf VRF-TEST
Thu Nov 28 21:16:45.417 SAST
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24140 Unlabelled 99.99.99.99/32[V] 100.126.36.44 0
24997 Aggregate VRF-TEST: Per-VRF Aggr[V] \
VRF-TEST 0

[Harold Ritter]

You are very welcome @preshalinp and thanks for the feedback