cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2097
Views
5
Helpful
12
Replies

Is XR x64 Actually a Monolithic OS - A Step Backwards?

philclemens1835
Level 1
Level 1

While the book "IOS XR Fundamentals" provides a lot of information about the 32-bit QNX microkernel operating system upon with classic XR is based, there is no such information available regarding 64-bit XR.

 

Since it is based on Linux, this leads me to believe that we have abandoned the microkernel concept, and its security and process advantages.  Instead, opting for a monolithic OS, where the only real difference between XR and XE becomes the command set.

 

Is this the case?  If so, is the motivation for going to 64-bit the ability to take advantage of the 3rd container for devops-related capabilities?

 

Yes, I'm being somewhat dramatic, hoping to get a reaction from somebody in the BU, but it is important to know and understand the platform when you're tasked with its support.

 

1 Accepted Solution

Accepted Solutions

So the xr 64-bit just means that the code is compiled with 64-bits instead of 32-bits giving us larger room for variables, memory allocation etc.

The underlying kernel is indeed linux based but our linux distro provides for a real-time kernel and scheduler.

 

Sam

View solution in original post

12 Replies 12

philclemens1835
Level 1
Level 1

I wouldn't call this a solution, but this PDF does somewhat address the architecture of 64-bit IOS-XR.

While it does show that 64-bit uses VM's and containers, I'm interested to know that, for instance, if OSPF gets locked up, its process could be restarted without restarting an entire container or VM.  In other words, do we retain that unique quality of IOS-XR 32-bit?

 

https://www.ciscolive.com/c/dam/r/ciscolive/emea/docs/2019/pdf/BRKSPG-2069.pdf

We still have separate memory spaces, except for those processes that run in shared memory. Unfortunately I do not have the details as to the kernel and how exactly we have maintained the benefits of 32-bit XR from QNX RTOS into Linux kernel. I can say that process management has changed, for instance now most threads will be in a sleeping state when not used, whereas in QNX sleeping was not a thread state, and we would always see threads in a reply or receive state typically.

Let me poke around and see if I can get some answers.

 

Sam

Very cool, Sam.  The fact that you've got eyes on this has me stoked.  Looking forward to seeing what you can find out and share.

 

So the xr 64-bit just means that the code is compiled with 64-bits instead of 32-bits giving us larger room for variables, memory allocation etc.

The underlying kernel is indeed linux based but our linux distro provides for a real-time kernel and scheduler.

 

Sam

Adam Vitkovsky
Level 3
Level 3

My understanding is that the industry is pushing for more decentralization and ability to create a custom install variant that best suites ones needs  -see golden ISO.

I certainly hope that in near future I’ll be able to go as far as installing only protocols I need (ISIS vs OSPF or RSVP-TE vs LDP, etc..) and seamlessly complement these with 3rd party protocols (where this last bit is already available)

 

Although on the question of monolith vs decentralized routing system myself and Saku Ytti we had a really good discussion on juniper-nsp mailing list. Search the archives for “Suggestions for Edge/Peering Router” thread.

While I like decentralization better for reasons I outlined in my arguments in the discussion Saku made a compelling case that in routing systems the robustness coming from decentralization and protected memory spaces doesn’t add much as these systems are all or nothing (either the box has isis+bgp+ldp up and running or if any one of those fails the box is done anyways)

 

adam

 

netconsultings.com

::carrier-class solutions for the telecommunications industry::  

adam

I understand what you're stating as Saku's point, but I do disagree that a system is all-or-nothing.  Specifically in terms of process hangs and restarts.  For example, wouldn't you prefer the OSPF process has the ability to restart independently without reloading the entire router?  Especially in an SP environment.  Even if you can say that, without OSPF, the router is nothing, the time to restart a process is much shorter than a box reload.  If the process is SNMP, then certainly I'd like to have the router reload that process and keep functioning, rather than possibly locking up the router requiring a site visit and reload.

 

It is this process independence that is a primary strength of XR.

 

eXR and XR7 both have golden ISO capabilities.

 

eXR did allow for installing just isis or ospf etc and XR7 which is out on one variant of the NCS540 and coming to a new platform soon will see us going from a handful of packages to a few hundred packages and much more native linux in many areas. As well as the ability to run disjointed releases for packages.

 

I would argue against Saku, true your ospf might be messed up and that affects TE and SR etc, but you can fix ospf easier on xr or in the case of a crash or memory leak an automatic restart with minimal disruption to other protocols. And what about protocols like bfd, ipsla, etc? What if they have a memory leak or crash? Their impact is only to the underlying protocols and interfaces using them and not the entire router. I would rather those processes not impact the rest of my router that is running fine with services. It sounds to me like he is trying to make excuses for juniper not having these OS features :)

And I'm sure you remember the days of IOS not having any of these features and how difficult a process crash etc are to troubleshoot because the whole box goes down.

 

Sam

> As well as the ability to run disjointed releases for packages

Hallelujah! This is exactly what we need, have the utmost flexibility in what packages and of what version we install (same as IT guys have on the server side of things)

 

In junos there are other processes taking care of other stuff like mgmt, or BFD… so the point of my discussion with Saku was more around routing protocols as separate processes (XR) or all of them mangled together in RPD (junos)

At one level I agree with you, as surely if OSPF crashes on XR it’s an easy restart -while on junos it might take the whole RPD monolith with it affecting all other protocols (BGP, etc..)

However if local OSPF crashes on XR there’s a good chance that BGP next-hops will be affected anyways causing massive purge followed by a lengthy recovery.

Also his argument was that the IPC as a result of separate processes (memory spaces) is more a complicated architecture -and complicated as we all know goes hand in hand with fragile/error prone.

 

But as I said I like XR approach way better cause it allows for better feature velocity and flexibility

-AND soon enough will allow me to get one version of ISIS and other version of BGP, which is simply not possible with a monolithic RPD in junos

 

adam

 

netconsultings.com

::carrier-class solutions for the telecommunications industry::  

 

adam

That IPC question was one of the core premises behind this initial post.  In 32-bit XR on QNX, qnet handles that task - the IPC is built in to the base OS.  I'm not so sure that there is an underlying capability in Linux.  I'm anxiously awaiting the yet-to-be-announced/written book, "64-bit IOX-XR Fundamentals".

Just ordered this, written by a Google engineer:

 

"Linux Kernel Development" by Robert Love

 

https://www.amazon.com/gp/product/0672329468/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1

 

 

Just google “ipc in linux” and you’ll find your answers
adam
adam

philclemens1835
Level 1
Level 1

The latest version of the BRKSPG-2900 presentation provides the attached section regarding the architecture of x64 IOS-XR.

 

 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: