Ask the Expert: ASR9000 Hardware Architecture

[ciscomoderator]

With Satya Narra and Xander Thuijs 

Welcome to the Cisco Support Community. This is an opportunity to learn and ask questions about software installation, SMU installation, ASR9000 architecture, installing satellite (ASR9000 nv), and troubleshooting of the Cisco ASR 9000 Series Aggregation Services Routers with experts Satya Narra and Xander Thuijs.

The Cisco ASR 9000 Series Aggregation Services Routers product family offers a significant added value compared to the prior generations of carrier Ethernet routing offerings. The Cisco ASR 9000 Series is an operationally simple, future-optimized platform using next-generation hardware and software. 

Satya Narra is a network consulting engineer and specialist in ASR9000 deployments for service provider networks. He is an expert in many technology areas, including IP routing, MPLS, BGP, OSPF, multicast, system architecture, and network design. Satya has a master of science degree in electrical engineering from Wichita State University, Wichita, KS. He started his career as a customer support engineer in TAC/HTTS supporting some of the biggest networks, including Comcast, Citi, Amazon, JPMorgan, AT&T, Vodofone, and others.   

Xander Thuijs is a principal engineer for the Cisco ASR 9000 Series and Cisco IOS-XR product family at Cisco. He is an expert and advisor in many technology areas, including IP routing, WAN, WAN switching, MPLS, multicast, BNG, ISDN, VoIP, Carrier Ethernet, System Architecture, network design and many others. He has more than 20 years of industry experience in carrier Ethernet, carrier routing, and network access technologies. Xander  holds a dual CCIE certification (number 6775) in service provider and voice technologies. He has a master of science degree in electrical engineering from Hogeschool van University in Amsterdam.

Remember to use the rating system to let Satya and Xander know if you have received an adequate response.

Satya might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation in Service Providers community,  sub-community, XR OS and Platforms discussion forum shortly after the event. This event lasts through February 14, 2014. Visit this forum often to view responses to your questions and the questions of other community members.

[Mohsin Sohail]

Hello,

I was hoping you could point me in the right direction to understand the QoS features in the ASR9K.

My use case is very simple. I have a service Provider environment where I have to deploy Traffic Classification ( based on ACL ) and MArking ( DSCP 46 for our VOIP traffic and DSCP 0 for eveything else ) on the Edge ( Our Edge is comprised of 3 seperate locations two of them have ASR9001 and 1 has a 9010 ( in pairs ). The Core is then comprised of various ASR9K variants as well. We have Low Queue Cards.

I am fairly well aquainted with the MQC and know how to create the appropriatte Class-maps, Traffic Policy and apply to interfaces. However, I am not sure about any HW/Architectural  limitations on the ASR9K and furthermore, I dont know the Queueing structure ( for e.g. how could I enable low latency Queuing for DSCP 46 )

Any pointers will be greatly appreicatted. Most of my knowledge is sourced from :

http://www.cisco.com/en/US/partner/docs/routers/asr9000/software/asr9k_r5.1/qos/configuration/guide/b_qos_cg51xasr_chapter_0101.html#ID691

Mohsin Sohail

Thank you

[xthuijs]

Hi Mohsin,

this article https://supportforums.cisco.com/docs/DOC-15592 probably provides for a lot of insight and detail already.

If you need more (and who doesn't ) pull cisco live 2013 presentation ID 2904 with a whole section about QOS.

cheers!

xander

Xander Thuijs CCIE #6775
Principal Engineer 
ASR9000, CRS, NCS6000 & IOS-XR

[Mohsin Sohail]

Thank You Kindly, Looks like I have tonnes to read now

[Mohsin Sohail]

AFter reading and listening to quite alot about the QoS Features on the 9K, I have one more question to ask. Is my understanding correct that without a service policy applied to an interface I cant really see the state of the Queues on a Line Card nor can I see the Packet profiles from a DSCP perspective.

Here is the Context: Currently I have no QoS enabled on my 9Ks which means the default trust behavior is occuring on the interface. I want to currently see what raffic is hitting what Queues and what DSCP markings are coming in ( per port,per queue or mayebe even a rate ) - Its kind of validating what I already know is happening( which is packets are just hitting the Normal Queue ). Lets call this a sanity check

Then once I apply my Service Policy ( Classify packets per an ACL, mark per the ACL classification and Place in Queue per markings ) - I would then again want to run the same sanity check to see what happens.

Mohsin

[xthuijs]

Mohsin; without a qos policy applied there really no queue other then the interface default queue. so there is no priority separation at that point.

In order to find out what the distribution is of the different TOS/DSCP's then you'd need to define a qos policy-map that matches all those different values you're interested in and basically do 'nothing' in the policy-map class definition.

That way you can find out from the pmap counters what the different rates per class are.

Note that you dont necessarily need a "queue" in order to do that verification. You only need that classification.

Priority and bandwidth create that queue on egress, bandwidth on ingress creates a queue. Priority on ingress just provides that marking, but doesnt instantitate a queue.

A queue instantiation provides for per class Q'ing in case of congestion, where as a different policed classes will use the same Queue ID and if the interface is congested there is no guarantee per class what makes it through as everything uses that same queue.

regards

xander

[DEEPAK ARORA]

Any plans to make ASR9K, CRS & NCS series a digestable piece as well for for people newly introduced to platform. Now don't get me wrong but there are lot of missing pieces when it comes to knowing these platforms.

> No Books.... which is perhaps always the best way to start with new things

> Self Placed Learning Material ...Maybe PEC

Hard to understand the piece like same line card in two flavors as transport or service

or maybe what should be criterias to help me decide among CRS Vs ASR9K Vs NCS

What are different features these platform support whcih regular or old ones like 7600 etc dont

How redundancy mechasims works within these platoforms, QOS etc

Here is good example about what I am trying to say:

http://www.ine.com/all-access-pass/training/playlist/ccie-data-center-nexus/nexus-hardware-architecture-100121011.html

I can find ASR material in bits and pieces from Cisco.com or Cisco Live. But As a Engineer I prefer single point. Also Cisco DOC Wiki might be another place where you may want all things to put together.

Make it friendly and it will bring more friends . Don't make it look like Rocket Science

HTH...

Deepak Arora

Evil CCIE

http://deepakarora1984.blogspot.com

[xthuijs]

hi deepak, that is great feedback.

I totally recognize the need for documentation and learning capabilities. And there are things in motion for that big time.

The book portion is interesting you note, because when we wanted to publish a comprehensive book about XR the publisher declined that because based on their "data" there was no interest. So I guess you proved them wrong .

Anyways, a book may not happen the way I had hoped and I guess you had wished for either. So my strategy is to leverage the XR OS support forum, blogs, documents and Q&A as the vehicle to get as much information out there as possible in bite size pieces pertaining to a particular technology (area) or design.

In the last year or two we have put a tremendous amount of technotes out there based on what we are learning from suggestions received like yourself.

Because many of the folks contributing are engineers whose primary focus is not documentation, I can't just have them write stuff for the sake of writing, so I need that direction in order to understand where the different topics of need lie.

Coming soon the support forums will get an overhaul into a new style based on open source technology which likely provide for more interaction capabilities then what is there today.

As for learning capabilities; soon we'll have XR-VR's avaiable to build and test configurations with on soft routers

which provides for a great deal of learning already.

For the technical pieces; I'd like to have some direction and a few topics for you that you are missing so we can get that piece out there (via the support forums).

thanks!!

xander

Xander Thuijs CCIE #6775
Principal Engineer 
ASR9000, CRS, NCS6000 & IOS-XR

[feene1]

Hello,

With IOS-XR 5.1.1 on the 9K I see that there is a QOS offload feature.

http://www.cisco.com/en/US/docs/routers/asr9000/software/asr9k_r5.1/qos/configuration/guide/b_qos_cg51xasr_chapter_0101.pdf

I think I am reading this right - are we now able to shape outbound on a client 9000V port and not use a queue on a TR card?  Essentially "offload"?

Any additional insight to what this features provides would be much appreciated. 

[xthuijs]

hi Feene1,

the qos offload helps with the satellite fabric extension connection oversubscription.

If the ICL (inter chassis link) is oversubscribed for whatever reason, then the qos offload will help

on ingress policing more or less on the sat ports to prevent the oversubscription to happen.

cheers

xander

[tenaro.gusatu.novici]

Hi there,

I'm new in this but hopefully you can give me quick hints why/when ASR9k is better choice than CRS-X or NCS6K series?

Thanks,

Tenaro

[xthuijs]

hi Tenerao,

it somewhat depends on where the device fits in the network; If you are looking at the edge with a need for edge like features, then I would say, you probably are looking at an ASR9000.

If you need a core device without an extensive need for flexibility and features, then probably the CRS/NCS6k is what you want to be focussing on.

There is overlap between the 3 products you mention, which is awesome because it makes them more versatile, but I can also see the confusion as to what to choose when.

Without having a lot of background about your network (needs), it is hard to provide a crisp and sound answer.

I would recommend to consult with your account team and have a Technical Marketing Engineer and a Product Manager piped in to give you the best advice for strategy on this one.

regards

xander

[Mark Pace Balzan]

Hi,

Ive followed the cisco live presentations which are great to explaining the architecture of the ASR9k.

I'd like to ask few questions to ensure Im getting things correctly:

Q1. I understand that with dual RSP440 in a system, we have total fabric capacity of 220G in / 220G out between each LC and each RSP, so total aggregate of 880G assuming both RSP are available.  For the typhoon 24x10GE card, we have 240G in / 240G out from the LC, so this is fine with dual RSP, but if we have one RSP, then we have very slight oversubscription right ?  Probably this is negligible in practical use as its unlikely all interfaces will simultaneosuly running at full rate but just want to ensure im getting this right.

Q2. For the MOD160 card, is the total capacity of this card 160G making it a rather low capacity card compared to the 24 and 36x10GE ?   Im not sure I follow since this is far less than even a single RSP can handle but seems to be the newer cards launched ?

Q3. Specifically for the ASR9001, which of the fixed 10GE onboard ports are mapped to which NPU internally and with which bay ?  Would it make sense to have 'core uplinks' from the fixed 10G ports coming from separate NPUs ?

thanks

Mark

[xthuijs]

Mark:

Q1) Correct, on single RSP, you are missing 20G, the way that it works is each of the 4 FIA's gets 5G back pressure to maintain fairness. So if you dont need the 24 interfaces immediately, it is best to wire them 0-4 then 6-9 etc

so you skip one interface per 2 NPU's so you dont run into any issues on a single RSP.

Q2) It is effectively the same as a MOD-80, which only has 1 NPU per bay, whereas the MOD-160 has 2 NPU's per bay. This MOD160 is useful when you want to use the 2x40G MPA, 8x10G MPA or when pps is a concern to you.

with the 2 NPU's per bay this card will provide top notch pps performance as each NP serves half the interfaces of the bay, which is overkil for most MPA's if you ask me .

Q3) 2 onbard 10G's of the 9001 are mapped to a single NPU. So each NPU on the 9001 serves 2x10G Onboard and an MPA bay.

regards

xander

Xander Thuijs CCIE #6775
Principal Engineer 
ASR9000, CRS, NCS6000 & IOS-XR

[Mark Pace Balzan]

Hi xander,

thanks for the fast and accurate reply

so for the asr9001, do we have fixed SFP+ 0,1 and MPA bay0 on one NPU, and SFP+ 2,3 and MPA bay1 on the other NPU, which is possibly also why the 9001S has one NPU and associated ports disabled ?

thanks

Mark

[smilstea]

Hi Mark,

Correct.

Here is a quick way to check for any LC on any ASR9K. Below I have a 20x1GE MPA and 4x10GE MPA installed in my 9001.

RP/0/RSP0/CPU0:ASR9001-A#show controller np ports all loc 0/0/CPU0

Thu Feb 13 09:07:03.220 EAST

                Node: 0/0/CPU0:

----------------------------------------------------------------

NP Bridge Fia                       Ports

-- ------ --- ---------------------------------------------------

0  0      0   GigabitEthernet0/0/0/0 - GigabitEthernet0/0/0/19

0  0      0   TenGigE0/0/2/0, TenGigE0/0/2/1

1  1      1   TenGigE0/0/1/0, TenGigE0/0/1/1, TenGigE0/0/1/2, TenGigE0/0/1/3

1  1      1   TenGigE0/0/2/2, TenGigE0/0/2/3

Thanks,

Sam