Are there any special procedures when I upgrade NX-OS(standalone) mode to ACI mode of NX-9K?

[moyeonlee]

Hi everyone,

I have two questions.

First, Are there any special procedures when I upgrade NX-OS(standalone) mode to ACI mode of NX-9K? For examples, NX-OS version, Line Cards, EPLDs, Licenses, and so on. And is it possible revert ACI mode to NX-OX mode, too?

Second, I know Fabric module(below, FM)s are paired FM1 and 2, 3 and 4, 5 and 6 respectively when there is 48-Port 1/10G T/F Module in the chassis. In that case, if the FM in slot 2 were to lose, both the links to slot 1 and 2 would be lost because the mux detects the loss of slot 2 and so takes out the traffic to slot 1 also, that is, the slots are paired. If so, what are the advantages of full FM placement? Please share your knowledge with me clear^^

Every your helps will be appreciated.

Paul

[Kanwaljeet Singh]

Hi Paul,

Please see the below link for N9K (standalone) to ACI conversion.

https://supportforums.cisco.com/document/12268066/nexus-9000-standalone-nxos-aci-conversion

Regards,

Kanwal

Note: Please mark answers if they are helpful.

[moyeonlee]

Hi Kanwalsi,

Thank you so much for your info. That's the statement I've wanted. How about my second question? Please share it with me in a customer's perspective.

Regards

Paul

[Tomas de Leon]

Paul,

Can you elaborate on your question on FM Pairing?  Where did you see the information that you are mentioning?

The FM pairing is for connection to the "Fan Tray" modules not the data plane links.  Each line card should have a HG link to an ASIC on the FM.  If an FM is loss, your switching fabric has less "BW" density so there can be a situation of oversubscription on the remaining FMs.  This just depends on the number of FMs & LCs that are installed.

Full population of FMs give you more BW between the LCs and the FMs which may be necessary to get full line rate depending on the type & quantity of LCs installed.

I hope this helps

T.

[moyeonlee]

Hi Tomas,

Thank you for your answer first.

Let's assume simply like this, a chassis 9508 is populated full FM(6)s and 48port 1/10G LC is also inserted into the chassis. In this case, one 40G link of two NFE of an FM connect to two ALEs of the LC, in other word one NFE link of FM connects to one ALE of LC, each FM has two NFEs, and each LC has two ALEs, so the LC has totally 12 * 40G that is equal to 480G. That capacity of BW is suitable for the LC without oversubscription. Meanwhile, one FM goes down. What happens on FM and LC? As you know, each FM has much more capability to provide the additional BW to LC but if losing a FM impact to the operation of LC, what is the benefit of full populating FM cards in customer's perspective? That's the point of my question.

Regards

Paul

[Tomas de Leon]

Paul,

Thanks for the follow up.  Maybe I am not understanding you question or I am not explaining it correctly. Lets try again.

Each LC NFE\ALE link and a connection to each FM NFE.  So if there was 6 x FMs that is 12 links from the each LC NFE\ALE.  So if one FM fails, Each NFE on the LC only has 10 links to the remaining 5 FMs.

The following link describes what I am saying.

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-729987.html

The section on the Data place talks about all this...

I hope this clears this up some.

Cheers!

T.

[moyeonlee]

Your explanation is correct. But if there are 3 FMs in the chassis with the same LC, Each FM links to the LC with 4 * 40G. My question is every FM has the ability to provide the full BW that every LC needs. If one FM goes down, the BW every LC needs should be automatically guaranteed by using additional links between FM and LC in case of populating 5 FMs(one FM fails). Otherwise why do customers pursuit 6 FMs instead of 3 FMs except other 36 port 40G LCs? Does it make sence ? :-)

 Thanks

Paul 

[Tomas de Leon]

Ok, 

Making assumptions here

* 9508
* 3 FMs
* all line cards are 48x 1/10G (N9K-X9564)

You are correct that only 3 FMs are required for full line rate for a fully populated 9508.  So you ask why more then 3 FMs?  Resiliency and Redundancy.  A customer may add a 4th FM for a backup to cover a single FM failure.  All FMs will be used but in case of a failure, the 9508 will be still operate at line rate.

So why 6 x FMs.

* Increased Resiliency and Redundancy with additional FMs
* Additional FMs may required for different LCs if the particular model requires higher BW resources

T.

[moyeonlee]

I'm still curious in two cases. It's not a debate, so I need your fully understanding. Here are two scenarios.

First one, when you populate 3FMs, the slot numbers are fixed, Slot 2, 4, and 6. If a customer need N+1 redundancy where should +1 FM be placed?

Second one, there are mix LCs half and half, for examples 36 port 40G and 48 port 1/10G. the customer pursuit 6FMs for 36 port 40G LCs. One of FMs goes down during the service, 36 port 40G LCs are absolutely degraded on their throughput. How about 48 port 1/10G LC? In this case, each 48 port 1/10G LC has the 10 connections between 5 FMs and its ALE for front end ports. Are they also decreased  in their bandwith? or Do the remain FMs automatically add an additional link to the ALE on each 1/10G LC as if the LCs operate with 3 FMs? 

Thanks in advance.

Paul

[Tomas de Leon · Accepted solution]

Paul,

Sorry for the delay... Responses to your two questions.

1. The FM can go in any of the remaining slots. The line cards have Muxes on them tied to each FM that is "installed" in the switch.

2. You initial question was related to the switches in ACI mode.  As of today, there are limited lines cards that work in ACI mode.  3 FMs provide the necessary requirements for line rate for most most line cards as mentioned earlier.  That said, once you install a N9K-X9736PQ a minimum of 6 FMs are required.  So if you have a failure on one FM, your system BW in the fabric degrades. The BW will be shared across all line cards until the failed FM is replaced.

T.