N5K+L3 FEX limit

[Roman Rodichev]

Latest N5k NXOS allows for 16 FEXs on one N5k with L3 module. Let's say I have two N5ks with L3 modules and with 16 N2ks FEX'ed up to each N5k (total 32 N2ks). No vPC between N5k and N2ks. N2k-1 connects to N5k-1 and N2k-2 connects to N5k-2.

Can I still do vPC between two N2ks and a server?

[Lucien Avramov]

You need to have vPC between the two N5ks to do a port-channel on two different FEX.

Now, what you can do in the scenario you describe is a regular port channel from server to N2k-1 with two ports for example.

[Roman Rodichev]

vPC between 5Ks, yes, but I meant to say that 2Ks won't use 5K's vPC, and each 2K would connect to its individual 5K.

My question is regarding FEX limit with L3 module. In my topology, I can have 16 FEX on 5K-1 and 16 FEX on 5K-2 (total 32), correct? ... and still be able to vPC servers across two 2Ks (one 2K going to 5K-1 and another 2K going to 5K-2), correct?

If I were to vPC each 2K to both 5Ks, then my total limit of 2Ks would drop to 16 instead of 32. Is this correct?

[Roman Rodichev]

Lucien, just to confirm, just because I have vPC enabled on the two N5Ks (w/L3), even if I have each N2K single homed to one of the two N5Ks, my overall limit remains at 16, not 32, correct?

[Roman Rodichev]

I opened a tac case, and was told the following:

"16 for each N5k w/ L3 is supported. So in this scenario, all 32 FEXs can be supported by two N5k. And you can enable vPC on the FEX host ports."

So far I had four Cisco folks tell me two different things: two said 16 and two said 32.

[allan.thomas]

Hi Roman,

Firstly we need to substaniate which 5K series you are refering to, simply because 5000 and 5500 support different L2 capabilities when it comes to the number of FEXs supported.  Please refer to following link which stipulates the configuration limits of either:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration_limits/limits_521/nexus_5000_config_limits_521.html

You will notice that version 5.2.1 the 5000 supports upto 12 FEXs whereas the 5500 supports 24 Fex.  The main design consideration regarding FEX deployment is whether you require Straight-Through connectivity or Active-Active, as the drawback will be the number of Port-Channels supported.

If I understand correctly I believe that you are looking at Active-Active where each FEX is singularly attached to a N5K (no vPC towards the N2K), as you will only have one link this not technically using a hardware Port-Channel. So technically you should be able to connect 12 or 24 FEXs per N5K depending on Model.

In this scenario you can configure vPC between the N2Ks, however will still require a vpc peer and keepalive link between N5Ks.  Please review at the DC SRND particularly around the deployment models I was refering to 'Straight-Through' or 'Active-Active.

Also take a look an older post of mine:

https://supportforums.cisco.com/message/3058829#3058829

Regards

Allan.

[Fabrizio Dall'Ara · Accepted solution]

Hi
The limit is 16 FEX managed by a Nexus 5500.
if the Fex are dual-homed through vPC to a pair of N5500 each N5k will "see" 16 FEX hence the max number of fex is 16

In the other case when the fex are connected in straight through mode each N5k will manage 16 distinct fex hence the max number of 2x16

But again this is a theroritical limit has you will also have to consider other sizing factors like then maximum of logicaĺ ports supported on a N5k

Hope this helps

Fab

Sent from Cisco Technical Support iPad App

[Roman Rodichev]

Finally had this confirmed from multiple sources, and Fab is correct too (I'll mark it as answered).

Pair N5K+L3 16+16 FEX single homed to N5Ks

Pair N5K+L3 16 FEX dual homed to N5Ks

This is what I was looking for.

[allan.thomas]

Hi Roman,

Yes this is as per the document link that I referenced within my post, and as I said depending on you TOR design this is restricted to 12 and 24 FEXs respecitively for L2, or in your case any my omission 16 or 32 FEXs for L2/L3 combined functionalilty.

Regards

Allan.