Is GEM (generic expansion module ) of nexus 56128P hot swappable

[satishmothukri]

Hi All ,

I need to install Generic expansion module on Nexus 56128P switch , which is in production currently .We have critical devices connected to Nexus 56128 P so i would like to know whether generic expansion module is hot swappable or not .

No where it is mentioned that its hot swappable .

Right now we do not have any expansion modules on Nexus 56128P. Only one expansion module will be installed on the switch.

Any precautions to be taken care while installing Generic expansion module on production nexus 56128P switch ? 

Any help would be appreciated ..

[Santhosh S]

Hello,

Nexus 5K supports OIR of GEM/LEMs. You can hot-swap the GEM by following the steps outlined here:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5600/hw/installation/guide/n56k_hig/chap4replacement.html#pgfId-1092929

Best Regards,

Santhosh

[satishmothukri]

Hi Santosh ,

thank you for the information and quick reply .

I have gone through the document before raising my question.

Faulty expansion modules can be replaced with new modules using power off and no power off commands .

But my question is whenever we insert expansion module on nexus 58128P switch ( it does not have any expansion modules on it ), will nexus switch needs a reload to detect expansion module  ?

OR expansion module will be detected as soon as we insert it into nexus 56128 P switch .

Do we need to execute any commands after installing expansion module ?

hope insertion of expansion module will not lead to any outage ..Reason for asking these questions is nexus 56128P are in production switch and Load balancers , Firewalls and core switches are connected to it .

[Santhosh S · Accepted solution]

Hi MSK,
I understand your concern; you will not have to bring down the entire switch to get it to detect the newly inserted module.
All you have to do is to issue command "no poweroff module <>" after inserting the module.
Just make sure you have adequate power for the new module to boot up. If you have two operational power supplies, you should be good.

Best Regards,
Santhosh

[satishmothukri]

Hi Santosh ,

Thank you very much for the detailed information.

In the cisco document , they would have mentioned that expansion modules are also hot swappable but with clause like power off and no power off options while removing and inserting modules.

have a nice day ..

[Santhosh S]

I might have overlooked a very important detail. Can you tell me which GEM are you planning to install? L2 or L3? It will be easier if you have the Model/PID of the module.

Thanks,

Santhosh

[satishmothukri]

Hi Santosh ,

Part model is N56-M24up2q .

Can you please let me know whether I can install it on 56128P without turning it off ..

As you mentioned , i need to execute no power off module <number> after inserting module into Nexus 56128P chassis , Am I right ?

[Santhosh S]

Hi,

I can confirm, N56-M24UP2Q can be installed without having to power down the Chassis.

The module should get powered up autmatically. If not, you can issue the command "no poweroff module 2" to bring it online.

You might also want to be aware of a known issue with certain batch of M24UP2Q: https://www.cisco.com/c/en/us/support/docs/field-notices/641/fn64106.html

Thank you,

Santhosh

[satishmothukri]

Hi Sanotsh ,

Thank you for the prompt reply.Today night we are going to install mentioned modules on Nexus 56128P switches .  We will see how it goes on ..

[satishmothukri]

Hi Santosh ,

Part model is N56-M24up2q .

Can you please let me know whether I can install it on 56128P without turning it off ..

As you mentioned , i need to execute no power off module <number> after inserting module into Nexus 56128P chassis , Am I right ?

[Wes Austin · Accepted solution]

You dont have to power anything off, just install the module. It will power on automatically.

If the module doesnt power on, execute the command outlined.

[satishmothukri]

Hi 

After installing the module we can see ports available on the nexus 56128P switch.

No nned to execute the command "no poweroff module 2" to bring it online.

Thank you for the quicker response..

[satishmothukri]

Hi Wes / Santosh ,

I have a small query related to fex and IOS loading from parent switch.

We have dual homed connectivity to FEX from nexus 5k as below.

Nexus 5k-1  Eth 1/47 ----> Fex 100  -----> Eth 1/47 Nexus 5k-2 (VPC100 . Port-chhannel 100)

Nexus 5k-1  Eth 1/48 ----> Fex 101  -----> Eth 1/48 Nexus 5k-2 (VPC 101, Port-channel 101 ) 

But by mistake onsite engineer has connected Nexus 5k-1 eth 1/48 to Fex 100 and Nexus 5k-1 eth 1/47  to Fex 101.

Means Ports on Nexus 5k-1 are mismatched as below.

Nexus 5k-1  Eth 1/48  (must be eth 1/47) ----> Fex 100  -----> Eth 1/47 Nexus 5k-2 (VPC100 . Port-chhannel 100)

Nexus 5k-1  Eth 1/47 ( must be eth 1/148 ) ----> Fex 101  -----> Eth 1/48 Nexus 5k-2 (VPC 101, Port-channel 101 ) 

With port mismatch on nexus 5k-1 ,  we have completed FEX configuration on both nexus 5k switches.

Today Onsite engineer will do correction for mismatched ports on nexus 5k-1 , cable will be swapped for ports eth 1/47 and eth 1/48 to have right ports connected to FEX 100 and 101 from Nexus 5k-1.

I would like to know if onsite engineer swaps the cables of fex 100 and 101 on Nexus 5k-1 , will the fex again go for loading of IOS from Parent switch ?

** currently nothing is connected to Fex 100 and 101.

After swapping the cables , what needs to be done so that ports on Fex 100 and 101 will work properly  ?  In case if i want both fex to download ios from parent switch after swapping cables what should be done ? Do I need to remove and connect back uplinks cables at FEX end so that IOS will download from the parent switch again ?

Actual scenario : 

Nexus 5k-1  Eth 1/47 ----> Fex 100  -----> Eth 1/47 Nexus 5k-2 (VPC100 . Port-chhannel 100)

Nexus 5k-1  Eth 1/48 ----> Fex 101  -----> Eth 1/48 Nexus 5k-2 (VPC 101, Port-channel 101 ) 

But onsite has connected cables as below.

Nexus 5k-1  Eth 1/48  (must be eth 1/47) ----> Fex 100  -----> Eth 1/47 Nexus 5k-2 (VPC100 . Port-chhannel 100)

Nexus 5k-1  Eth 1/47 ( must be eth 1/148 ) ----> Fex 101  -----> Eth 1/48 Nexus 5k-2 (VPC 101, Port-channel 101 ) 

Fiber Cables connected to fex 100 and 101 on Nexus 5k-1 will be swapped today so that we have correct ports connected to both fex from both Nexus 5ks. 

[satishmothukri]

Hi Wes / Santosh ,

I have a small query related to fex and IOS loading from parent switch.

We have dual homed connectivity to FEX from nexus 5k as below.

Nexus 5k-1  Eth 1/47 ----> Fex 100  -----> Eth 1/47 Nexus 5k-2 (VPC100 . Port-chhannel 100)

Nexus 5k-1  Eth 1/48 ----> Fex 101  -----> Eth 1/48 Nexus 5k-2 (VPC 101, Port-channel 101 ) 

But by mistake onsite engineer has connected Nexus 5k-1 eth 1/48 to Fex 101 and Nexus 5k-1 eth 1/47  to Fex 102.

Means Ports on Nexus 5k-1 are mismatched .

Nexus 5k-1  Eth 1/48  (must be eth 1/47) ----> Fex 100  -----> Eth 1/47 Nexus 5k-2 (VPC100 . Port-chhannel 100)

Nexus 5k-1  Eth 1/47 ( must be eth 1/148 ) ----> Fex 101  -----> Eth 1/48 Nexus 5k-2 (VPC 101, Port-channel 101 ) 

With port mismatch on nexus 5k-1 we have completed FEX configuration on both nexus 5k switches.

Today Onsite engineer will do correction for mismatched ports on nexus 5k-1 , cable will be swapped for ports eth 1/47 and eth 1/48 to have right ports connected to FEX 100 and 101 from Nexus 5k-1.

I would like to know if onsite engineer swaps the cables of fex 100 and 101 on Nexus 5k-1 , will the fex again go for loading of IOS from Parent switch ?

currently nothing is connected to Fex 100 and 101.

After swapping the cables , what needs to be done so that ports on Fex 100 and 101 will work properly  ?  In case if i want to both fex to download ios from parent switch after swapping cables what should be done ?

Actual scenario : 

Nexus 5k-1  Eth 1/47 ----> Fex 100  -----> Eth 1/47 Nexus 5k-2 (VPC100 . Port-chhannel 100)

Nexus 5k-1  Eth 1/48 ----> Fex 101  -----> Eth 1/48 Nexus 5k-2 (VPC 101, Port-channel 101 ) 

But onsite has connected cables as below.

Nexus 5k-1  Eth 1/48  (must be eth 1/47) ----> Fex 100  -----> Eth 1/47 Nexus 5k-2 (VPC100 . Port-chhannel 100)

Nexus 5k-1  Eth 1/47 ( must be eth 1/148 ) ----> Fex 101  -----> Eth 1/48 Nexus 5k-2 (VPC 101, Port-channel 101 ) 

Fiber Cables connected to fex 100 and 101 on Nexus 5k-1 will be swapped today so that we have correct ports connected to both fex from both Nexus 5ks. 

[Santhosh S]

Hi,

Once the cabling is corrected, the FEXs should get automatically detected, and download the image from the parent switch.

Best Regards,

Santhosh