New VDC and new FEX

[fsebera]

Anyone know if this new setup will cause ANY type of reboot  or outage of the N7K or end-nodes?

We have a pair of Nexus 7010 switches

Sup-1 modules

NX-OS 6.2 

3 VDCs (admin, 2 other VDCs).

We want to create the last available VDC as a test bed for End-to-End proof of concept of FEX and end servers.

We have verified open and free ports for connectivity for this setup.

We have learned that upgrading the NX-OS causes Orphan FEX to reboot upon N7K reloading and this is the reason to use a FEX per N7K and dual home the end-node servers.

Our plan for this new test bed is to connect:

FEX-A <-------> N7K-A 

FEX-B <-------> N7K-B

Dual home servers to both FEX in 1 of 2 different configurations.

1. Server NICs configured in a port-channel, bonding, NIC-Teaming configuration in an active-active configuration

2. Server NICs connected to both FEX but in an active/standby configuration. (no port-channel, bonding or NIC teaming configuration).

Anyone know if this new setup will cause ANY type of reboot or outage of the N7K or end-nodes?

Thanks

Frank

[ahmedshoaib]

Hi;

Nexus 7000 Sup-1 support 4 VDCs. If you are using a separate admin vdc then you can create 4 + 1 (admin vdc).

For creating VDC you don’t required any downtime or maintenance window. Whenever you assign the port to VDC you need to take care about port-allocation & module type (i.e F2 module will not work in same VDC with F1, M1, F2e, F3). Port will allocate to VDC in form of group (i.e 2/4/8 ports).

Nexus 7000 Switch still don’t support FEX as dual home. FEX only support single home on Nexus 7000 switches.

You can connect your server's on FEX as:

1. Active/Active (via port-channel or Active/Active NIC teaming on Server Side and vPC on Nexus side) and

2. Active/Standby (via Active/Standby NIC teaming on Server Side & Access/Trunk port on Nexus side).

In any above configuration modification/setup on Nexus switch you don’t get any outage and switch reboot. You required maintenance window only for those server you want to transfer on new VDC & FEX.

Thanks & Best regards;

[fsebera]

We currently have end-nodes connected to FEX-1 and FEX-1 is currently connected to N7k-1.

If I connect the end-node 2nd NIC to FEX-2 and FEX-2 is only connected to N7k-2 should I experience and outage when making this connection?

I think when I setup the end-node NICs into bonding (NIC teaming) I will probably experience an outage on the end-node due to the bonding software configuration --- right?

The outer issue is, N7K NX-OS upgrades.

If we have an end-node connected to FEX-1 and FEX-2 in a vPC configuration 

FEX-1 is only connected to N7K-1

FEX-2 is only connected to N7K-2

N7k-1 and N7k-2 make up the domain

If we upgrade the NX-OS on N7k-1 there should not be a FEX reboot until N7k-2 is upgraded ---RIGHT?

I plan on testing this in the Proof-of-Concept environment, but just looking for a good basis to start with.

Thanks for your advise and guidance!

Oh couple notes:

The Sup-1 only supports 4 VDCs as there is no separate admin/default VDC. Admin/Default VDC is used in our environment as a production VDC.

NX-OS code 7.2 supports dual-homed FEX but requires F2 cards - I think F2.

Frank

[ahmedshoaib · Accepted solution]

Hi;

As per my understanding, please find my comments on your queries:

Q1: If I connect the end-node 2nd NIC to FEX-2.

I think when I setup the end-node NICs into bonding (NIC teaming) I will probably experience an outage on the end-node due to the bonding software configuration --- right?

Ans: Yes, whenever you configure NIC team (either Active/Active or Active/Standby) you need outage for that specific server. If you are using Active/Active everyone will recommend LACP.

Q2: The outer issue is, N7K NX-OS upgrades. End-node connected to FEX-1 and FEX-2 in a vPC configuration. 

FEX-1 is only connected to N7K-1 & FEX-2 is only connected to N7K-2

If we upgrade the NX-OS on N7k-1 there should not be a FEX reboot until N7k-2 is upgraded ---RIGHT?

Ans: Before plan to go for upgrade, please make sure your Sup-1 should have 8GB RAM. 6.2(2) onward NXOS required min. 8GB RAM. Please also go through the upgrade path and ISSU Path for specific release.

If FEX is in a single home (FEX-1 is only connected to one Nexus only), during NXOS upgrade all FEX will be reload which is connected to nexus where the OS is upgrading.

If you have a server which is connected to dual home vPC with both FEX then you don’t face network outage.

Q3: NX-OS code 7.2 supports dual-homed FEX but requires F2 cards - I think F2.

Ans: Yes, you are right now FEX is supported dual home FEX of all line card. But still I will recommend you go with single home FEX. Reason there is a bug reported with reference to dual-home FEX.

Reported Bug: https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuv93032

Release note: http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/7_x/nx-os/release/notes/72_nx-os_release_note.html#pgfId-811992

Q4: The Sup-1 only supports 4 VDCs as there is no separate admin/default VDC. Admin/Default VDC is used in our environment as a production VDC.

Ans: Cisco NX-OS 6.2(2) onward support Admin VDC on Sup-1.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/nx-os/virtual_device_context/configuration/guide/b-7k-Cisco-Nexus-7000-Series-NX-OS-Virtual-Device-Context-Configuration-Guide/configuring-an-admin-vdc.pdf

Thanks & Best regards;