Nexus 93180 vpC and VMWare recommendations

[janit]

Probably should've bought different gear but got two 93180s in vPC config.

3 VMWare hosts. 3 Vswitches on them. Each with 3 dual 25gbps NICs. Plus some 1gbps.

Plan to use 2 ports from each host per vswitch connected to each Nexus for redundancy. Management and VM networks tagged on one, datastores access and vmotion on the other ones. VPC basic setup and working OK. Looking for recommendations for the rest.

interface port-channel1
description **** VMotion vPC ****
switchport
switchport mode trunk
vpc 1

interface port-channel2
description **** VMWare Trunked vPC ****
switchport
switchport mode trunk
switchport trunk allowed vlan 20,30,50-51,56,60
vpc 2

interface port-channel3
description **** 10.48 to Netap vPC ****
switchport
switchport mode trunk
vpc 3

***************

interface Ethernet1/7
description **** VMotion ****
switchport
switchport mode trunk
channel-group 1 mode active

interface Ethernet1/11
description **** VMWare Trunked vPC ****
switchport
switchport mode trunk
switchport trunk allowed vlan 20,30,50-51,56,60
channel-group 2 mode active

We don't aggregate links on Netapp. Anyhow. I would prefer not to use distributed switches. But I'm really stuck with this. We have it working OK on Catalyst but ..

Last question, does any payload data from one Peer Switch travel over Peer Link at all ? All I see is mac sync etc. but nothing about actual traffic. That would mean Peer Link should be a bit wider. Or would orphan port on one switch to to same on the other over Peel Link ? 

Thx

[Reza Sharifi]

Or would orphan port on one switch to to same on the other over Peel Link ? 

orphan ports traffic will traverse the vPC peer link. As for VM switches to Nexus, if you use "Route Based on Originating Port ID" on the VM switches, all you need on the Nexus side are trunk ports and no need for any Portchannels.

HTH

[janit]

This Route Based .. is on VMWare vSwitchx settings, Teaming and Failover, "Route based on originating virtual port" ?

I understand you're saying to just configure all ports as orphan trunk ports with allowed VLans. As in the config below without channel-group line:

nterface Ethernet1/11
description **** VMWare Trunked vPC ****
switchport
switchport mode trunk
switchport trunk allowed vlan 20,30,50-51,56,60
no channel-group 2 mode active ( ***remove***)

Man, this is excellent help. Thx

[MHM Cisco World]

if you decide to use orphan connect it to primary vPC not to secondary 

[janit]

Again, you answered before I asked. Was thinking that too. I asked about "no suspend" in my last post.

[Reza Sharifi]

This Route Based .. is on VMWare vSwitchx settings, Teaming and Failover, "Route based on originating virtual port" ?

Correct, that is one of the options to use. If you use "ip hash", you would need to create Portchannels, but with "Route based on originating virtual port" no Portchannles are needed and the failover works really well from one NIC to another. If you ever need to use  Portchannels with VMs, use mode "on" on the switch side and of course "IP hash" on the VM side.

no channel-group 2 mode active ( ***remove***)

Correct, no need for this.

HTH

[MHM Cisco World]

no channel-group 2 mode active ( ***remove***) <<<- why he need to remove this ?

[Reza Sharifi]

Because if OP uses "Route Based on Originating Port ID" as the hashing algorithm, there is no need for any Portchannels.

[janit]

This is just to show the line is not there. I understood if I can tag the VLans on vSwitch Reza says just use standard trunk ports. I assume same for Netapp ports. If can't tag I guess I can just use standard access. Right ?

[MHM Cisco World]

just let summary issue here 
you need to run vPC vlan from one vPC nexus to your VM 
if that correct then you need to config the link with VPC 
this need PO and hence you need to config mode but instead of active make it ON.
am I right here 

[Reza Sharifi]

That is correct. For Netapp, just access port because your entire storage is one vlan.

We actually tested this config with Netapp storage and the failover between the NICs was seamless.

You should test it as well to make sure the failover works as expected.

HTH

[janit]

No need for PortChannel to VMware at all. I'm not planning to use distributed switches on VMware. And yes, you already answered Netapp question. For VMotion I could use just access ports on specific vlan or trunk and tag it (right?). For VirtualMachines I will use trunks with only allowed VLans. Etc. 

"this need PO and hence you need to config mode but instead of active make it ON."

Really not sure what this means. Perhaps you thought I need VPC to VMware but I don't. 

Just need redundancy and simplicity. Couldn't find a confirmation that outside of VPC ports Nexus pair acts almost like stacked switch. The only VPC outside of Peer-Link would be in this case Link-to-the-world means uplink to the rest of the network. Catalyst switch in this case.

[Reza Sharifi]

For VMotion I could use just access ports on specific vlan or trunk and tag it (right?)

Sure, if you are using one vlan, use access ports with the specific vlan you need, but if you have multiple vlans then trunk them. 

Just need redundancy and simplicity. 

This is the key. One can make this as complicated as possible by using POs, VPC, LACP, etc.. but if you can simply use trunk ports and have redudancy why not go that route. The simpler the better. The other thing to rememeber is that VMs don't always play well with POs. So, if you don't need it why bother, but that is just my opinion.

HTH

[janit]

I assume POs you mean PortChannels. 

I quickly made the recommended changes. Second switch will obviously be almost the same, but for VPC Links.

There is this "no suspend" command for orphan ports I should use I think right ?

Uploading config should you have a moment to comment.

[Reza Sharifi · Accepted solution]

The config looks good. Now, make sure the VM side is configured correctly, and then test by unplugging one link at a time.

I am not familiar with it but I think the "suspend" command is used with PO running LACP. See the below document.

https://davidsudjiman.wordpress.com/2018/01/31/lacp-suspend-individual/

HTH