Various questions on uplink profiles, CoS, native VLAN, downlink trunking

[ADAman]

I will be using vPC End Host Mode with MAC-pinning. I see I can further configure MAC-Pinning. Is this required or will it automatically forward packets by just turning it on? Is it also best not to enable failover for the vnics in this configuration? See this text from the Cisco 1000V deployment Guide:

"

Fabric Fail-Over Mode

Within the Cisco UCS M71KR-E, M71KR-Q and M81KR adapter types, the Cisco Unified Computing System can

enable a fabric failover capability in which loss of connectivity on a path in use will cause remapping of traffic

through a redundant path within the Cisco Unified Computing System. It is recommended to allow the Cisco Nexus

1000V redundancy mechanism to provide the redundancy and not to enable fabric fail-over when creating the

network interfaces within the UCS Service Profiles. Figure 3 shows the dialog box. Make sure the Enable Failover

checkbox is not checked."

What is the 1000V redundancy?? I didn't know it has redundancy. Is it the MAC-Pinning set up in the 1000V? Is it Network State Tracking?

The 1000V has redundancy and we can even pin VLANs to whatever vNIC we want. See Cisco's Best Practices for Nexus 1000V and UCS.

Nexus1000V management VLAN. Can I use the same VLAN for this and for ESX-management and for Switch management? E.g VLan 3 for everything.

According to the below text (1000V Deployment Guide), I can have them all in the same vlan:

There are no best practices that specify whether the VSM

and the VMware ESX management interface should be on the same VLAN. If the management VLAN for

network devices is a different VLAN than that used for server management, the VSM management

interface should be on the management VLAN used for the network devices. Otherwise, the VSM and the

VMware ESX management interfaces should share the same VLAN.

I will also be using CoS and Qos to prioritize the traffic. The CoS can either be set in the 1000V (Host control Full) or per virtual adapter (Host control none) in UCS. Since I don't know how to configure CoS on the 1000V, I wonder if I can just set it in UCS (per adapter) as before when using the 1000V, ie. we have 2 choices.

Yes, you can still manage CoS using QoS on the vnics when using 1000V:

"

The recommended action in the Cisco Nexus 1000V Series is to assign a class of service (CoS) of 6 to the VMware service console and VMkernel flows and to honor these QoS markings on the data center switch to which the Cisco UCS 6100 Series Fabric Interconnect connects. Marking of QoS values can be performed on the Cisco Nexus 1000V Series Switch in all cases, or it can be performed on a per-VIF basis on the Cisco UCS M81KR or P81E within the Cisco Unified Computing System with or without the Cisco Nexus 1000V Series Switch.

Something else: Native VLANs

Is it important to have the same native VLAN on the UCS and the Cisco switch? And not to use the default native VLAN 1?   I read somewhere that the native VLAN is used for communication between the switches and CDP amongst others. I know the native VLAN is for all untagged traffic. I see many people set the ESXi management VLAN as native also, and in the above article the native VLAN (default 1) is setup. Why? I have been advised to leave out the native VLAN.

Example:Will I be able to access a VM set with VLAN 0 (native) if the native VLAN is the same in UCS and the Cisco switch (Eg. VLAN 2)? Can I just configure a access port with the same VLAN ID as the native VLAN, i.e 2 and connect to it with a PC using the same IP network address?

And is it important to trunk this native VLAN? I see in a Netapp Flexpod config they state this: "This configuration also leverages the native VLAN on the trunk ports to discard untagged packets, by setting the native VLAN on the port channel, but not including this VLAN in the allowed VLANs on the port channel". But I don't understand it...

What about the downlinks from the FI to the chassis. Do you configure this as a port channel also in UCS? Or is this not possible with the setup described here with 1000V and MAC-pinning.

No, port channel should not be configured when MAC-pinning is configured.

--

[Robert] The VSM doesn't participate in STP so it will never send BPDU's.  However, since VMs can act like bridges & routers these days, we advise to add two commands to your upstream VEM uplinks - PortFast and BPDUFilter.  PortFast so the interface is FWD faster (since there's no STP on the VSM anyway) and BPDUFilter to ignore any received BPDU's from VMs.  I prefer to ignore them then using BPDU Gaurd - which will shutdown the interface if BPDU's are received.

-Are you thinking of the upstream switch here (Nexus, Catalyst) or the N1kV uplink profile config?

Edit: 26 July 14:23. Found answers to many of my many questions...

[Robert Burns]

Answers inline.

Atle Dale wrote:
Something else: Native VLANs
Is it important to have the same native VLAN on the UCS and the Cisco switch? And not to use the default native VLAN 1?   I read somewhere that the native VLAN is used for communication between the switches and CDP amongst others. I know the native VLAN is for all untagged traffic. I see many people set the ESXi management VLAN as native also, and in the above article the native VLAN (default 1) is setup. Why? I have been advised to leave out the native VLAN.
[Robert] The native VLAN is assigned per hop.  This means between the 1000v Uplinks port profile and your UCS vNIC definition, the native VLAN should be the same.  If you're not using a native VLAN, the "default" VLAN will be used for control traffic communication.  The native VLAN and default VLAN are not necessarily the same.  Native refers to VLAN traffic without an 802.1q header and can be assigned or not.  A default VLAN is mandatory.  This happens to start as VLAN 1 in UCS but can be changed. The default VLAN will be used for control traffic communication.  If you look at any switch (including the 1000v or Fabric Interconnects) and do a "show int trunk" from the NXOS CLI, you'll see there's always one VLAN allowed on every interface (by default VLAN 1) - This is your default VLAN.
Example:Will I be able to access a VM set with VLAN 0 (native) if the native VLAN is the same in UCS and the Cisco switch (Eg. VLAN 2)? Can I just configure a access port with the same VLAN ID as the native VLAN, i.e 2 and connect to it with a PC using the same IP network address?
[Robert] There's no VLAN 0.  An access port doesn't use a native VLAN - as its assigned to only to a single VLAN.  A trunk on the other hand carries multiple VLANs and can have a native vlan assigned.  Remember your native vlan usage must be matched between each hop.  Most network admins setup the native vlan to be the same throughout their network for simplicity.  In your example, you wouldn't set your VM's port profile to be in VLAN 0 (doens't exist), but rather VLAN 2 as an access port.  If VLAN 2 also happens to be your Native VLAN northbound of UCS, then you would configured VLAN 2 as the Native VLAN on your UCS ethernet uplinks.  On switch northbound of the UCS Interconnects you'll want to ensure on the receiving trunk interface VLAN 2 is set as the native vlan also.  
Summary:
1000v - VM vEthernet port profile set as access port VLAN 2
1000v - Ethernet Uplink Port profile set as trunk with Native VLAN 2
UCS - vNIC in Service Profile allowing all required VLANs, and VLAN 2 set as Native
UCS - Uplink Interface(s) or Port Channel set as trunk with VLAN 2 as Native VLAN
Upstream Switch from UCS - Set as trunk interface with Native VLAN 2
From this example, your VM will be reachable on VLAN 2 from any device - assuming you have L3/routing configured correctly also.
And is it important to trunk this native VLAN? I see in a Netapp Flexpod config they state this: "This configuration also leverages the native VLAN on the trunk ports to discard untagged packets, by setting the native VLAN on the port channel, but not including this VLAN in the allowed VLANs on the port channel". But I don't understand it...
[Robert] This statement recommends "not" to use a native VLAN.  This is a practice by some people.  Rather than using a native VLAN throughout their network, they tag everything.  This doesn't change the operation or reachability of any VLAN or device - it's simply a design descision.  The reason some people opt not to use a native VLAN is that almost all switches use VLAN 1 as the native by default.  So if you're using the native VLAN 1 for management access to all your devices, and someone connects in (without your knowing) another switch and simply plug into it - they'd land on the same VLAN as your management devices and potentially do harm.
What about the downlinks from the FI to the chassis. Do you configure this as a port channel also in UCS? Or is this not possible with the setup descrived here with 1000V and MAC-pinning.
[Robert] On the first generation hardware (6100 FI and 2104 IOM) port channeling is not possible.  With the latest HW (6200 and 2200) you can create port channels with all the IOM - FI server links.  This is not configurable.  You either tell the system to use Port Channel or Individual Links.  The major bonus of using a Port Channel is losing a link doesn't impact any pinned interfaces - as it would with individual server interfaces.  To fix a failed link when configured as "Individual" you must re-ack the Chassis to re-pinn the virtual interfaces to the remaining server uplinks.  In regards to 1000v uplinks - the only supported port channeling method is "Mac Pinning".  This is because you can't port channel physical interfaces going to separate Fabrics (one to A and one to B).  Mac Pinning gets around this by using pinning so all uplinks can be utilized at the same time.
--
[Robert] The VSM doesn't participate in STP so it will never send BPDU's.  However, since VMs can act like bridges & routers these days, we advise to add two commands to your upstream VEM uplinks - PortFast and BPDUFilter.  PortFast so the interface is FWD faster (since there's no STP on the VSM anyway) and BPDUFilter to ignore any received BPDU's from VMs.  I prefer to ignore them then using BPDU Gaurd - which will shutdown the interface if BPDU's are received.
-Are you thinking of the upstream switch here (Nexus, Catalyst) or the N1kV uplink profile config?
[Robert] The two STP commands would be used only when the VEM (ESX host) is directly connected to an upstream switch.  For UCS these two commands to NOT apply.

[ADAman]

Thanks again for your good answers. I am fairly new to networking (I am a storage guy) and I am struggeling a bit on some important concepts (you will see later). I will reply as I read your comments:

Let's try to make this simple (not that you don't..) and assume a newly installed switch (my case). The default VLAN is 1 on all ports in the FIs and 1000V, and it is used for "control traffic"  This default vlan 1 is both default and native on all switchports in the upstream Cisco switch (also newly installed). VLAN 1 is default in the FIs and the 1000V. If we wanted to use native VLAN1, this would have to be configured in the 1000V uplink profile and in the UCS vNIC definition. Correct?

So, in this case, VLAN 1 is used for control traffic communication. But are you talking about one of the three types of traffic we setup in the 1000V: control, packet and management? We still need to set this up? E.g set them all to use VLAN 10. The control traffic you talk about is a different switch management traffic?  And this defult VLAN must be configured to be the same for the FIs and the 1000V when the native VLAN is not used?

But what about the upstream switch?  I think I have seen questions on this problem, that the Nexus or another series use VLAN 2 as default? Or am I mistaken? In this case the default VLAN must be set to to 1 on the upstream switch?

-----

So there is no VLAN 0 in vSphere if I use 1000V? I did not know that. With the vDS, VLAN 0 is available and denotes untagged traffic. How do you denote untagged traffic in VMware when using the 1000V? By setting it to a defined native VLAN?

Question: If I wanted to use my revised setup with native VLAN 2 with a normal VLAN 2 (as is the best practice), would the only difference in the configuration be to swap all the native VLANs with VLAN 2? And then in order to connect to this VLAN, you would have to connect to a access port with VLAN 2? But this seems cubersome for a management network. How can people get away with not using native VLANs for management purposes? They obviously want to be able to connect to a management interface from their PC.  How do they connect their Pc to many VLANs? By making the switchport a trunk-port? Can you connect a PC to a trunk port? (Sorry for my low knowledge in switching, but I am working very hard to overcome this limitation).

-----

Regarding the downlinks, just to make sure, with MAC pinning set for the uplink port profiles, we cannot set the server links in port-channel mode. Correct?