Native VLAN

erwin.sobe · ‎12-23-2011

Hello everyone!

I have this network configuration:

6500 connected with 6120XP using a trunk port with all VLANs and VLAN 1 set as the native VLAN.

In the UCS LAN Cloud are these VLANs:

VLAN abc (400)
VLAN cde (401)
VLAN default (1) with native=yes

VLAN 1 is the management network of our Cisco switches and needed on some blades for network monitoring software, so i need it in the UCS.

For all other blades there should be no way to get into VLAN 1, so i configured their service profiles

with VNICs having these VLANs

VLAN abc (400)
VLAN cde (401).

Neither of them is defined as native, and as you can see there is no VLAN 1 defined.

I define a ESXi5.0 vSwitch with 3 portgroups

abc VLAN ID 400
cde VLAN ID 401
no-tag VLAN ID None (0) (for testing)

VMs with interfaces in portgroup "abc" and "cde" are working as expected.

BUT: Why can a VM with an interface in the "no-tag" portroup reach other devices outside the UCS on VLAN 1, although the VNIC down to the blades has no VLAN 1 defined?

Thank you in advance!

Best regards,

Erwin

Robert Burns · ‎12-23-2011

Erwin,

1. What version of UCSM are you running?

2. From UCSM, find the blade under the Equipment Tab, expand the Adapters, Adapter 1, and highlight "NICs". On the Right side expand NIC 1 and NIC 2 so you can se ethe VIF #'s.

Take note of your two NIC VIFs.

Next, log into the UCS CLI - issue "connect nxos" and then:

show int trunk | inc VethXXXX (case sensitive, where XXXX = the VIF # for the NIC(s) of the Fabric Interconnect you're currently connected to (A/B).

Paste this output along with your VIF #s here for further investigation.

3. Also from the UCS CLI, NXOS context, plaste the output of:

show vlan

Regards,

Robert

erwin.sobe · ‎12-27-2011

Hi Robert,

1. UCSM Version

---------------

2.0(1t)

2. VNIC Information

-------------------

M72KR-E Adapter of Blade #2 (B200-M2)

To verify the VLAN configuration:

UCS1-A /org/service-profile # show vnic expand

vNIC:

Name: VNIC-A1

Fabric ID: A

Dynamic MAC Addr: XX:XX:XX:XX:XX:X1

Ethernet Interface:

Name: abc

Dynamic MAC Addr: XX:XX:XX:XX:XX:X1

Default Network: No

VLAN ID: 400

Operational VLAN: fabric/lan/net-abc

Name: cde

Dynamic MAC Addr: XX:XX:XX:XX:XX:X1

Default Network: No

VLAN ID: 401

Operational VLAN: fabric/lan/net-cde

Name: VNIC-B1

Fabric ID: B

Dynamic MAC Addr: XX:XX:XX:XX:XX:X2

Ethernet Interface:

Name: abc

Dynamic MAC Addr: XX:XX:XX:XX:XX:X2

Default Network: No

VLAN ID: 400

Operational VLAN: fabric/lan/net-abc

Name: cde

Dynamic MAC Addr: XX:XX:XX:XX:XX:X2

Default Network: No

VLAN ID: 401

Operational VLAN: fabric/lan/net-cde

UCSM GUI:

NIC 1: VIF 743 (FI A)

NIC 2: VIF 744 (FI B)

UCS1-A(nxos)# show int trunk (same for FI B)

--------------------------------------------------------------------------------

Port Native Status Port

Vlan Channel

--------------------------------------------------------------------------------

Eth1/15 1 trunking --

Eth1/1/2 1 trunking --

Eth1/1/9 4044 trunking --

--------------------------------------------------------------------------------

Port Vlans Allowed on Trunk

--------------------------------------------------------------------------------

Eth1/15 1,400-401

Eth1/1/2 1,400-401

Eth1/1/9 4044

--------------------------------------------------------------------------------

Port Vlans Err-disabled on Trunk

--------------------------------------------------------------------------------

Eth1/15 none

Eth1/1/2 none

Eth1/1/9 none

--------------------------------------------------------------------------------

Port STP Forwarding

--------------------------------------------------------------------------------

Eth1/15 1,400-401

Eth1/1/2 1,400-401

Eth1/1/9 4044

--------------------------------------------------------------------------------

Port Vlans in spanning tree forwarding state and not pruned

--------------------------------------------------------------------------------

Eth1/15 --

Eth1/1/2 --

Eth1/1/9 --

There are no Veth743 or Veth744 ports (because it is a M72KR-E?). But what i have is a Eth1/1/2 port, which should be for blade #2? And there is a VLAN 1 allowed in the trunk although not configured for the VNICs.

3. VLAN information

-------------------

UCS1-A(nxos)# sh vlan (same for FI B)

VLAN Name Status Ports

---- -------------------------------- --------- -------------------------------

1 default active Eth1/1, Eth1/2, Eth1/3, Eth1/4

Eth1/5, Eth1/6, Eth1/7, Eth1/8

Eth1/13, Eth1/14, Eth1/15

Eth1/16, Eth1/17, Eth1/18

Eth1/19, Eth1/20, Eth1/1/1

Eth1/1/2, Eth1/1/3, Eth1/1/4

Eth1/1/5, Eth1/1/6

Eth1/1/7, Eth1/1/8

10 fcoe-vsan-10 active

15 fcoe-vsan-15 active Veth8919, Eth1/1/2

400 VLAN0400 active Eth1/15, Eth1/1/2

401 VLAN0401 active Eth1/15, Eth1/1/2

4044 SAM-vlan-management active Eth1/1/9

4047 SAM-vlan-boot active

4048 VLAN4048 active

(Ports 9-12 are Server Links)

I'll try a blade with the M81KR adapter tomorrow.

Best Regards,

Erwin

zmeng · ‎12-27-2011

Hi Erwin,

from vmware port-group point of view, If you enter 0 or leave the option blank, the port group can see only untagged (non-VLAN) traffic.

From UCS side,even though you do not configure vlan 1 in vnic, by default the system will pick up vlan 1 as native vlan, as you can see in the output of command.

So when VM traffic without vlan-tag is sent out, it will be broadcasted in un-tagged segment, including native vlan.

Yes, M72KR-E adapter has no veth presented in nxos.

rettuc_ccnp · ‎01-04-2012

zmeng pretty much answered your question already but I'd still like to expand on this a little. thumbs up!

The first thing to note is that VLAN1 is used by default on both access and trunk ports in a Cisco LAN switching infrastructure. This means all access ports are assigned to VLAN1 and all trunk ports will use VLAN1 as the native VLAN for tagging all untagged traffic (i.e. “no-tag” port group members) prior to sending the traffic out. The only way to change this default behavior is to explicitly configure the VLAN on each access port and/or the native VLAN on each trunk port. Note these default behaviors are not reflected in the running configurations of the access or trunk ports.

Since your Cisco ports are configured as trunk ports without a native VLAN explicitly defined…it’s the default “native VLAN” behavior of these trunk ports which is allowing the VMs in your “no-tag” port group to reach other devices outside the UCS on VLAN1. Meaning, when your untagged traffic wants to traverse a trunk link it will be tagged with the native VLAN of the trunk. In your case since a native VLAN hasn’t been configured the untagged traffic from your VMs will be tagged with VLAN1 before it is sent across the LAN trunk links. As a result this traffic will become part of the VLAN1 segment thus allowing communication with other devices outside the UCS which also belong to the VLAN1 segment.

There are a number of ways to address this... Here are few that come to mind; 1) You can explicitly configure the native VLAN so VLAN 1 not used for untagged traffic. Keep in mind that untagged traffic will become part of any segment you define as the native VLAN. Also, depending on your overall strategy you might want to consider using a VLAN not configured for any other environment. 2) assign all vNICs to a port group and ensure all port groups are configured with the appropriate VLAN ID (assign VLANs to all VM’s)...tag all of your traffic. 3) ACLs

erwin.sobe · ‎01-09-2012

Hi everyone!

First let me thank you for answers!

Please forgive me being late with the promised test with a M81KR adapter (i show only FI A)

UCSM GUI:

NIC 1: VIF 739 (FI A)

VLANS 400 and 401 configured.

Now i can see the Veth Port with "show int trunk" and the UCS is respecting the configuration: No VLAN1 "allowed" on the Veth.

It still says VLAN1 is the Native VLAN for the Veth Ports, but there is no traffic from VLAN 1 down to the ESXI vSwitch, neither tagged nor untagged (tried both).

--------------------------------------------------------------------------------

Port Native Status Port

Vlan Channel

--------------------------------------------------------------------------------

...

Veth739 1 trunking --

...

--------------------------------------------------------------------------------

Port Vlans Allowed on Trunk

--------------------------------------------------------------------------------

...

Veth739 400-401

...

--------------------------------------------------------------------------------

Port Vlans Err-disabled on Trunk

--------------------------------------------------------------------------------

...

Veth739 none

...

--------------------------------------------------------------------------------

Port STP Forwarding

...

Veth739 400-401

...

--------------------------------------------------------------------------------

Port Vlans in spanning tree forwarding state and not pruned

--------------------------------------------------------------------------------

...

Veth739 --

...

So far so good!

Now i added VLAN1 to the VNICs but not configured as Native VLAN, as you can see with "show vnic expand"

Ethernet Interface:

Name: default

Dynamic MAC Addr: XX:XX:XX:XX:XX:X2

Default Network: No

VLAN ID: 1

Operational VLAN: fabric/lan/net-default

VLAN1 is now part of the "allowed" VLANs on the trunk:

--------------------------------------------------------------------------------

Port Native Status Port

Vlan Channel

--------------------------------------------------------------------------------

...

Veth739 1 trunking --

...

--------------------------------------------------------------------------------

Port Vlans Allowed on Trunk

--------------------------------------------------------------------------------

...

Veth739 1,400-401

...

--------------------------------------------------------------------------------

Port Vlans Err-disabled on Trunk

--------------------------------------------------------------------------------

...

Veth739 none

...

--------------------------------------------------------------------------------

Port STP Forwarding

...

Veth739 1, 400-401

...

--------------------------------------------------------------------------------

Port Vlans in spanning tree forwarding state and not pruned

--------------------------------------------------------------------------------

...

Veth739 --

...

Now i configured a portgroup on the vSwitch for VLAN1 - but no connection. I put the VM interface into the no-tag port group - now i have connection => the traffic from VLAN1 is going down untagged to the ESXI vSwitch.

I don't know if this is the expected behavior by the developers. It is important to me, that with the M81KR a VLAN that is not configured for a VNIC does not go down to the blade.

Addendum (hint from rettuc_ccnp :

Only i configure another (unused) VLAN as the native VLAN (such as 999), "show int trunk" now says:

--------------------------------------------------------------------------------

Port Native Status Port

Vlan Channel

--------------------------------------------------------------------------------

...

Veth739 999 trunking --

...

VLAN 1 traffic is now tagged und VLAN400 untagged.

Best Regards,

Erwin

zmeng · ‎01-09-2012

Hi Erwin,

It is expected behavior. it is not related to any adapter .

In first configuration, (Veth739 is trunk port with native vlan 1), when tagged traffic from PC entering the trunk port, since the vlan ID is "1" same as native vlan, the trunk port will consider the traffic is for itself, so it will untag it and then discard it because the traffic is not really for him. (need to find doc to support the statement, but from behavior and experience, it is). while untagged traffic entering the trunk port, the traffic will be transferred within native vlan domain.

After you change the native vlan from 1 to 999, the tagged traffic from PC in vlan 1 will be handled normally with same way as other vlans. That mean the traiffic will be transferred within L2 vlan 1 domain.

Basically,when a port is designated as a trunk Port, it will forward and receive tagged frames.if an untagged frame is received on a trunk port, the frame is associated with the Native VLAN for this port. Frames belongs to native vlan do not carry VLAN tags when sent over the trunk.

Hopefully it is helpful for understanding. As per workaround/solution, rettuc_ccnp has given clear statement.

BR,

John Meng