UCS VLAN help

[wormvine1]

Hello forum,

I apologize if this topic has been asked before and I have researched and read till I am blue in the face.

I have a couple questions regarding VLAN behavior on a UCS Mini.

My Environment:

Cisco UCS Mini in End Host mode.

UCS Manager v. 4.0(1a)

I am reconfiguring my LIVE UCS mini system due to a major iSCSI upstream switch failure.

Everything is peachy on the DATA uplink connections.

I am changing my iSCSI SAN connections and am getting confusing behavior. 

I have created a Fabric B specific VLAN (32).   

Why does it not appear in the available VLAN list of the new VLAN Group I created ??

VLAN 32 is NOT assigned to any vNICs yet.

The new VAN group (iSCSI-B) has no "Ethernet Uplink Ports" nor any "Port channels" added yet.

Many thanks in advance.  There is much more to my setup and current situation but I wanted to keep the question simple for now.

Adam

[Wes Austin]

Are you adding the VLANs under the LAN Tab > LAN Cloud section or LAN Tab > Appliances section?

[wormvine1]

The VLAN is created under the LAN Cloud.  No appliance VLANs.

Thanks for the response...

[Wes Austin · Accepted solution]

Thanks for the captures. I tried this in the lab and what you are seeing is expected behavior. VLAN groups are global and not specific to Fabric A or B. For your use case, I would just not use a VLAN group for that VLAN or you "could" create whatever VLANs as global and then create a VLAN group A and VLAN group B. YMMV.

[wormvine1]

Ok I understand that behavior is normal. Thank you.

I am reconfiguring my iSCSI connections to my NIMBLE SAN.

I am following their design recommendation to use 2 separate VLANs to 2 independent upstream switches.

I originally used one(1) global vlan which worked great till the switchstack it was connected to bugged out. 

We ordered 2 Nexus 3172T and waited 6 months to get all the necessary hardware.  

So I am swapping out the old Brocade stack (not my choice) for the new switches.  We are in 

Currently I have a port channel on Fabric A (ports 3/4) and a port channel on Fabric B (ports 3/4).

They are both mapped to a VLAN Group for my Original iSCSI single global VLAN 31.

The Nimble guide says to use 2 Fabric specific VLAN's and not Global but I don't know how each Fabric specific VLAN will

made available to each Fabric specific port channel ?? 

I dont fully understand the UCS networking nuances and I thought I would need to use the VLAN groups to add vlans to the interfaces and port channel.  

I am in contact with a TAC engineer but these config nuances have not fully been explained.  It's awesome you have a lab.

Could you show me the CLI output of a ethernet interface that is joined to a port channel with a Fabric Only (not global)  VLAN ?

Do not join the POrt channel or interfaces to a VLAN Group.  

[Wes Austin]

When you create a VLAN in UCSM it is distributed to all the uplink interfaces by default. You dont specify a VLAN group to go to a specific port or port-channel.

If you have a disjointed layer 2 network with multiple upstream switches, where some switches utilize certain VLANs while others do not, you would use LAN Uplinks manager to pin a specific VLAN to a specific uplink. If you only have a single layer 2 network northbound of the fabric interconnects, you want to create your VLAN under Fabric A or B and then it will only be distributed to only uplinks on the A or B. If you create them as global VLANs, they are distributed to A and B.

In my lab I have multiple global VLANs which you will see configured on my both Br1/25/1 uplinks, but also VLANs specific to Fabric A/B. I have a single uplink on A side and single uplink for B.

[wormvine1]

I understand.  Thank you.

I appreciate the screenshots.

So I do have a Fabric Specific VLAN on Fabric B.  (Vlan 32)

All 4 ports of Fabric B (UCS-Mini) are in port channels and those 2 port channels are attached to VLAN groups.

Vlan 32 does not show up in the interface config. 

Is that because the VLAN groups are blocking the fabric specific vlans from being added to the config?

[Wes Austin]

Should not be related, I just configured a VLAN group and both my VLAN group VLANs and my Fabric B VLANs all show up on the correct uplinks.

If you create a new test VLAN on A only or B only, does it show up correctly on Fabric A/B?

If you create a new test VLAN that is global/common, does it show up correctly on both fabrics?

If you add a VLAN to a VLAN group, does it show up correctly on both fabrics?

Can you send outputs for VLAN 32 and another working VLAN?

B(nxos)#show vlan id 32

[wormvine1]

Hey Wes,

I am off till Monday so I wont be able to check until then.

I dont know if creating a new global vlan will show on the fabric configs before being added to a VLAN group because I never looked but I just created a new Global VLAN recently and it was added to my LAN VLAN group and it does show up on those interfaces.  

That new VLAN (55) only shows up on the Port channels (and member interfaces) that are mapped to that VLAN group.  My other port channels (and member int) only show their VLAN group vlans.  

I will do some of your tests on monday and see what happens.  

I want to say I really appreciate you taking the time to hash it out with me.

Thank you,

Adam

[Wes Austin]

Creating a new global VLAN should make it appear on all of the FI uplinks by default.

Let me know the results of the tests in my previous posts, and can you also confirm what each uplink physically connects to and what the running configs are for those uplink interfaces are? Similar to the outputs from my lab.

[wormvine1]

So for my first test,  I created a Global VLAN 2030.   

Here is the message after creating the VLAN.

Currently I have 2 Port channels per fabric.

Ports 1 and 2 on each fabric are members of 1 Port channel and both are members of VLAN group "LAN".

Ports 3 and 4 on each fabric are members of the second port channel and both are members of VLAN Group "iSCSI".

As you can see in the attached snips, The newly created VLAN is not present on either Port channel.  Nor is it present on any of Fabric B's 4 interfaces.  

If you want more outputs or snips from Fabric A, let me know.  I checked Fabric A and it is identical to fabric B.  The newly created VLAN 2030 is not mapped to any interface.  

[Wes Austin]

Okay, so I am seeing some similar behavior when I add a VLAN group to the specified uplink. It is taking away my global VLANs and only adding the VLANs defined in my VLAN group.

You do not have many VLANs on your uplinks, so I think you should just get rid of VLAN groups all together, and just add VLANs as global or as Fabric A or B specific. I think the VLAN groups in your use case are causing more confusion. I have typically only used VLAN groups when adding VLANs to uplinks when you have 50 or more VLANs.

[wormvine1]

OK I am glad to see that you are seeing the same behavior. I was worried my system was bugged.

Since I only want LAN specific VLAN traffic to be present on Fabric Ports 1 and 2 , I must use a VLAN group to restrict that.

But I will be removing the current VLAN group mapped to ports 3 and 4 so I can enable fabric specific vlans for those ports. 

I don't want anything but iSCSI traffic on those ports.  

Here is the topology I am migrating to for reference.  Notice the iSCSI switches are NOT linked and there are separate physical paths to the NIMBLE SAN.

Thanks for all your help enlightening me to the UCS behaviors.  My training only reaches CCNP switch and router and I have no formal UCS LAN training.  

[Wes Austin]

If you want to restrict certain VLANs, you can use LAN Uplinks manager instead of VLAN Groups.

https://community.cisco.com/t5/unified-computing-system/vlans-not-in-a-vlan-group/td-p/3839018

[wormvine1]

Ok.  I understand.  Thank you. 

Is there any reason NOT to use the VLAN Groups ???