cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
830
Views
6
Helpful
9
Replies

VIC 1387 40G connection to 3850 switch

I have a thread in the general switching forum, but I thought I would ask the question here.

https://community.cisco.com/t5/switching/ucs-240-m4-with-vic-1387-works-with-nexus-3064-but-not-with-3850/td-p/4748032 

I have a VIC 1387 in a C240 M4 server that connects correctly to a Nexus 3064. I am trying to move my limited 40G connections to the NM-2-40G connections in a 3850 stack, but I haven't been able to get it to work.The interface comes up on both sides and I see CDP neighbors on both sides. I can't get any IP traffic to pass over the link when it is connected to the 3850. The 1387 data sheet mentions compatible Nexus switches. Does that mean it will only talk to a Nexus switch? I am hoping that isn't that case.

1 Accepted Solution

Accepted Solutions

From the document previously referenced:

"The VIC always tags packets with an 802.1p header. While the upstream switchport can be an access port, different switch platforms behave differently when a 802.1p packet is received without a VLAN tag. Therefore, Cisco recommends that you have the upstream switchport configured as a trunk port."

View solution in original post

9 Replies 9

Wes Austin
Cisco Employee
Cisco Employee

It likely has to do with the way the 3850 is interpreting the traffic from the VIC 1300. Have you adjusted the VLAN settings from the VIC adapter in the CIMC to tag VLAN 252 and see if it makes any difference? You will likely need to adjust between trunk/access and VLAN tagged/untagged from CIMC side to resolve the connectivity issue.

 

This should also help:

https://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-c-series-rack-servers/117637-technote-UCS-00.html 

The port to which the VIC is connected is configured as a static access port in a single VLAN. It is 'switchport mode access' and 'switchport access vlan 252' in both the 3850 and the Nexus. That is what is so maddening is that the switch port in the Nexus and 3850 look the same.

What do your settings look like from the CIMC side? Can you provide screen captures?

There is some tweaking in VLAN configuration from the VIC 1300 side if I remember correctly when you are connecting to Catalyst vs Nexus, which is why I mentioned those things to check.

Nexus:
interface Ethernet1/51
  description vm-ucs1 - vmnic3
  switchport access vlan 252
  spanning-tree link-type point-to-point

3850:
interface FortyGigabitEthernet1/1/2
 description vm-ucs1 - NFS
 switchport access vlan 252
 switchport mode access

Above is how the ports are configured in the devices. Extremely simple as you can see. The vNICs in the 1387 were configured as default. I have even reset them to default just in case. The only non default parameter is MTU of 9000 since this is a storage network. I am trying changing the VLAN mode in VIC to Access instead of the default of trunk. If there is something else you want to see, I will certainly post it.

The document I reference in my original answers outlines all the different combinations you can try, but here are a few examples to try from the VIC 1300 in CIMC:

1. Access mode, tagging default VLAN 252

2. Access mode, not tagging VLAN 252

3. Trunk Mode, tagging default VLAN 252

Really need to get a packet capture from the switch to see info on the packets coming from the VIC and what the VLAN tagging looks like. You could also test the port on the 3850 in Trunk mode with combinations above and see if it changes the behavior. 

I am experimenting with the things you mentioned. The core question to me is why does what appear to be identically configured switch ports work with the VIC on one platform and not the other?

The Catalyst and Nexus switching platforms are different in their architecture and the way the 3850 interprets traffic from the Cisco VIC adapter is different than the way the Nexus interprets the same traffic. The VIC does priority-tagging for native-vlan traffic and Cisco VIC will always send a .1q VLAN tag which some switches do not handle properly.

 

From the document previously referenced:

"The VIC always tags packets with an 802.1p header. While the upstream switchport can be an access port, different switch platforms behave differently when a 802.1p packet is received without a VLAN tag. Therefore, Cisco recommends that you have the upstream switchport configured as a trunk port."

I did look at that document, but that section did not leap out at me. After some experimenting I can see that the Nexus doesn't care about an 802.1p header with no VLAN, but the 3850 clearly does. Ugh! Thanks for the clarification.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: