I'm attempting to setup Adapter-FEX between a Cisco C220 M3 server with a Broadcom BCM57810 (Cisco P/N UCSC-PCIE-B3SFP) and a pair of Nexus 5596 switches. The server is running Windows 2012 R2 (also tried 2008 R2). I've setup both the Nexus switches and Broadcom card according to what documentation that I could find (which isn't much). When configuring the Broadcom card, the port profiles do show up so that I can choose them (so it is seeing the Nexus switch). I've tried various combinations of assigning the profiles to different channels on the Broadcom card. However, when the server is booted up, both uplinks go into a down state and show "link failure" in the system log of the Nexus. Yet when the system is undergoing POST tests and such (pre-Windows boot), both links are active and online. Is there a configuration setting that I may be missing? The Broadcom card is setup for AFEX mode with DCB enabled, and flow control is turned off for the channels that I was configuring. The Nexus switches each have the VM-Fex license installed.
The setup that I'm trying to accomplish is rather simple. I just need to have a minimum of two virtual adapters created which are on two separate VLANs. One physical port on the Broadcom card goes to Nexus switch #1, and the second port goes to Nexus switch #2. The Nexus switches are setup in a redundant fashion with VPCs between them. All drivers on the Windows server are up-to-date from Cisco's website. The Nexus switches never show any vethernet interfaces being created. And manually created vethernet interfaces don't come online either.
Commands entered into the Nexus switches(and synced between them):
install feature-set virtualization
feature-set virtualization
vethernet auto-create
port-profile type vethernet PRIMARY_DATA_VLAN103
switchport access vlan 103
state enabled
port-profile type vethernet BACKUP_DATA_VLAN400
switchport access vlan 400
state enabled
interface Ethernet1/32
description New Backups Server
switchport mode vntag
no shutdown
Output of Nexus switch interface after server is booted up:
Output:
CU-DC-5K1# sh int virtual summ
Veth Bound Channel/ Port Mac VM
Interface Interface DV-Port Profile Address Name
-------------------------------------------------------------------------
Total 0 Veth interfaces
CU-DC-5K1#
CU-DC-5K1# sh int e1/32
Ethernet1/32 is down (Link not connected)
Hardware: 1000/10000 Ethernet, address: 002a.6a48.e7a7 (bia 002a.6a48.e7a7)
Description: New Backups Server
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is vntag
auto-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 00:36:29
Last clearing of "show interface" counters never
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 0 bits/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
RX
0 unicast packets 71045 multicast packets 17043 broadcast packets
88088 input packets 8066953 bytes
0 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
36425 unicast packets 11806 multicast packets 18962 broadcast packets
67193 output packets 21437797 bytes
0 jumbo packets
0 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
40 interface resets
