Nexus 5020 vpc to 1000v

NguyenT11 · ‎12-14-2009

Hi all,

I have two 5K's in a VPC configuration. I'm trying to figure out how to connect the 1000v using CNA's, especially pertaining to FCOE.

Currently we have servers with a 10 gig connection to each 5K using VPC-HM. In addition to the regular traffic/control VLANs, each server port on the 5K's has it's own VLAN to VSAN mapping (5K-1 uses VLAN 100 for VSAN, 5K-2 uses VLAN 200 for VSAN). This appears to be working OK, however I would prefer that we use the VPC on the 5K's instead.

My question is, on the 5K's how do I need to configure the FCOE connection on the VPC? I know I can bind the vfc to the port-channel... but do I use a different VLAN -> VSAN on each 5K for each server? If this is the case, would this not fail the VPC consistency check? Or can I simply use the same VLAN/VSAN on each 5K for that VPC?

Thanks,

pzpgd1mlf · ‎01-16-2010

Curiously, I have the same exact question. Have you been able to sort it out by now?

Regards, --Marina

Ganesh Hariharan · ‎01-16-2010

Hi ,

Check out the below link hope that helps out your query!!

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/nxos/Cisco_Nexus_5000_Series_NX-OS_Software_Configuration_Guide_chapter32.html#topic_64BF925367764DFC9D100CF26D805D22

Regards

Ganesh.H

NguyenT11 · ‎01-17-2010

Thank you, I had already read this document before I posted the question. They do not answer the question I am asking. In particular this line here makes me think that N5K vpc to a CNA equipped server is not possible since you need to use unique VLANs on each 5K to accomodate the VSAN.

"Mapping a VSAN to a VLAN

A unique, dedicated VLAN must be configured at every converged access switch to carry traffic for each Virtual Fabric (VSAN) in the SAN"

I'm ok with using VPC-HM if the answer comes back as my scenario not being supported, however it makes me wonder why I even implemented vpc on the 5K's in the first place.

I'd love to hear if anyone has gotten this to work and how.

Thanks

pzpgd1mlf · ‎01-17-2010

I agree, that statement and also the topologies samples calls for different FCoE VLANs when dealing with dual CNA's, as long as each N5K goes to a different MDS/SAN network. Is that your case as well? Each of your N5K has their own MDS and VSAN's assigned, like SAN A & SAN B?

Now, going over the VSAN trunking as well as vPC documentation more detailed, my assumptions are:

a) The FCoE VLANs are locally significant on a per switch basis while VSAN's can be trunked between N5K's

b) The peer-keepalive link needs to be dedicated with L3 connectivity either through mgmt VRF or default VRF with SVI

c) The peer-link needs to have the same parameters on both sides, including VLANs for consistency checks purposes, which does not match option a)

So, I do not know how vPC is suppose to work with FCoE VLANs, unless they can also be trunked when there is just one SAN coming off both N5K's.

Let's just hope this is not a limitation.

Regards, --Marina

NguyenT11 · ‎01-19-2010

Yes, my setup is exactly as you described, and my concerns are exactly the same as yours

I don't see how this will work with anything other than VPC-HM.

I'll post back if I find anything out.

kanglee · ‎02-07-2010

This is the sample configuration for FCOE over VPC. This toplogy has two seperate path to MDS switches. So I used the seperate vsan and fcoe vlan for each switch. Fcoe vlan is local specific but it has to be defined both N5K same since VPC is doing consistency check.

Then N1K side is just port-channel auto mode on uplink port-profile. However, VPC-HM mac-pining is easy to configure and workes good. However, if you have multicast application then port-channel with N5K is better choice.

##Nexus 5k-01
vlan 25,100,200
vsan database
vsan 100
vsan 200

vlan 100
fcoe vsan 100
vlan 200
fcoe vsan 200

interface vfc4
bind interface port-channel38
no shutdown

vsan database
vsan 200 interface vfc4

interface Ethernet1/4
switchport mode trunk
switchport trunk allowed vlan 1,25,100,200
spanning-tree port type edge trunk
channel-group 38

interface Ethernet1/19
switchport mode trunk
channel-group 1 mode active

interface Ethernet1/20
switchport mode trunk
channel-group 1 mode active

interface port-channel1
switchport mode trunk
vpc peer-link
switchport trunk allowed vlan 1,25
spanning-tree port type network
speed 10000

interface port-channel38
switchport mode trunk
switchport trunk allowed vlan 1,25,100,200
vpc 38
spanning-tree port type edge trunk
speed 10000

##Nexus 5k-02

vlan 25,100,200
vsan database
vsan 100
vsan 200

vlan 100
fcoe vsan 100
vlan 200
fcoe vsan 200

interface vfc38
bind interface port-channel38
no shutdown

vsan database
vsan 100 interface vfc38

interface Ethernet1/38
switchport mode trunk
switchport trunk allowed vlan 1,25,100,200
spanning-tree port type edge trunk
channel-group 38

interface port-channel38
switchport mode trunk
switchport trunk allowed vlan 1,25,100,200
spanning-tree port type edge trunk
vpc 38
speed 10000

interface Ethernet1/19
switchport mode trunk
channel-group 1 mode active

interface Ethernet1/20
switchport mode trunk
channel-group 1 mode active

interface port-channel1
switchport mode trunk
vpc peer-link
switchport trunk allowed vlan 1,25
spanning-tree port type network
speed 10000

pzpgd1mlf · ‎03-07-2010

what about your VLAN 25, is that your data/management VLAN traversing the peer link? If you are not specifying the FCoE VLANs on the peer link, the vpc consistency check should be ok, but I assume if you have any other regular VLANs configured in the N5K for any reason, you will always have to update the peer link with those additional VLANs along with 1, 25.

kanglee · ‎03-09-2010

Vlan 25 is your data vlan and yes, you need to update if there is more data vlans defined.

Nuno Filipe Alves Ferreira · ‎04-20-2010

Hello all,

I have a question regarding a similar set up.

I have a Server (Windows 2008 with Dual port QLogic CNAs) connected to 2 Nexus (Nexus-1 and Nexus2)

QLogic CNAs -> Model : QLE8152

port 1 of the CAN connects to port eth1/1 on Nexus-1. Port 2 of the CNA connects to port eth1/1 of the Nexus-2

Ports eth19-20 on both Nexus are in portchannel 1 which also has the vpc peer-link.

I tried to use the above configuration to create a port-channel to the CNAs. What I did was the following:

Nexus-1

vpc domain 1
role priority 100
peer-keepalive destination 172.16.1.231

vlan 1
vlan 100
fcoe vsan 1
name SAN1_FCOE

interface port-channel1
switchport mode trunk
switchport trunk allowed vlan 1
vpc peer-link
spanning-tree port type network

interface port-channel10
switchport mode trunk
switchport trunk allowed vlan 1,100
spanning-tree port type edge trunk

vpc 10
speed 10000

interface vfc101
bind interface port-channel10
no shutdown

interface Ethernet1/1
switchport mode trunk
switchport trunk allowed vlan 1,100
spanning-tree port type edge trunk
channel-group 10

Nexus-2

vpc domain 1
role priority 200
peer-keepalive destination 172.16.1.230

vlan 1
vlan 200
fcoe vsan 1
name FABRIC2_FCoE

interface port-channel1
switchport mode trunk
switchport trunk allowed vlan 1
vpc peer-link
spanning-tree port type network

interface port-channel10
switchport mode trunk
switchport trunk allowed vlan 1,200
spanning-tree port type edge trunk

vpc 10
speed 10000

interface vfc201
bind interface port-channel10
no shutdown

interface Ethernet1/1
switchport mode trunk
switchport trunk allowed vlan 1,200
spanning-tree port type edge trunk
channel-group 10

Ok so the vPC comes up but I can't ping the host (The host is configured with NIC Teaming - fail over but I wanted Load Balance),

As soon as I just remove the vpc 10 statement from the Port-Channel10 (int po10 ... no vpc 10) it is OK

When I was configuring it it gave me a Warning to check some compatibility (I ignored and to be honest I couldn't reproduce again unfortunately ... and I dont know where that line log went)

Is the CNA not compatible ?I have updated everything to above the versions stated in this document

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/white_paper_c11-569320_v1.pdf

Is this not supported on this CNA ?

Thanks

Nuno Ferreira