Re: ASK THE EXPERTS - NEXUS 5000 AND 2000 SERIES - Page 3

ciscomoderator · ‎08-13-2010

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to learn how to troubleshoot Nexus 5000 & 2000 series with Lucien Avramov. Lucien Avramov is a Customer Support Engineer at the Cisco Technical Assistance Center. He currently works in the data center switching team supporting customers on the Cisco Nexus 5000 and 2000. He was previously a technical leader within the network management team. Lucien holds a bachelor's degree in general engineering and a master's degree in computer science from Ecole des Mines d'Ales. He also holds the following certifications: CCIE #19945 in Routing and Switching, CCDP, DCNIS, and VCP #66183.

Remember to use the rating system to let Lucien know if you have received an adequate response.

Lucien might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through August 30, 2010. Visit this forum often to view responses to your questions and the questions of other community members.

Lucien Avramov · ‎08-27-2010

You can certainly have: ESX--(fcoe)--N5K---SAN. Also VPC to the N5K along with FCoE is supported.

Regarding the question is VPC-HM better than VPC, it depends if you have also a Nexus 1000v in the picture. In the above, it can work directly with the Nexus 5000, which looks simpler at this point to me.

In typical FC topologies, the vfc are connected to different switches. The diagram attached respects the SAN A / SAN B boundaries.

FCoE configuration guide:

http://www.cisco.com/en/US/partner/docs/switches/datacenter/nexus5000/sw/fcoe/421_n1_1/Cisco_n5k_fcoe_config_gd_re_421_n1_1_chapter3.html

Quick FCoE example:

Here are the steps needed to configure the Nexus 5000 for FCoE capability. In this example, we will use the default VSAN 1 for the virtual fibre channel (vfc) interface. The server has a CNA connected to the Nexus 5010 on interface ethernet 1/10.

1. Create a VLAN for FCoE traffic (Configured through CLI only)

switch# configure terminal
switch(config)# vlan 50
switch(config-vlan)# fcoe vsan 1

2. Create the virtual fibre channel interface (Configured through CLI or GUI)

switch(config)# interface vfc 10 
switch(config-if)# bind interface ethernet 1/10
switch(config-if)# no shut

Note: the vfc interface number can be any number ranging from 1 - 8019

3. Configure Ethernet interface to allow FCoE VLAN to traverse the interface (Configured through CLI only)

switch(config)# interface ethernet 1/10
switch(config-if)# switchport mode trunk 
switch(config-if)# switchport trunk allowed vlan 1, 50

Note: since FCoE traffic is on VLAN 50 and regular Ethernet traffic is in VLAN 1, 
the interface will need to be in trunk mode

4. Validate vfc interface is up (Done through CLI or GUI)

switch# show interface vfc 10
vfc10 is up
Bound interface is Ethernet1/10
    Hardware is GigabitEthernet
    Port WWN is 20:00:00:0d:ec:a4:26:7f
    Admin port mode is F
    snmp link state traps are enabled
    Port mode is F, FCID is 0x660001
    Port vsan is 1
    Beacon is turned unknown
    5 minutes input rate 1164208 bits/sec, 145526 bytes/sec, 480 frames/sec
    5 minutes output rate 5024 bits/sec, 628 bytes/sec, 0 frames/sec
      81498131 frames input, 164638973896 bytes
        0 discards, 0 errors
      8986261 frames output, 4902182712 bytes
        0 discards, 0 errors

In case you have an UCS, the following may be useful:

http://www.cisco.com/en/US/partner/products/ps10281/products_configuration_example09186a0080afd130.shtml

NguyenT11 · ‎08-27-2010

Thanks for the reply. Sorry, I should've been more specific with my question. Let me try again.

We have ESX hosts running N1000v with uplinks to 2xN5K using CNA adapters, 1 FC fabric off of each N5K, exactly like your diagram. I'm running VPC-HM on the N1000V, everything is working as expected with FCOE and such.

Rather than VPC-HM, I'd like to run a VPC from the N5K's to the ESX hosts. The part that I'm a little confused about is that, according to the documents, since each VSAN requires a uniques VLAN on each N5K, the VPC consistency check will fail since the VPC interface (trunk) on each N5K must have the same VLANs allowed on the trunk.

As in the diagram, I have VLAN 900 -> VSAN 900 for Fabric A and VLAN 901-> VSAN 901 for Fabric B, VPC 10 will have different allow VLANs on each N5K, and will fail the vpc consistency check. I can probably get this running by trunking both VLAN 900 and 901 on the vpc 10 configs on both N5K, and only do the VLAN-VSAN mapping on the necessary VLAN...

Specifically I was asking for a sample config of this scenario and looking for confirmation that the recommended configuration would have me trunking both VLANs on both N5K in order to pass the consistency check for VPC.

Thanks

TN

Lucien Avramov · ‎08-27-2010

Okay, got it.

You don't need to do VPC to the 5Ks if you have already VPC-HM.

Just have regular port-channels to the upstream switches and use two different subgroups.

Look at the following:

https://www.cisco.com/en/US/docs/switches/datacenter/nexus1000/sw/4_0/interface/configuration/guide/if_5portchannel.html