09-14-2010 10:40 AM
I am having problems getting my interface to come up for this configuration. We have installed the UTA (FCoE target cards) in our NetApp heads and cabled them to our Nexus 5Ks. I have the Nexus interfaces configured as follows.
vlan 1010
fcoe vsan 10
name Storage_Mapped_VSAN10
interface e1/15
description NetApp-3140-01-3A
switchport mode trunk
switchport trunk native vlan 1010
switchport trunk allowed vlan 1010
spanning-tree port type edge trunk
interface vfc115
bind interface e1/15
no shut
vsan database
vsan 10 interface vfc115
There is nowhere in the NetApp device to specify the VLAN information for them so I tried doing it as an access port and as a trunk port making the VLAN on the Nexus the native VLAN and only allowed VLAN but it still does not work. It shows up on the NetApp side as Link Not Connected. I don't know if it really matters on the fact that it is running in a NetApp or not so I'll include the QLogic part number I'm dealing with as well.
Fibre Channel Target Adapter 3a (Dual-channel, QLogic CNA 8112 (8152) rev. 2)
Any ideas would be appreciated.
09-15-2010 09:07 AM
Configuration should not cause the link to not come up. If the interfaces show up as not connected or down, I would first look into the
physical layer. After this is taken care of, I see you are not allowing your FCoE VLAN on the trunk. You will need to correct this.
10-07-2010 12:21 PM
I'm having the same exact issue -- what am i missing ?
did u get a solution?
10-08-2010 05:17 AM
No. I have not gotten anywhere yet. I can see the Ethernet side as up in the NetApp but the FC interface is still down in the NetApp and the vFC interface is down in the Nexus. I got absolutely no where with the NetApp case I opened and gave up. I will probably start looking at it again next week. Let me know if you figure it out between now and then. I think there has to be a privileged command in the NetApp side to configure the FCoE VLAN or something.
You can see the CNA properties under privileged options in the NetApp which shows the Ethernet Link status as up.
netapp-3140-01*> cna show
Slot: 3
Description: Dual-channel, QLogic CNA 8112(8152) rev. 2
FC Adapter Name: 3a
MPI Firmware Rev: 1.38.0
PHY Firmware Rev: 1.6.0
FC Firmware Rev: 5.2.0
FC Port ID: 0x000000
FC Nodename: 50:0a:09:80:89:ca:c7:41 (500a098089cac741)
FC Portname: 50:0a:09:81:99:ca:c7:41 (500a098199cac741)
FC Status: LINK NOT CONNECTED
Link Speed: 10 GBit
Link Status: LINK UP
Slot: 3
Description: Dual-channel, QLogic CNA 8112(8152) rev. 2
FC Adapter Name: 3b
MPI Firmware Rev: 1.38.0
PHY Firmware Rev: 1.6.0
FC Firmware Rev: 5.2.0
FC Port ID: 0x000000
FC Nodename: 50:0a:09:80:89:ca:c7:41 (500a098089cac741)
FC Portname: 50:0a:09:82:99:ca:c7:41 (500a098299cac741)
FC Status: LINK NOT CONNECTED
Link Speed: 10 GBit
Link Status: LINK UP
10-08-2010 05:27 AM
I got mine working.
What data ontap are you using?
10-08-2010 05:44 AM
ONTAP 7.3.4
How did you configure it to work?
10-08-2010 10:33 AM
I'll share my configs with u when I get home to night. I'm running ontap 8.01rc1. I had the same issue. All I did was clear the configs and reboot the heads and the links came up
Sent from my iPhone
10-08-2010 08:36 PM
Here are my Configs that worked
I have 2 N5020 with a 3140HA running ontap 8.0.1RC1
The CNA's are on e1/9 and e1/10 on each n5k switch.. I've only show the onse side but it pretty much the same for the other side only differecen is the vsan #. I've rebooted the heads a numbe of times with no issues. I can see the LUNS from MY UCS Blade servers.
---- MY Nexus 5020 -----
NK5-01# sh run
version 4.1(3)N2(1)
feature npiv
feature fcoe
vlan 1010
fcoe vsan 1000
vsan database
vsan 1000 name "Fabric-A"
interface vfc9
bind interface Ethernet1/9
no shutdown
interface vfc10
bind interface Ethernet1/10
no shutdown
vsan database
vsan 1000 interface vfc9
vsan 1000 interface vfc10
vsan 1000 interface fc2/1
vsan 1000 interface fc2/2
interface fc2/1
no shutdown
interface fc2/2
no shutdown
interface Ethernet1/9
switchport mode trunk
spanning-tree port type edge trunk
interface Ethernet1/10
switchport mode trunk
spanning-tree port type edge trunk
zone name vSphere vsan 1000
member pwwn 50:0a:09:83:89:ab:8b:e5
member pwwn 50:0a:09:83:99:ab:8b:e5
member pwwn 20:00:00:25:b5:00:00:31
zoneset name UCS6100 vsan 1000
member vSphere
zoneset activate name UCS6100 vsan 1000
NK5-01# sh flogi database
--------------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
--------------------------------------------------------------------------------
fc2/1 1000 0x050003 20:41:00:05:73:a9:8a:c0 23:e8:00:05:73:a9:8a:c1
fc2/1 1000 0x050004 20:00:00:25:b5:00:00:31 20:00:00:25:b5:01:01:8f
fc2/1 1000 0x050006 20:00:00:25:b5:00:00:5f 20:00:00:25:b5:01:01:6f
vfc9 1000 0x050001 50:0a:09:83:99:ab:8b:e5 50:0a:09:80:89:ab:8b:e5
vfc10 1000 0x050002 50:0a:09:83:89:ab:8b:e5 50:0a:09:80:89:ab:8b:e5
Total number of flogi = 5.
============== 3140 Head A with Ontap 8.0.1RC1 7-Mode
3140A*> rdfile /etc/rc
hostname 3140A
ifgrp create single c10g-a e3a e3b
vlan create c10g-a 11 26
ifconfig c10g-a-11 xxx.xxx.xxx.xxx netmask 255.255.255.0 partner c10g-b-11
ifconfig c10g-a-26 xxx.xxx.xxx.xxx netmask 255.255.255.0 partner c10g-b-26
route add default xxx.xxx.xxx.xxx 1
routed on
options dns.enable off
options nis.enable off
savecore
3140A*> sysconfig -v 3
slot 3: Dual 10G Ethernet Controller CNA SFP+
(Dual-port, QLogic CNA 8112(8152) rev. 2)
e3a MAC Address: 00:c0:dd:15:da:b8 (auto-10g_sr-fd-up)
e3b MAC Address: 00:c0:dd:15:da:ba (auto-10g_sr-fd-up)
Device Type: ISP8112
slot 3: Fibre Channel Target Host Adapter 3a
(Dual-channel, QLogic CNA 8112 (8152) rev. 2,
Firmware rev: 5.3.4
Host Port Addr: 050001
Cacheline size: 16
SRAM parity: Yes
FC Nodename: 50:0a:09:80:89:ab:8b:e5 (500a098089ab8be5)
FC Portname: 50:0a:09:83:99:ab:8b:e5 (500a098399ab8be5)
Connection: PTP, Fabric
SFP Vendor Name: FINISAR CORP.
SFP Vendor P/N: FTLX8571D3BCL-QL
SFP Vendor Rev: A
SFP Serial No.: AH60DND
SFP Connector: LC
SFP Capabilities: 10 Gbit/Sec
slot 3: Fibre Channel Target Host Adapter 3b
(Dual-channel, QLogic CNA 8112 (8152) rev. 2,
Firmware rev: 5.3.4
Host Port Addr: 880001
Cacheline size: 16
SRAM parity: Yes
FC Nodename: 50:0a:09:80:89:ab:8b:e5 (500a098089ab8be5)
FC Portname: 50:0a:09:84:99:ab:8b:e5 (500a098499ab8be5)
Connection: PTP, Fabric
SFP Vendor Name: FINISAR CORP.
SFP Vendor P/N: FTLX8571D3BCL-QL
SFP Vendor Rev: A
SFP Serial No.: AH60DDE
SFP Connector: LC
SFP Capabilities: 10 Gbit/Sec
3140A*> fcp show adapter
Slot: 3a
Description: Fibre Channel Target Adapter 3a (Dual-channel, QLogic CNA 8112 (8152) rev. 2)
Adapter Type: Local
Status: ONLINE
FC Nodename: 50:0a:09:80:89:ab:8b:e5 (500a098089ab8be5)
FC Portname: 50:0a:09:83:99:ab:8b:e5 (500a098399ab8be5)
Standby: No
Slot: 3b
Description: Fibre Channel Target Adapter 3b (Dual-channel, QLogic CNA 8112 (8152) rev. 2)
Adapter Type: Local
Status: ONLINE
FC Nodename: 50:0a:09:80:89:ab:8b:e5 (500a098089ab8be5)
FC Portname: 50:0a:09:84:99:ab:8b:e5 (500a098499ab8be5)
Standby: No
Hope this helps... Perhaps updaint you firmware might help.... I did not specifiy and vlans for the CNA for the FCOE on the netapp.
10-12-2010 05:27 AM
I appreciate you putting this out here. I do have one question from the configs post above. In the Nexus 5k, you appear to use VLAN 1010 as your FCoE VLAN which is then mapped to VSAN 1000. I see where you have all of the configuration for VSAN 1000 done, but in the NetApp, you do not appear to have VLAN 1010 configured, only 11 and 26. If this is due to cleansing your configs before posting, I understand, but if not, please explain where that VLAN definition is on the NetApp side as I think that is what I'm missing.
I'm probably going to go back to NOW support and try again later this week. In ONTAP 8.0.1rc, the adapters support all functionality like a true CNA. You can even run iSCSI and CIFS/NFS traffic across them as they show up as actual Ethernet interfaces with the ifconfig options available. Unfortunately, in ONTAP 7.3.4 they are seen by the system but only for FCoE access which is fine except that the ability to configure the Ethernet side seems to be absent.
If I come up with a solution for mine, aside from another ONTAP upgrade, I will be sure to post it here...
09-27-2011 06:04 AM
Did you ever figure this out, how the fcoe vlan gets configured on the filer side?
09-27-2011 06:26 AM
Negative. We essentially came to terms with the fact that the adapter is recognized in 7.3.4 but doesn't have enough functionality to use it. We will be upgrading the system soon and running a recent version of ONTAP 8 which has full support for the adapter and we hope to get it going then for iSCSI and FCoE.
09-27-2011 06:57 AM
That is correct. For full CNA support, you will need Ontap 8.0.x 7-mode. Ontap 7.3.4 or7.3.5 has limited CNA support. There is no FCoE vlan configuration on the Filer. All of that is done on the 5K as shown above.
I did a small blog a yar ago -- http://blog.bhagat.com/wordpress/2010/11/cisco-ucs-vmware-netapp-nexus-fcoe-part1-4/
05-16-2012 04:15 AM
Actually, it looks like the difference here is that the working config by kbhagat has only 2 interfaces per FAS controller. In Justin's config he has 4 ports per FAS controller. We have the same issue. apparently you can not create a VPC accross two Nexus 5K switches with more than 1 port each *IF* and only *IF* you are using FCOE. See page 15 of the document below.
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572835-00_NX-OS_vPC_DG.pdf
Now, here is my question. Can you create a regular port channel accross the e1a and e2a ports (PC101) and a regular port channel accross the e1b and e2b ports (PC102). Then a VPC accross PC101 and PC102??
Would this work???
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide