12-05-2011 06:53 AM
Environment:
IBM BCH blade in bay 12 with QLogic 10Gb Convergence adapter.
CISCO Nexus 5010 (sw10) ports 3, 4 trunked via port group to BCH Chassis bay 7 ports 15, 16 port group.
First time setting this up. Following the examples guide for the 4001i and 5010 as well as the read the admin guide.
Issue: It seems I have everything setup. I simply need to see the WWPN show up in the fabric so I can map it through to the Fibre Storage devices, but when I run "show fip-snooping vlan-discovery" no MACs show. I manually went into the blade managment module, recorded the MAC and built the vrf table entry, but not working, so I beleive something is missconfigured. Need some guidance... clue I beleive is that no MAC shows up in the FIP query.
Current Configuration (I tried to cut it down to just things related to pertinent data)
Nexus 5010 | Nexus 4001i |
---|---|
feature fcoe feature npiv feature telnet feature lacp feature lldp feature fex class-map type qos class-fcoe class-map type queuing class-all-flood match qos-group 2 class-map type queuing class-ip-multicast match qos-group 2 class-map type network-qos class-all-flood match qos-group 2 class-map type network-qos class-ip-multicast match qos-group 2 vlan 11 name serverdata vlan 28 fcoe vsan 28 name vsan vsan database vsan 28 name "vsan" fcdomain domain 29 static vsan 28 interface port-channel11 description sw10-03_04 to sw14-15_16 switchport mode trunk switchport trunk native vlan 11 speed 10000 interface vfc112 bind mac-address 00:c0:dd:17:8e:10 switchport description bch112 no shutdown vsan database vsan 28 interface vfc9 vsan 28 interface vfc10 vsan 28 interface vfc28 vsan 28 interface vfc112 vsan 28 interface fc2/1 vsan 28 interface fc2/2 vsan 28 interface fc2/3 vsan 28 interface fc2/4 vsan 28 interface fc2/5 vsan 28 interface fc2/6 vsan 28 interface fc2/7 vsan 28 interface fc2/8 interface fc2/1 switchport description fc01-09-12 (E Mode Port) no shutdown interface Ethernet1/3 description sw14-15 switchport mode trunk switchport trunk native vlan 11 channel-group 11 interface Ethernet1/4 description sw14-16 switchport mode trunk switchport trunk native vlan 11 channel-group 11 boot kickstart bootflash:/n5000-uk9-kickstart.5.0.2.N2.1.bin boot system bootflash:/n5000-uk9.5.0.2.N2.1.bin | feature telnet feature lacp feature fip-snooping policy-map type queuing policy-fcoe-bandwidth class type queuing 1p7q4t-out-q-default bandwidth percent 50 class type queuing 1p7q4t-out-pq1 bandwidth percent 7 class type queuing 1p7q4t-out-q2 bandwidth percent 7 class type queuing 1p7q4t-out-q3 bandwidth percent 7 class type queuing 1p7q4t-out-q4 bandwidth percent 7 class type queuing 1p7q4t-out-q5 bandwidth percent 7 class type queuing 1p7q4t-out-q6 bandwidth percent 7 class type queuing 1p7q4t-out-q7 bandwidth percent 7 class-map type network-qos class-fcoe match cos 3 class-map type network-qos class-non-fcoe match cos 0-2,4-7 policy-map type network-qos policy-fcoe class type network-qos class-fcoe pause no-drop mtu 2500 class type network-qos class-non-fcoe system qos service-policy type network-qos policy-fcoe service-policy type queuing output policy-fcoe-bandwidth vlan 11 name serverdata vlan 28 fip-snooping enable fip-snooping fc-map 0x0efc00 name vsan port-channel load-balance ethernet source-dest-mac interface port-channel1 description sw10-03_04 to sw14-15_16 switchport mode trunk switchport trunk native vlan 11 spanning-tree port type edge trunk speed 10000 interface Ethernet1/12 description bch112 switchport mode trunk switchport trunk native vlan 11 fip-snooping port-mode fcf spanning-tree port type edge speed 10000 interface Ethernet1/15 description sw10-03 switchport mode trunk switchport trunk native vlan 11 speed 10000 channel-group 1 interface Ethernet1/16 description sw10-04 switchport mode trunk switchport trunk native vlan 11 speed 10000 channel-group 1 boot kickstart bootflash:/n4000-bk9-kickstart.4.1.2.E1.1d.bin boot system bootflash:/n4000-bk9.4.1.2.E1.1d.bin |
sw10# show license usage Feature Ins Lic Status Expiry Date Comments Count -------------------------------------------------------------------------------- FM_SERVER_PKG No - Unused - ENTERPRISE_PKG Yes - Unused Never - FC_FEATURES_PKG Yes - In use Never - -------------------------------------------------------------------------------- sw10# sh int po 11 port-channel11 is up sw10# show vsan membership vsan 1 interfaces: vsan 28 interfaces: fc2/1 fc2/2 fc2/3 fc2/4 fc2/5 fc2/6 fc2/7 fc2/8 vfc9 vfc10 vfc28 vfc112 sw10# show vlan fcoe Original VLAN ID Translated VSAN ID Association State ---------------- ------------------ ----------------- 28 28 Operational | sw14# show license usage Feature Ins Lic Status Expiry Date Comments Count -------------------------------------------------------------------------------- BASIC_STORAGE_SERVICES_PKG No - In use Grace 96D 1H -------------------------------------------------------------------------------- sw14# sh int po1 port-channel1 is up sw14# sh port-channel summary -------------------------------------------------------------------------------- Group Port- Type Protocol Member Ports Channel -------------------------------------------------------------------------------- 1 Po1(SU) Eth NONE Eth1/15(P) Eth1/16(P) sw14# show fip-snooping vlan-discovery sw14# sh int e1/12 Ethernet1/12 is up Hardware: 1000/10000 Ethernet, address: 588d.090e.cfcd (bia 588d.090e.cfcd) |
Any thoughts on what I am missing would be appreciated.
Thanks,
Solved! Go to Solution.
12-07-2011 01:23 PM
the fip-snooping port-mode fcf command must be applied on the uplink etherchannel, not the server facing port.
Change your config and reboot.
12-07-2011 02:10 PM
your config is wrong. The nexus 4000 has to know where to forward FIP messages, and the command is applied on the wrong interface
12-07-2011 01:23 PM
the fip-snooping port-mode fcf command must be applied on the uplink etherchannel, not the server facing port.
Change your config and reboot.
12-07-2011 02:03 PM
No details on whichever interface I try to query.
sw14# show fip-snooping vlan-discovery interface ?
ethernet Ethernet IEEE 802.3z
port-channel Port Channel interface
sw14# show fip-snooping vlan-discovery interface port-channel 1
sw14# show fip-snooping vlan-discovery interface ethernet 1/12
sw14#
Is this some kind of real time query in that the adapter must have an active kernel / driver sending out packets. I booted the server which rotates through the PXE boot mode which would (I A$$ume) accomolish this.
12-07-2011 02:10 PM
your config is wrong. The nexus 4000 has to know where to forward FIP messages, and the command is applied on the wrong interface
12-08-2011 05:40 AM
Thanks for the response but If you could provide some better guidance as to the correct command and where my following of the configuation guide was not correct:
(page 8) Enable FIP Snooping on the Port channel on the 4000 for link between the 4000 and 5000
interface port-channel1
description sw10-03_04 to sw14-15_16
switchport mode trunk
switchport trunk native vlan 11
fip-snooping port-mode fcf
spanning-tree port type edge trunk
speed 10000
I was missing this and now when I run
sw14# show fip-snooping vlan-discovery
Legend:
-------------------------------------------------------------------------------
Interface VLAN FIP MAC
-------------------------------------------------------------------------------
Eth1/2 1 00:c0:dd:17:8d:f1
I see fip mac.... but not the slot 12 like I had defined. I will have to review this again to see what I am missing.
12-08-2011 05:42 AM
Issue a shut / no shut on the interface 1/12 ? Do you see LLDP negociation correctly ?
12-09-2011 08:39 AM
The issue was that my host facing port configuation was incorrect:
Incorrect:
interface Ethernet1/12
description bch112
switchport mode trunk
switchport trunk native vlan 11
fip-snooping port-mode fcf
spanning-tree port type edge
speed 10000
Once I changed it to:
interface Ethernet1/12
description bch112
switchport mode trunk
switchport trunk native vlan 11
spanning-tree port type edge trunk
speed 10000
then the fip snooping started showing devicse
sw14# show fip-snooping vlan-discovery
Legend:
-------------------------------------------------------------------------------
Interface VLAN FIP MAC
-------------------------------------------------------------------------------
Eth1/14 11 00:c0:dd:17:90:5d
Eth1/12 11 00:c0:dd:17:8e:11
Eth1/2 1 00:c0:dd:17:8d:f1
Now I added the binding to the vfc interface on the 5k...
interface vfc102
bind mac-address 00:c0:dd:17:8d:f1
switchport description bch102
no shutdown
interface vfc112
bind mac-address 00:c0:dd:17:8e:11
switchport description bch112
no shutdown
interface vfc114
bind mac-address 00:c0:dd:17:90:5d
switchport description bch114
no shutdown
Make sure the vfc are in the VSAN:
sw10# conf t
Enter configuration commands, one per line. End with CNTL/Z.
sw10(config)# vsan database
sw10(config)# show vsan membership
vsan 1 interfaces:
vsan 28 interfaces:
fc2/1 fc2/2 fc2/3 fc2/4
fc2/5 fc2/6 fc2/7 fc2/8
vfc9 vfc10 vfc102 vfc112
vfc113 vfc114
(Redbook and CISCO guide shows that I should be done here and the adapters show up in MDS Manager)
Now that the interfaces are present in the VSAN I would "assume" they would show up in fabric manager.. but they do not. I go into BIOS for QLogic (ctrl+Q) and tell it to scan the SAN, but the device still do not show.
I think something is wrong with the VSAN binding..
sw10# sh interface vfc114
vfc114 is trunking (Not all VSANs UP on the trunk)
Bound MAC is 00:c0:dd:17:90:5d
Port description is bch114
Hardware is Virtual Fibre Channel
Port WWN is 20:71:00:05:9b:21:60:bf
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
sw10# sh interface port-channel 11
port-channel11 is up
Hardware: Port-Channel, address: 0005.9b21.608a (bia 0005.9b21.608a)
Description: sw10-03_04 to sw14-15_16
MTU 1500 bytes, BW 20000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s
Beacon is turned off
Input flow-control is off, output flow-control is off
Switchport monitor is off
EtherType is 0x8100
Members in this channel: Eth1/3, Eth1/4
Last clearing of "show interface" counters never
30 seconds input rate 48 bits/sec, 6 bytes/sec, 0 packets/sec
30 seconds output rate 173032 bits/sec, 21629 bytes/sec, 73 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 112 bps, 0 pps; output rate 91.87 Kbps, 31 pps
VSAN 28 shows status as "active" and oper state of "up"
I am not sure about the "Inorder Delivery". No details on this and when I tried no change seems to have been made.
You have been very helpful and it is appreciated. Any thought would be of help
12-11-2011 11:58 PM
Is there anything not working now ? Everything seems to be good to me.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide