Nexus 7010 fabric extender timing out

cbuzzard · ‎04-25-2011

Hello -

We have a Nexus 7010 and we are testing out using the fabric extenders for a need. We have a demo 2224 unit and have it connected to our M132XP-12 10G blade in the 7K but the FEX won't come online. I would have figured a possible software incompatability but looking at the supported list for that as well as hardware everything seems to be in order. This is what the status shows after it spends about 15 mins in the image download state.

FEX: 111 Description: FEX0111 state: Offline

FEX version: 4.2(1)N2(1a) [Switch version: 5.1(2)]

FEX Interim version: 4.2(1)N2(1a)

Switch Interim version: 5.1(2)

Module Sw Gen: 21 [Switch Sw Gen: 21]

pinning-mode: static Max-links: 1

Fabric port for control traffic: Eth2/20

Fabric interface state:

Po11 - Interface Up. State: Active

Eth2/20 - Interface Up. State: Active

Fex Port State Fabric Port Primary Fabric

This is looped in the log details until it times out:

04/25/2011 15:31:41.986978: Module register received

04/25/2011 15:31:41.987713: Registration response sent

04/25/2011 15:31:41.987889: Requesting satellite to download image

04/25/2011 15:32:00.105031: Module register received

04/25/2011 15:32:00.105779: Registration response sent

04/25/2011 15:32:00.105956: Requesting satellite to download image

04/25/2011 15:32:20.191181: Module register received

04/25/2011 15:32:20.191957: Registration response sent

04/25/2011 15:32:20.192144: Requesting satellite to download image

We ran a debug during this and these entries are displayed when looking for errors.

2011 Apr 25 15:30:31.443745 fex: Reg resp: Failed to get card info for swcardid 132

2011 Apr 25 15:30:35.472721 fex: Cardinfo: Unknown card id to get (132)

2011 Apr 25 15:30:35.472753 fex: Reg resp: Failed to get card info for swcardid 132

2011 Apr 25 15:30:41.495302 fex: Cardinfo: Unknown card id to get (132)

I'm still doing some more searching which so far hasn't turned up much, wanted to see if anyone has some other insight??

Thanks!

jacyu2 · ‎04-27-2011

Hi,

Did you upgrade the EPLD to 5.1?

Regards,

Jack

cbuzzard · ‎04-28-2011

Hi Jack -

Thanks for the response. Unfortunately, yes that is already complete. I was hoping that would be an easy fix. When we upgraded to 5.1(2) we did the 5.1 EPLD. I ran the install all impact command noted below for the 5.1 EPLD just to make sure it didn't report anything else as needing upgrade.

sho install all impact epld bootflash:n7000-s1-epld.5.1.1.img

Compatibility check:

Module Type Upgradable Impact Reason

------ ---- ---------- ---------- ------

1 LC Yes disruptive Module Upgradable

2 LC Yes disruptive Module Upgradable

5 SUP Yes disruptive Module Upgradable

1 Xbar Yes disruptive Module Upgradable

2 Xbar Yes disruptive Module Upgradable

3 Xbar Yes disruptive Module Upgradable

1 FAN Yes disruptive Module Upgradable

2 FAN Yes disruptive Module Upgradable

3 FAN Yes disruptive Module Upgradable

4 FAN Yes disruptive Module Upgradable

Copy complete, now saving to disk (please wait)...

Retrieving EPLD versions... Please wait.

Images will be upgraded according to following table:

Module Type EPLD Running-Version New-Version Upg-Required

------ ---- ------------- --------------- ----------- ------------

1 LC Power Manager 4.008 4.008 No

1 LC IO 1.006 1.006 No

1 LC Forwarding Engine 1.006 1.006 No

1 LC SFP 1.004 1.004 No

2 LC Power Manager 4.008 4.008 No

2 LC IO 1.016 1.016 No

2 LC Forwarding Engine 1.006 1.006 No

2 LC FE Bridge(1) 186.006 186.006 No

2 LC FE Bridge(2) 186.006 186.006 No

2 LC Linksec Engine(1) 2.006 2.006 No

2 LC Linksec Engine(2) 2.006 2.006 No

2 LC Linksec Engine(3) 2.006 2.006 No

2 LC Linksec Engine(4) 2.006 2.006 No

2 LC Linksec Engine(5) 2.006 2.006 No

2 LC Linksec Engine(6) 2.006 2.006 No

2 LC Linksec Engine(7) 2.006 2.006 No

2 LC Linksec Engine(8) 2.006 2.006 No

5 SUP Power Manager 3.009 3.009 No

5 SUP IO 3.028 3.028 No

5 SUP Inband 1.008 1.008 No

5 SUP Local Bus CPLD 3.000 3.000 No

5 SUP CMP CPLD 6.000 6.000 No

1 Xbar Power Manager 2.010 2.010 No

2 Xbar Power Manager 2.010 2.010 No

3 Xbar Power Manager 2.010 2.010 No

1 FAN Fan Controller (1) 0.007 0.007 No

1 FAN Fan Controller (2) 0.007 0.007 No

2 FAN Fan Controller (1) 0.007 0.007 No

2 FAN Fan Controller (2) 0.007 0.007 No

3 FAN Fan Controller (1) 0.007 0.007 No

3 FAN Fan Controller (2) 0.007 0.007 No

4 FAN Fan Controller (1) 0.007 0.007 No

4 FAN Fan Controller (2) 0.007 0.007 No

jacyu2 · ‎04-28-2011

Hi,

The EPLD is OK to support FEX. Could you double check if the SFP is supported, in case you use copper connection.

http://www.cisco.com/en/US/partner/docs/switches/datacenter/hw/nexus7000/installation/guide/n7k_connect_specs.html#wp1044250

Also, please be aware that 5.1.2 is a deffer code. It would be better if you could upgrade to 5.1.3 in your production enviroment.

http://www.cisco.com/cisco/software/advisory.html?imageGuId=0E71CAA06C8E4F37F7DBA85136033F14D66356B3&i=rs

if the issue is the same after upgrade, please open a TAC case.

Cheers, Jack

cbuzzard · ‎04-29-2011

FYI - I finally got confirmation from Cisco that the 2224 isn't supported on this release even though the N2K document says it is. The 2224 support comes with the Delhi release, only the 2248 is supported right now.