FEX 2232TM not recognized by Nexus 5010P

PrimergyQA · ‎07-27-2012

Hi there,

I don't have any clue what this problem might be so please help.

I have a brandnew FEX 2232TM which is connected to a NEXUS 5010P running the following firmware:

oftware

BIOS: version 1.5.0

loader: version N/A

kickstart: version 5.2(1)N1(1)

system: version 5.2(1)N1(1)

power-seq: Module 1: version v1.2

BIOS compile time: 11/30/10

kickstart image file is: bootflash:///n5000-uk9-kickstart.5.2.1.N1.1.bin

kickstart compile time: 7/12/2012 19:00:00 [07/13/2012 02:08:13]

system image file is: bootflash:///n5000-uk9.5.2.1.N1.1.bin

system compile time: 7/12/2012 19:00:00 [07/13/2012 03:23:16]

According to the Cisco download page it is the latest version and should definitely support the 2232TM.

The problem is that the FEX is completely not recognized by the parent Nexus.

It is connected via 8 FET modules. But when ich check the light on both sides I only have light on the NEXUS side never on the FEX side on all modules.

From my understanding this should be the root cause of the issue because no light = no link.

I tried standard 10GBase-SR and Twinax cables as well. Same behavior - the FEX is not recognized.

The status LED if of course orange on the FEX.

This is the FEX related config ont he NEXUS:

feature fex

fex 101

pinning max-links 8

description "FEX0101"

interface Ethernet1/1

switchport mode fex-fabric

fex associate 101

interface Ethernet1/2

switchport mode fex-fabric

fex associate 101

interface Ethernet1/3

switchport mode fex-fabric

fex associate 101

interface Ethernet1/4

switchport mode fex-fabric

fex associate 101

interface Ethernet1/5

switchport mode fex-fabric

fex associate 101

interface Ethernet1/6

switchport mode fex-fabric

fex associate 101

interface Ethernet1/7

switchport mode fex-fabric

fex associate 101

interface Ethernet1/8

switchport mode fex-fabric

fex associate 101

Can anyone give me a hint what I am missing here??

Any help is really appreciated.

Thanks a lot!

Reza Sharifi · ‎07-27-2012

Hi,

What is the output of "sh fex"?

Also, are you using Multimode fiber patch cords?

HTH

PrimergyQA · ‎07-30-2012

Hi Reza,

thanks for answering.

The "sh fex" output shows nothing which is actually expected because the FEX wasn't discovered yet.

And yes Multimode fiber cables are used for the setup. But as mentioned before I can see no light in the transceiver on the FEX side. The problem is I dont' know why :-(

Leo Laohoo · ‎07-30-2012

But as mentioned before I can see no light in the transceiver on the FEX side.

Faulty SFP or faulty fibre optic patch cable?

PrimergyQA · ‎07-31-2012

I am using 8 ports so it's very unlikely that all transceivers and cables are defect.

I have also connected this FEX to another NEXUS -> no success

and also another FEX to this NEXUS -> success

So it looks like the 2232TM has a hardware problem.

vdsudame · ‎07-31-2012

I have a two lab switches running 5.2.1.N1.1 and connected in vpc with 2232TM and I am able to see the fex online without any problems. One side I have SFP-10G-SR and other side I have 3M Cisco Twinax cable.

here is the output from my lab,

5010-1# show fex

FEX FEX FEX FEX

Number Description State Model Serial

------------------------------------------------------------------------

100 FEX0100 Online N2K-C2232TM-10GE SSI15380CPM

5010-1# show fex 100 detail

FEX: 100 Description: FEX0100 state: Online

FEX version: 5.2(1)N1(1) [Switch version: 5.2(1)N1(1)]

FEX Interim version: 5.2(1)N1(1)

Switch Interim version: 5.2(1)N1(1)

Extender Serial: SSI15380CPM

Extender Model: N2K-C2232TM-10GE, Part No: 73-13626-03

Card Id: 164, Mac Addr: 28:94:0f:2e:12:82, Num Macs: 64

Module Sw Gen: 21 [Switch Sw Gen: 21]

post level: complete

Pinning-mode: static Max-links: 1

Fabric port for control traffic: Eth1/1

FCoE Admin: false

FCoE Oper: true

FCoE FEX AA Configured: false

Fabric interface state:

Po100 - Interface Up. State: Active

Eth1/1 - Interface Up. State: Active

Fex Port State Fabric Port

5010-1# show interface e1/1 transceiver details

Ethernet1/1

transceiver is present

type is SFP-H10GB-CU3M rev A

name is CISCO-MOLEX INC

part number is 74752-9026

revision is A

serial number is MOL12240343

nominal bitrate is 12000 MBit/sec

Link length supported for copper is 3 m

cisco id is --

cisco extended id number is 4

DOM is not supported

5010-2# show fex

FEX FEX FEX FEX

Number Description State Model Serial

------------------------------------------------------------------------

100 FEX0100 Online N2K-C2232TM-10GE SSI15380CPM

5010-2# show fex 100 detail

FEX: 100 Description: FEX0100 state: Online

FEX version: 5.2(1)N1(1) [Switch version: 5.2(1)N1(1)]

FEX Interim version: 5.2(1)N1(1)

Switch Interim version: 5.2(1)N1(1)

Extender Serial: SSI15380CPM

Extender Model: N2K-C2232TM-10GE, Part No: 73-13626-03

Card Id: 164, Mac Addr: 28:94:0f:2e:12:82, Num Macs: 64

Module Sw Gen: 21 [Switch Sw Gen: 21]

post level: complete

Pinning-mode: static Max-links: 1

Fabric port for control traffic: Eth1/1

FCoE Admin: false

FCoE Oper: true

FCoE FEX AA Configured: false

Fabric interface state:

Po100 - Interface Up. State: Active

Eth1/1 - Interface Up. State: Active

Fex Port State Fabric Port

5010-2# show interface e1/1 transceiver details

Ethernet1/1

transceiver is present

type is 10Gbase-SR

name is CISCO-AVAGO

part number is SFBR-7700SDZ-CS2

revision is B5

serial number is AGD133133S4

nominal bitrate is 10300 MBit/sec

Link length supported for 50/125um OM2 fiber is 82 m

Link length supported for 50/125um OM3 fiber is 300 m

Link length supported for 62.5/125um fiber is 26 m

cisco id is --

cisco extended id number is 4

SFP Detail Diagnostics Information (internal calibration)

----------------------------------------------------------------------------

Current Alarms Warnings

Measurement High Low High Low

----------------------------------------------------------------------------

Temperature 44.18 C 75.00 C -5.00 C 70.00 C 0.00 C

Voltage 3.26 V 3.63 V 2.97 V 3.46 V 3.13 V

Current 6.48 mA 10.50 mA 2.50 mA 10.50 mA 2.50 mA

Tx Power -2.62 dBm 1.69 dBm -11.30 dBm -1.30 dBm -7.30 dBm

Rx Power -2.80 dBm -- 1.99 dBm -13.97 dBm -1.00 dBm -9.91 dBm

----------------------------------------------------------------------------

Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning

So dont believe we have any interoperability issue with 5010 and 2232TM here. Before we swap out this fex, can you please open a case with Cisco support or let me know when you are available and I can do a webex to look at this switch to see if there is something more to this problem than what we already know ?

Thanks,

Vinayak

PrimergyQA · ‎07-31-2012

Hi Vinayak,

thanks for your help. In the meantime I have already opened a case with Cisco (622584585).

I have send debug output of the FEX to the assigned TAC engineer. Maybe he has attached it to the case so you can also have a look.

vdsudame · ‎07-31-2012

Ok thanks, i will follow up on this one with the tac engineer.

Vinayak

PrimergyQA · ‎08-01-2012

Hi Vinayak,

no need to investigate further. According to the TAC the FEX needs to be RMAed.

vdsudame · ‎08-01-2012

yes, I had a chat with the TAC engineer yesterday on this. Was aware of this

thanks, Vinayak