FEX 2232pp connected to dual 5k. No connectivity.

[ty.masse]

I have a FEX connected to dual 5Ks. E1 & 2 connected to one switch, and E3 connected to the other configured in a VPC. However all the links on the FEX shows solid yellow status. They're all copper.

I decided to put a fiber SFP in the FEX to see if I see the laser coming out of it. Result: I don't see any laser transmitting from the SFP and all ports are in solid Yellow status.
However when I unplug the uplink going to the second switch, I could then see the laser light coming out of the SFP, and all the port LEDs turned off. I then plugged my laptop to the FEX and could get to the internet.
When I plugged the fiber back in, all port came back yellow and the laser on the sfp turned off and I loose connectivity.

Something it doesn?t like, but I don?t know what. Requesting help from the user community.

Thanks


Sent from Cisco Technical Support iPad App

[Marvin Rhoads]

Has it ever worked?

You have a FEX with three connections - 2 to one 5k and 1 to the other? That's not a standard supported configuration - the connections should be the same to each 5k, as should the 5k switchports used to uplink to the FEX.

Is this intended to be an active-active deisgn? Please look at the VPC Design guide and try to match your setup with one of the recommded designs.

[ty.masse]

That's puzzling. What if I lost one of my links? I shouldn't loose connectivity. I should stay up even if I loose a whole switch. By the way I have them set up in a VPC. This is a new install so it's not in production yet. What do you think?

Sent from Cisco Technical Support iPad App

[Reza Sharifi]

Have you configured VPC peer link and VPC keep alive between the 5ks?

What is the output of "sh vpc" and "sh fex"

Can you post sh run?

HTH

[Marvin Rhoads]

Re your question about losing a switch or link, yes the FEXes shoud remainup.

However, for them to come up properly in the first place you need to have a correct configuration in place or else NX-OS will detect the issue and fail to bring up the FEX in a non-workble configuration.

In addition to Reza's suggestions, also look at "show vpc consistency-parameters global"

[ty.masse]

In RE: to Marvin, I added another connection as suggested with no changes. 

In RE: to Reza attached are the show commands.

I also have a TAC case open and they're recommending that I upgrade the NX-OS.

Let me know if you guys find out anything, and I'll let you know if upgrading resolves the issue.

thanks,

[Marvin Rhoads · Accepted solution]

The config does look correct at quick glance.

I do see you are running NX-OS 5.1.(3)N1(1) - that was the first release to support the enhanced VPC for dual-homed FEXes. So it doesn't surprise me they are advising you upgrade.

We are mostly advising customers currently on 5k's to run NX-OS 5.2(1)N1(5) as of this time (October 2013).

[ty.masse]

Thanks for your help.  upgrading to NX-OS 5.2(1)N1(5) resolved the issue.

[Marvin Rhoads]

Good news.

Thanks for the update and the rating.