UCS mini - FC port remains down when sync is lost

[Charles Morrall]

I am currently deploying a Cisco UCS mini solution with two FI 6324 modules. I have set the first two ports in FC mode and configured the FI modules in FC switch mode. This is a smaller solution with an EMC VNX connected directly to the FI modules. I have followed the guides and I have successfully configured all connectivity.

I have however noticed that if there is a sync or link loss on the FC ports, the ports report an error and the port stays down. This could for example be a firmware upgrade on the storage system which will cause a brief link down status. Another example is to simply replace a cable, causing a short time of link down. I still have redundant paths, meaning I do not get a service disruption as such, but the only solution to bring back full redundancy is to manually disable and then re-enable the FC port in the FI module.

I don't have a lot of experience with Cisco as a whole, but I assume this behaviour could be by design. The FI module detects an excessive number of errors and disables the port. Is this assumption correct? If so, can I configure the port error threshold or the number of attempts to re-establish connection? I would like to automate the operations as much as possible.

[Wes Austin]

Hey Charles,

Not sure what version of Mini you are running, but there are some known defects with UCS Mini and EMC storage:

https://tools.cisco.com/bugsearch/bug/CSCur86104/?reffering_site=dumpcr

https://tools.cisco.com/bugsearch/bug/CSCux05707/?reffering_site=dumpcr

Give these a glance and see if any of the conditions match yours. This is probably what you are experiencing.

Let me know what further questions you have.

--Wes

[Charles Morrall]

Thanks for the links, I have reviewed the suggestions.

However, setting the Fill Word (or Fill Pattern) to IDLE I think would indirectly mean I would have to set the FC link to 4 Gbps, which is not at all acceptable.

The ARB(FF) Fill Word is the recommended (or even only official correct way, not sure) according to the T11 committe when running at 8 Gbps.

It is also the default as far as I could tell in the Cisco UCS mini.

We just now upgraded firmware to I think the latest 3.0(2e), and we still have a problem with the ports coming up, although now the ports stays in an "initializing" state, instead of "loss of sync".

I have set the corresponding VNX ports to 8 Gbps instead of auto.

A bounce of the port will bring up the link, so we're not completely down but I of course would like to avoid this step.

[jochance209]

Any permanent resolution to this issue other than setting to 4gb?