IOM Configuration Error

[Justin Holland]

The following condition is being reported on IOM 2 in one of my chassis.  This is the information from the General tab in the gui:

Overall Status:  Fabric Conn Problem

Configuration Error:  fabric port channel member link automatic activation blocked to prevent VIF capacity decrease.

The IOM is in an operable state and there are no actual faults.  Firmware version 2.2(4b).  Below is a bit more detail.  Any input would be appreciated.

Regards,

Justin

FI6248-A-B /chassis/iom # show detail

IOM:

    ID: 2

    Side: Right

    Fabric ID: B

    User Label:

    Overall Status: Fabric Conn Problem

    Oper qualifier: Fabricpc Link Auto Ack Blocked

    Operability: Operable

    Presence: Equipped

    Thermal Status: OK

    Discovery: Online

    Config State: Ok

    Peer Comm Status: Connected

    Product Name: Cisco UCS 2208XP

    PID: UCS-IOM-2208XP

    VID: V03

    Part Number: 73-13196-06

    Vendor: Cisco Systems Inc

    Serial (SN): FCH18397TV7

    HW Revision: 0

    Mfg Date: 2014-10-19T01:00:00.000

    Controller Subject: Iocard

    Fabric Port Aggregation Capability: Port Channel

    Oper Evacuation: Off

    Current Task 1:

    Current Task 2:

[Qiese Dides]

Hi Justin,

Under the Equipment Tab. Click on Equipment then go to Policies under Chassis / Fex Discovery Policy how many links is that set to?

Now click on the IOM with the issue and click on Hybrid Display. How many links are going from the IOM to the Fabric Interconnect. If for example it is set to 2, then the Chassis / Fex Discovery policy should be set to 2. A valid configuration is 1,2,4,8. Any number inbetween that will be invalid.

If all that looks correct then the best option would be to unconfigure all those ports from the IOM to the FI and reconfigure it. Once you do that you need to re-acknowledge the chassis. This should clera all the errors.

Let me know your status after doing all this.

Regards,

Qiese Dides

[Justin Holland]

Qiese, thank you for responding.  I have attached screen shots of the information you requsted.  The Chassis/FEX Discovery Policy is set to 2.  The hybrid display shows 4 links from the IOM to the FI.  We have 3 chassis total and there are no errors on any of the other IOMs.

If I change the discovery policy to 4, what will happen?  This is a production environment and I do not want any type of outage.

Justin

[Qiese Dides]

Justin thanks for the images.

When you change it to 4 links nothing will happen until you re-acknowledge the chassis. Since this is not effected production right now I would suggest you to do this in a maintenance window, change the policy to 4 link and then re-acknolwedge the chassis with the issue. If the problem sitll persists I would bounce the ports (unconfigure / re configure) them. This should elevate these errors.

Regards,

Qiese Dides

[Justin Holland]

Thanks, Qiese.  From my understanding, the Chassis/FEX Discovery Policy describes the minimum number of links from the IOM to the FI.  So, I want that to remain at 2.

The real question is, why is only 1 IOM out of 6 showing the issue?  That indicates a unique problem that should not be corrected by changing a global policy.

Any further suggestions are appreciated!

Justin

[Qiese Dides]

Yeah the best plan would be to try to reacknowledge the chassis having the issue or the IOM. I would not do that during working hours though. If that doesn't work try to unconfigure the ports and reconfigure them, I have seen that as a solution.

[Walter Dey]

I would always set the discovery policy to 1 !

This is the minimum number of links ! If you set it to 2, and a chassis has only one link, this link will not come up.

[Justin Holland]

I am finally following up on this.  During my plan to upgrade UCS from 2.2(4b) to 3.1(2e), I rediscovered that this IO module was still reporting the "Fabric Conn Problem" status.

During my maintenance window, I worked with TAC.  We unconfigured and reconfigured each link to the fabric interconnect.  That did not change the status.  At that point, we re-acknowledged the IO card and it came back into a operable state.  Hope this helps someone.

Justin

[Keny Perez]

I would also be interested in seeing the output of the commands:

#connect nx a|b  <<< We will run the below commands from both FIs so use it accordingly

#show interface fex-fabric

#show fex # detail  <<<  # will be the chassis ID where the affected IOM is

#exit  << after this, go to the first command and try the other Fabric Interconnect

-Kenny

[Justin Holland]

Kenny, thanks for answering.  I have attached the output from my network guy...

Justin

[Keny Perez]

Still missing the "show fex # detail"

-Kenny

[Justin Holland]

FEX detail now attached.

[Keny Perez · Accepted solution]

Runnig 2.2.4 you have the chance to reack a single IOM in the chassis... I would confirm that failover and multipath are properly configured and then try to bounce that IOM.

Be careful if you want to try that during working hours...

-Kenny