11-09-2015 12:11 PM - edited 03-01-2019 12:27 PM
Hi,
I have recently started seeing this message within our UCS enviorment.
XXXXXX00-A# show fault
Severity Code Last Transition Time ID Description
--------- -------- ------------------------ -------- -----------
Major F0885 2015-11-01T20:48:03.980 2489810 Fabric Interconnect B inventory is not complete card-inventory,eth-pc-inventory,eth-port-inventory,fc-pc-inventory,fc-port-inventory,mgmt-port-inventory,remote-eth-port-inventory,switch-fru,switch-inventory
I was wondering if anyone has any ideas on how to clear this error or where to start looking for clues as to its origin? Searching Google has not yeilded anything fruitful. I do not see any issues in operations that I can tell as it seems everything is working okay on both FI's. Is there maybe a way to force another inventory check or something like it?
Thanks for any information.
Regards,
Brandon
Solved! Go to Solution.
11-10-2015 09:29 AM
Brandon,
This looks similar to CSCuw36128.
https://tools.cisco.com/bugsearch/bug/CSCuw36128/?reffering_site=dumpcr
11-10-2015 09:29 AM
Brandon,
This looks similar to CSCuw36128.
https://tools.cisco.com/bugsearch/bug/CSCuw36128/?reffering_site=dumpcr
11-13-2015 09:06 AM
Thank You for the reply. Thois does look like the issue so I have a reboot scheduled
01-07-2016 12:10 AM
Hi Brandon,
Quick check, how did you fix the issue for the bug?Based on the workaround
its mention will need to restart UCSM processes to recover statsAG mts buffer space.
How do we restart this process?Appreciate if you can provide me the step.
01-07-2016 05:35 AM
Ajay,
You can restart the FI or you can run a "pmon stop" and "pmon start" from the local-mgmt CLI.
This should be done in a maintenance window. Let me know if you have more questions.
--Wes
01-07-2016 05:18 PM
thanks for the prompt response Wesley. I will perform the recommended action and will let you know the outcome
01-07-2016 07:12 PM
Hi Guys,
I have run "show pmon state" on both FI's but i don't find any errors.
There was no errors found on the "show fault" output.
I have done a cross check on the UCSM and there is no alerts too.
However, SMTP is throwing alerts stating both Fabric Interconnect (A&B) inventory is not complete card-inventory,switch-fru with error code : F0885.
Not too sure if stop and starting pmon state will be worthwhile.
Hi Qiese,
Quick check, you mention on option 2
4) This should drop your SSH session. Reconnect and connect back to local-mgmt. Do a show pmon state to verify everything is running <<<<<<
Lets say I have stopped pmon on FI-B, how long will it take to re-connect back?
Assuming I should be able to reconnect to it remotely, correct me if I am wrong.
01-07-2016 07:19 PM
Hey Ajay,
If you look at the first line in the defect you will notice:
*No statsAG core/restart*
So the pmon state outputs may be clean, however to ultimately clear the fault, you will need to restart pmon.
If you SSH directly to the Fabric interconnects management IP, you can stop and start pmon without having to disconnect the SSH session.
Let me know what questions you have.
--Wes
01-07-2016 07:40 PM
Hi Wesley,
Let me try the recommended step. Will keep you posted.
Cheers
01-08-2016 06:28 AM
As Wes Austin stated that is the correct behavior. keep us updated.
01-14-2016 06:11 PM
Hi Wesley,
The issue is resolve now after stopping and starting the pmon service.
Thanks for your advice.
Regards,
Anand
01-07-2016 05:40 AM
For bug CSCuw36128.
There is currently no target date on a fix.
To clear the errors you can try the following steps in a maintenance window.
1) Reboot secondary Fabric interconnect
2) Verify all paths are up and tested in hosts.
3) ssh to VIP of the cluster.
4) connect local-mgmt
5) "cluster lead a or b" whichever is secondary now. This will cause your session to drop. Reconnect after about 30 seconds.
6) ssh to VIP , connect local-mgmt, show cluster state and verify HA is ready, please make sure.
7) Reboot the other FI.
8. Please give enough time, and verify the status of both FIs.
You can Also do the following if you do not want to reboot the FI's.
1) ssh to FI-A and another ssh session to FI-B
2) connect local-mgmt A on the A side and connect local-mgmt B on the B side
3) pmon stop / pmon start on the secondary side.
4) This should drop your SSH session. Reconnect and connect back to local-mgmt. Do a show pmon state to verify everything is running
5) cluster lead a or b whichever is secondary. From the primary connection
6) pmon stop / pmon start on the old primary
7) This should drop your SSH session. Reconnect and connect back to local-mgmt. Do a show pmon state to verify everything is running
01-07-2016 05:17 PM
thanks for the prompt response Qiese. I will perform the recommended action and will let you know the outcome
04-16-2017 05:28 AM
Was going to follow these instructions (pmon stop/start) this AM on my 2.2(3.b) which has this exact issue, and I get a password prompt when issuing the pmon commands. Never seen that before and can't find ANY doc on it. My admin account has full rights and I ever tried the actual 'admin' login, no change.
Any points? Sitting at the local-mgmt # prompt. Thanks!
04-18-2017 09:54 AM
Lee,
Would it be possible to attach a screenshot here of the prompt?
Or perhaps just paste the output?
I haven't seen any prompts when logging in as the default/local 'admin' user.
Are you trying to kill the processes on the subordinate?
--
Niko
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide