cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
903
Views
0
Helpful
1
Replies

fsm report node status fsm fail

Medlarboc
Level 1
Level 1

Hi Community,

 

An aci leaf appeared as inactive in fabric membership and  unrechable from apic side. Actually, i can ping other leaves, spines and all apics but in leaf side i typed "moquery -c faultInfo" i got this message :

Fult.PNG

 

Can someone help me to resolve this issue ?

Thanks.

1 Reply 1

Sergiu.Daniluk
VIP Alumni
VIP Alumni

Seems like there was a crash on the leaf, and there is an automatic task which will copy core files from leaf to APIC, but since there is no reachability from leaf to APIC, the fault is raised.

Check the following commands to confirm if there is a core file:

show cores
dir /logflash/core

The first command will show the process which crashed and when, second command will show the core file itself.

Make sure you copy it to /data/techsupport:

cp /logflash/core/<file-name> /data/techsupport

Also, this is another very good set of commands which can be simply copy pasted to Leaf switch to create a support file for crashes:

bash -c '
# set this to correct leaf name
leaf="$(hostname)""_data"

# collect data
mkdir /data/techsupport/$leaf
cd /data/techsupport/$leaf
show system reset-reason > show_sys_rr.log
vsh -c "show logging onboard internal reset-reason" > show_logg_onb_internal_rr.log
vsh -c "show logging onboard stack-trace" > show_logg_onb_stack-trace.log
vsh -c "show logging onboard card-boot-history" > show_logg_onb_card-boot-history.log
vsh -c "show processes log details" > show_process_log_detail.log
df -h > df.log
cp /var/log/dmesg ./
cp -rf /mnt/ifc/log/last_run/ ./
mkdir bootflash; cp /bootflash/mem_log* ./bootflash/
mkdir mnt_pss; cp -rf /mnt/pss/* ./mnt_pss/
mkdir mnt_pstore; cp -rf /mnt/pstore/* ./mnt_pstore/

# compress and combine files
cd /data/techsupport
zipfile="$leaf"".tgz"
tar -zcvf ./$zipfile ./$leaf/*
rm -rf ./$leaf/*
rmdir ./$leaf


echo ""
echo " ///// Please collect /data/techsupport/"$zipfile" and upload to SR ///////"
'

Once you have these, you will need to open a case with TAC to investigate the crash, or you can verify the bug search tool or release notes for known bugs/crashes.

 

Stay safe,

Sergiu

 

 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Save 25% on Day-2 Operations Add-On License