Solved: N3K-C3548P-10GX keep rebooted

todd.hsieh · ‎03-14-2023

Hi,

My customer N3K-C3548P-10GX keep rebooted and caused a service disruption about 20 minutes.

We removed this 3548 from the production network. When we did some investigation, we found a successive restart with death reason.

May I know if this is about to hardware or software issue ?

Thank you!

Service: pfma
Description: N5k platform manager Daemon
Executable: /isan/bin/pfm

Started at Wed Nov 9 15:52:47 2016 (608989 us)
Stopped at Thu Mar 9 13:51:13 2023 (763768 us)
Uptime: 2310 days 21 hours 58 minutes 26 seconds

Start type: SRV_OPTION_RESTART_STATELESS (23)
Death reason: SYSMGR_DEATH_REASON_FAILURE_SIGNAL (2)

Last heartbeat 0.79 secs ago

-------------------------------

Service: pfma
Description: N5k platform manager Daemon
Executable: /isan/bin/pfm

Started at Thu Mar 9 14:01:56 2023 (512410 us)
Stopped at Thu Mar 9 14:03:36 2023 (752789 us)
Uptime: 1 minutes 40 seconds

Start type: SRV_OPTION_RESTART_STATELESS (23)
Death reason: SYSMGR_DEATH_REASON_FAILURE_SIGNAL (2)
Last heartbeat 0.93 secs ago

Lebowski1991 · ‎03-22-2023

Hi,

First, let me say that the only appropriate way to get a definitive answer to this issue should be through Cisco TAC as further troubleshooting would require analyzing core dumps and show tech-support which could contain customer sensitive data.

I believe PFM would stand for Power Fabric module and as such it should be related to environment driven event.
I don't have C3500 in my deployment to check however you could try with the following commands to get a bit more of an understanding of the issue:
show environment power detail
Show environment temp detail
show environment fan detail
Show logging nvram
show logging onboard exeption-log
show logging onboard environmental-history
show logging onboard kernel-trace

If however you insist on troubleshooting this with the community rather than with TAC, do share the said logs ( filter out any customer sensitive data) and I can give it a go.

Hopefully this helps!

View solution in original post

Lebowski1991 · ‎03-22-2023

Hi,

First, let me say that the only appropriate way to get a definitive answer to this issue should be through Cisco TAC as further troubleshooting would require analyzing core dumps and show tech-support which could contain customer sensitive data.

I believe PFM would stand for Power Fabric module and as such it should be related to environment driven event.
I don't have C3500 in my deployment to check however you could try with the following commands to get a bit more of an understanding of the issue:
show environment power detail
Show environment temp detail
show environment fan detail
Show logging nvram
show logging onboard exeption-log
show logging onboard environmental-history
show logging onboard kernel-trace

If however you insist on troubleshooting this with the community rather than with TAC, do share the said logs ( filter out any customer sensitive data) and I can give it a go.

Hopefully this helps!

todd.hsieh · ‎03-22-2023

Hi Lebowski1991

Thank you. This is helpful information. I keep these command in memo.

We have sent this N3K back to our supplier who will do the RMA.