04-05-2010 03:56 AM
I have major alarms on WAVE-274 (Edge-WAAS) and WAVE-7341(Core-WAAS).
Alarm Name: Core_dump
Alarm informations: "User space core files were detected"
Is it possible to learn more about this - what caused it, ...ect
Any ideas on how to proceed?
04-06-2010 03:31 PM
Go to subdirectory crash, probably there is a dump file that you need to open a TAC case and send it
04-07-2010 09:21 AM
Hi Claudiols;
Thank you for your reply,
But I wanted to know the exact subdirectory crash, and the dump file name if it's possible.
04-07-2010 09:45 AM
the name of file can be very diferent, but the directory are
/local1/core_dir or /local1/crash
the file would be something like core.java.4.1.3.b55.cnbuild.5940.gz
09-30-2010 06:08 AM
What can I extract of this file core.java.4.1.3.b55.cnbuild.5940.gz ? It´s only information of
crash?
Only the TAC to resolve this problem?
regards
10-11-2010 11:16 AM
Hi,
This is correct; you should open a TAC case and attach not only the core or crash files found in the directories mentioned, but also a sysreport generated in the device where the issue was observed as this investigation encompasses the analysis of several logs and possible correlation with known issues.
To generate a sysreport and store it to an FTP server, use this form of the command: copy sysreport ftp server-ip remote-directory remote-file-name
Best regards,
Fabio Bergamo
Cisco TAC
01-27-2015 10:30 AM
hi! 4 years after I have the same problem, but in to directory core_dir no have files, in to the crash have the files above:
size time of last change name
-------------- ------------------------- -----------
17053068576 Tue Nov 11 11:18:43 2014 vmcore
434 Tue Nov 11 13:53:42 2014 waas_version.txt
Can I delete vmcore? That file don't have extension .gz
Regards
08-01-2012 06:10 PM
when you see alarm "core file detected" most of the time means that an internal process crash, it is hard to tell what exactly happens by looking at the core file itself so you better open a TAC case.
If you still want to learn about it, I can help with the following steps ( this is not documented anywhere so I don't promise covering all the steps, you'd need to be already familiar with reviewing WAAS logs to give it more sense)
scenario sample:
1- 'core files detected alarm shows up and you have no idea what it means.
2- We need to know when exactly the core file was created.
do WAE#dir find the folders "core_dir" and "crash" and open them
with core_dir folder :
WAE#dir core_dir
size time of last change name
-------------- ------------------------- -----------
618747 Sat Apr 19 06:30:14 2012 core.cnbuild.29405.gz
361568 Mon Aug 08 13:20:22 2008 core.cnbuild.11689.gz
so we found two core files over here and it's respective date.
***
delete the core files that are older ( deleting them fixes the alarm issue, why would we need to be alarm for a core file from a long time ago? corefiles are logs that help our developers to understand what process failed so this means whatever issue happen WAAS already recover from it, at least that's what we expect.
< how to delete a core file>
WAE#cd core_dir
WAE#delfile core.cnbuild.11689.gz
give it a minute
WAE# show alarms < the alarm will go again >
***
So once we know the date when the core file was created go and check the *syslog*
If you don't know how to check the syslog:
a- log/save your putty session ( so you can later open the log on a wordpad or any application to simplify research)
b- WAE#type-tail syslog.txt 500 < this number indicates how many lines you want the log to be, this can vary depending on how much log lines your WAE generates or how long ago the core file was created> for example if the core file was first seen 24 hours ago, I guess a syslog of 200 lines should do it but it could be more or less.
c- open the saved session(notepad or whatever..) search for the closer date to the core file date's, and we should see helpful logs that we could easily do research on the internet.
Most of the time if the core file is coming from a problem that requires urgent assistance you'll notice how the alarm that looks critical often repeats itself through the log ( because WAAS will try to recover from the unhealthy state) sometimes you delete a core file*clear the alarm* and it comes back few hours later, that's bad ..so you either help yourself trobleshooting it based on the logs or open a TAC case soon cause it might be causing trouble to your applications)
On the other hand if it was just one time behavior the core file can be easily deleted.
for example:
WAE device stops responding at 8:00 then it comes back online at 8:05, when it came back an internal process got crash, it then generates a core file when the service get's restore (let's say at 8:15) a week later you're trying to figure out what's the core file alarm about.
If you follow the steps above you'll end up reviewing the syslog and realize that before the core file was created the WAAS box got rebooted at 8:05!! end of the research...you might think why was this box rebooting? may be a power outage,hardware issue or someone messing up at the DC?
I am not saying this will give you an exact answer but you'd have something more than a core file not saying much, I have help many customers without asking higher support to decrypt the core file itself.
note:
-There are many logs for many different reasons the syslog I mention on this post is the 'general' log to say it like that... for more detailed logs for specific processes go to WAE#dir errorlog
-Don't forget to vist the WAAS troubleshooting page
-Don't start playing with WAAS on production devices!!!
I know this post is from months ago but I hope it can help other people too, cheers!
Felix
05-19-2017 10:50 PM
Hi,Felix
Thanks for clearing the air ,have one question on how can I download the core damp file in my local directory using CLI Command or GUI just like "sysreport", using FTP Server and or tftp server.
06-06-2017 03:44 AM
Hi,
cd crash (or cd core_dir)
copy disk ftp <ftp-srv IP> <remote dir> <remote filename> <local filename>
...it will prompt you for credentials for the FTP server !
you can use the "." (period) as <remote dir> for the current (working) directory of the ftp server
Best regards
Finn
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide