ASA crash ??

[secureIT]

Hi Folks,

Does any ASA firewall crashes with crashinfo generated due to HARDWARE problem ??

According to my knowledge, the crash occurs if there is a bug in the running version of image.

So far I have seen a firewall in standby crashes due to dispatch in thread name of crashinfo file. This firewall was initially primay-active, then rebooted automatically. Later on primary-standby this firewall rebooted. Upon upgrading from 8.4.3 to 8.4.6 again we noticed that the same firewall in standby mode got rebooted with dispatch process in crashinfo ?

I know that only Cisco can interpret the crashinfo., But to some level I have done the analysis and found like CSCtz46866, CSCuf26262, CSCug60235

Secondly, from inside to dmz, do we require any kind of identity nat to access any resource from DMZ ?

regards

Rajesh

[Jouni Forss]

Hi,

I can't really comment on the crashing of the ASA. I have always opened a TAC case in these situation since probably the only thing I could do is try another software level or check the latest changes done to the device before this started occuring.

In the new software you should not need any NAT configurations between your local networks interface on the ASA. As long as the routing is fine from end to end and the ACLs allow the traffic then the networks should be able to communicate with their original IP addresses.

If for some reason there is some type of wide NAT rule configured between these interfaces at the moment then you might need additions/changes in the NAT configuration to enable connectivity in both directions (with regards to connection initiation)

You can use the "packet-tracer" command easily to confirm if the ASA configuration support connectivity between certain hosts between certain ASA interfaces.

- Jouni

[secureIT]

OK, i think ACL is also not required for inside networks to communicate with DMZ network, because by default higher to lower traffic flow is allowed...

[Marius Gunnerud · Accepted solution]

Just to add to what Jouni has mentioned.  If you are running an earlier version of ASA and do not want to configure NAT to enable access between the inside and DMZ, you can issue the command no nat-control and this will remove the requirement for a NAT statement to allow access.

If no ACLs are configured on an interface then the security level is what counts.  But if there are ACLs configure the security level has no meaning and only the configured ACL rules count (as well as the implicit deny at the end of the ACL).

[secureIT]

Hi TAC TEAM,

Does the crashinfo collected time indicate the time when the firewall is rebooted ??

For example :

1--As per the crashinfo, the crash file collected date and time is 22-aug-2013 at 8 AM

2--Up time of the ASA firewall is 1 hrs

3--show clock output is 22-aug-2013 10 AM

So, when would have this firewall rebooted ? I was under the impression that the reboot time would be immly after 8AM. If this is wrong, then the firewall was undergoing a crash state for 1hr and then finally it slept ? May be I have to think in that angle as well.

There are some cpu hogs generated between 22-aug-2013 at 8 AM AND 22-aug-2013 at 9.15 AM

Which parameter I have to see in show cpu hog output ?

Is there any way to interpret the crashinfo ?? I know that this is only with TAC Team. But still atleast one step ahead ??

regards

Rajesh

[Marius Gunnerud]

So, when would have this firewall rebooted ? I was under the impression  that the reboot time would be immly after 8AM. If this is wrong, then  the firewall was undergoing a crash state for 1hr and then finally it  slept ? May be I have to think in that angle as well. 

The core dump is written to flash just before the reboot.

Which parameter I have to see in show cpu hog output ?

You can use the following commands to see what is taking up the most CPU

show processes cpu-usage sorted non-zero

show processes cpu-hog

You can use the commands show coredump log and show coredump filesystem to view the contents of the coredump file.  But it is recommended to contact Cisco TAC support to interpret and debug.

[secureIT]

OK. What about the crashinfo generated time and show version up time comparison ??

[Marius Gunnerud]

I don't have a good explanation for this.  It could be that crash happened at 8 AM and then there was something that cause the ASA to reboot again an hour later but it was not a crash so a dump was not created.

[secureIT]

OK. So can I presume that the crash file generated time is the same time when the ASA got rebooted (may be 5 2 seconds after the crash) ??

In a crashinfo, I generally look at the thread name & bug tool kit. Is there anything else i need to check ?

Is there any need to check the show proces cpu hog, in an automatic reboot of any cisco asa firewall ?

Which field i have verify from a cpu hog output ? NUMHOG: 1 OR MAXHOG: 12 OR LASTHOG: 12

regards

Rajesh

[Marius Gunnerud]

The crashdump should have been created just before the reboot after the crash.

I have always contacted Cisco TAC when interpreting the coredump file.  They have access to tools that analyzes the info contained within.

I normally look at the MAXHOG and LASTHOG at.

[secureIT]

Thanks for the reply. I have noted your answer.

Does the maxhog and lasthog mean the no. of time the cpu was read/polled by this process and it caused the cpu to go high ?

Does this always indicate a high cpu in the system or will indicate only a momentory spike ?

What is this NUMHOG & PROC_PC_TOTAL?

Process:      snp flow bulk sync, PROC_PC_TOTAL: 12, MAXHOG: 16, LASTHOG: 16

LASTHOG At:   11:27:08 PHST Aug 8 2011

PC:           86badfe (suspend)

Process:      vpnfol_sync/Bulk Sync - Import , NUMHOG: 23, MAXHOG: 6, LASTHOG: 6

LASTHOG At:   11:27:17 PHST Aug 8 2011

PC:           80635a5 (suspend)

Traceback:    80635a5  8d9ff96  8062413

regards

Rajesh

[Marius Gunnerud · Accepted solution]

PU hog events are also recorded by the system. The output of the       show proc cpu-hog command displays these       fields:

• Process - the name of the process that hogged the           CPU.

• PROC_PC_TOTAL - the total number of times that this process hogged           the CPU.

• MAXHOG - the longest CPU hog time observed for that process, in           milliseconds.

• LASTHOG - the amount of time the last hog held the CPU, in           milliseconds.

• LASTHOG At - the time the CPU hog last occurred.

• PC - the program counter value of the process when the CPU hog           occurred. (Information for the Cisco Technical Assistance Center           (TAC))

• Call stack - the call stack of the process when the CPU hog occurred.           (Information for the Cisco TAC)

These links can provide some more info on the show process cpu-hog and general troubleshooting of CPU usage.

http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a0080bfa10a.shtml

http://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_tech_note09186a008009491c.shtml

hope this helps

[secureIT]

Thank U