asa 5520 "loosing" code after code has been put in and operating

[Mike Baker]

Sorry to ask this if it has all ready been covered.  We have an asa 5520 running 8.3.2(1) code.  Three times now I have entered code and rules in our asa and had things working, only to have the code "dissapear" and thus things stop working.  We upgraded to 8.3.2(1) back in January of 2011, and have not had this problem until the last month.  I was wondering if there is a bug with 8.3.2(1) code that has decided to show itself for whatever reason now.  We have also had some other things relating to the VPN that were "working" and at some point just stopped working.  We do have a second asa 5520 that is the failover/standby.  We also have two 6509 with firewall services modules, one primary and the other standby.  Just wondering how to troubleshoot something like this.  I have putty logs of me putting the code in and doing a write mem saving the changes, yet on three occations those things stopped working, and I had to put the code in again.

**update** as I was typing this, we realised there was a problem with the two ASA's.  For some reason, failover had stopped working, and both ASA's were trying to be the primary and causing issues.  After several reboots, we wound up turning failover back on  on the second ASA, and things seem to be normal now.  No idea what would have caused the failover to break.  Not sure how long this had been going on, it may have had to do with my code seeming to dissapear?

[Marius Gunnerud]

Does this issue come when the device is rebooted.  (stupid question) Are you sure you have saved the configuration after adding it?

Could you post the full output of show version please.  I am wondering if the registery configuration has changed.

To change the configuration registry back to the default value you can issue one of the following commands:

config-register 0×01

or

no config-register

[Mike Baker]

Here is the output of the show ver.  I removed the serial number.

ACH-2nd-EXT-ASA01#sh ver

Cisco Adaptive Security Appliance Software Version 8.3(2)1
Device Manager Version 6.4(7)

Compiled on Wed 04-Aug-10 21:41 by builders
System image file is "disk0:/asa832-1-k8.bin"
Config file at boot was "startup-config"

ACH-2nd-EXT-ASA01 up 4 days 22 hours
failover cluster up 4 days 22 hours

Hardware:   ASA5520, 2048 MB RAM, CPU Pentium 4 Celeron 2000 MHz
Internal ATA Compact Flash, 256MB
BIOS Flash M50FW080 @ 0xfff00000, 1024KB

Encryption hardware device : Cisco ASA-55x0 on-board accelerator (revision 0x0)
                             Boot microcode   : CN1000-MC-BOOT-2.00
                             SSL/IKE microcode: CNLite-MC-SSLm-PLUS-2.03
                             IPSec microcode  : CNlite-MC-IPSECm-MAIN-2.06
0: Ext: GigabitEthernet0/0  : address is 001d.a298.c41c, irq 9
1: Ext: GigabitEthernet0/1  : address is 001d.a298.c41d, irq 9
2: Ext: GigabitEthernet0/2  : address is 001d.a298.c41e, irq 9
3: Ext: GigabitEthernet0/3  : address is 001d.a298.c41f, irq 9
4: Ext: Management0/0       : address is 001d.a298.c420, irq 11
5: Int: Internal-Data0/0    : address is 0000.0001.0002, irq 11
6: Int: Internal-Control0/0 : address is 0000.0001.0001, irq 5

Licensed features for this platform:
Maximum Physical Interfaces    : Unlimited      perpetual
Maximum VLANs                  : 150            perpetual
Inside Hosts                   : Unlimited      perpetual
Failover                       : Active/Active  perpetual
VPN-DES                        : Enabled        perpetual
VPN-3DES-AES                   : Enabled        perpetual
Security Contexts              : 2              perpetual
GTP/GPRS                       : Disabled       perpetual
SSL VPN Peers                  : 10             perpetual
Total VPN Peers                : 750            perpetual
Shared License                 : Disabled       perpetual
AnyConnect for Mobile          : Enabled        perpetual
AnyConnect for Cisco VPN Phone : Disabled       perpetual
AnyConnect Essentials          : Enabled        perpetual
Advanced Endpoint Assessment   : Disabled       perpetual
UC Phone Proxy Sessions        : 2              perpetual
Total UC Proxy Sessions        : 2              perpetual
Botnet Traffic Filter          : Disabled       perpetual
Intercompany Media Engine      : Disabled       perpetual

This platform has an ASA 5520 VPN Plus license.

Failover cluster licensed features for this platform:
Maximum Physical Interfaces    : Unlimited      perpetual
Maximum VLANs                  : 150            perpetual
Inside Hosts                   : Unlimited      perpetual
Failover                       : Active/Active  perpetual
VPN-DES                        : Enabled        perpetual
VPN-3DES-AES                   : Enabled        perpetual
Security Contexts              : 4              perpetual
GTP/GPRS                       : Disabled       perpetual
SSL VPN Peers                  : 20             perpetual
Total VPN Peers                : 750            perpetual
Shared License                 : Disabled       perpetual
AnyConnect for Mobile          : Enabled        perpetual
AnyConnect for Cisco VPN Phone : Disabled       perpetual
AnyConnect Essentials          : Enabled        perpetual
Advanced Endpoint Assessment   : Disabled       perpetual
UC Phone Proxy Sessions        : 4              perpetual
Total UC Proxy Sessions        : 4              perpetual
Botnet Traffic Filter          : Disabled       perpetual
Intercompany Media Engine      : Disabled       perpetual

This platform has an ASA 5520 VPN Plus license.

Serial Number: xxxxxxxxxxx
Running Permanent Activation Key: 0xf730cf7a 0x0449cabf 0xc922e5d4 0xc7bc5cb0 0x851ed6bb
Configuration register is 0x1
Configuration has not been modified since last system restart.
ACH-2nd-EXT-ASA01#

[Mike Baker]

And yes, I was saving my changes.  I keep putty logs when I log into the switches and firewalls, and the logs show I did the wr mem.

We have been told we need to upgrade our ASA's to a version in 8.4 something, or higher.  I guess that completely changes how nat is done.  We had originally tried to upgrade from 8.2 to 8.4 in January of 2011 (for license for iPads and iPhones to work), and it was a disaster!  We went back to 8.2, then upgraded to 8.3.2.  That had some kind of bug in it, so TAC had us go to 8.3.2(1), our current version.  We've been told the "no failover" getting changed on the standby is a bug and we need to upgrade.

Since the last firewall upgrade was a disaster, I am leary of this upgrade too, especially with nat changing.  We are working with cisco partner to plan the upgrade, will have to be done on a non-payroll weekend for us.

[Marius Gunnerud]

This does sound like a bug, though I have not been able to find any documentation about it.  I would follow TAC's suggestion and upgrade to a newer version.

What happened during your last upgrade? 

Did you make sure you met the memory requirements? 

Did you try to upgrade directly from 8.2 to 8.4? If you did, then this is where your upgrade issue may have come.  You need to upgrade to 8.3 before going to 8.4.

[Mike Baker]

We did upgrade both asa's to 2GB prior to the upgrade.  We actually found out at that time that the ssm-20 card was damaged on the primary.  When we opened the asa up to do the memory, the heat sync had come off the processor of the ssm-20 card.  So we had to wait until that was RMA'd and replaced.  We did go from 8.2 to 8.3 then 8.4.  After the upgrade we lost access to the DMZ.  Three TAC engineers worked on it, and the very first one said "why did you upgrade to 8.4?  it is too new and full of bugs!".  So we backed out to 8.2 again, then tried 8.3.2.  We had one outside client (MasterCard, an important one) who could not transfer files.  We tried for a week to get it to work, then TAC said the version of 8.3.2 had a bug, so they had us go to a "special" release 8.3.2(1).  That is what we are on now.  Lots of things have happened with 8.3.2(1).  In July last year, the DAP assessment on the VPN stopped working.  When TAC looked at it, they said what we were doing would not work (VPN clients would be checked to make sure they had symantec, and that virus definitions were within 2 weeks).  When I told them it had been working, they just said they were not sure how, it never should have worked.  And now the no failover is acting up.

I just hope what we go to is stable, and we can get all the nats to work right.

[Marius Gunnerud]

I find it strange that you have experienced so many issues.  I have not experienced any problems with 8.4 versions, but then again I did not have DAP configured on these firewalls.  But everything else worked fine.

NAT should be the least of your worries.  It is not very difficult to configure once you wrap your head around using object groups in the NAT statements.

Good luck with the upgrade.

[Mike Baker]

Thank you.  I am trying to keep a positive attitude.  May I ask another question?  We used to have checkpoint firewall before we went to cisco.  Checkpoint seemed very easy to monitor what traffic was going through, especially if we were setting up a new server, getting the rules right and seeing what ports were being used.  I struggle with the asa to do this.  Even in asdm, if a create a rule open for IP, and try to monitor the rule in asdm, nothing shows up.  When you first open the show logging window, there is a hex value in the filter field.  If you clear it out, nothing happens.  I do not know what to put in that field.  And if you click on show all, you get ALL the traffic going through the firewall, which is not what I am trying to do.  I have had very limmited success in the cli with the capture command, that is kinda hard to figure out too.  Anyway, there are many times we are trying to troubleshoot a connection to a server, and I feel like an idiot since I can't seem to get the asa to tell me what is going on with specific rules.  I dislike the asdm too, especially the vpn area.  I will go in there an look at things, just look, and then go to exit, and it says I need to save my changes.  I didn't change anything, I was just looking!

Anyway, thank you for letting me vent a little bit.  Have a great day.

Mike Baker

[Marius Gunnerud]

I work with both Cisco and CheckPoint products, but I prefer Cisco.  I suppose it could be because Cisco is where I started my network career.  If you enter the log keyword after an ACL entry that is assigned to an interface you will be able to see whenever traffic matches that ACL in the log.

Another way to view this traffic is the packet capture feature.  This can be run from the CLI and the ASDM, whichever you like best.

In ASDM it is located under the Wizard tab.

This link explains how to use the packet capture in both ASDM and CLI

http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a0080a9edd6.shtml

[Mike Baker]

Thank you Marius.  I do have the log keyword on the acl's.  I will check out the link you have provided.

Have a great day.

Mike Baker

[Marius Gunnerud]

Thanks,  You too have a good day and a good weekend.

Please remember to rate any posts that you found useful.

[Mike Baker]

Do you know how to see traffic from the show logging screen?  This is what I was trying to do, show the traffic on a specific rule.  I have attached screenshot.

[Marius Gunnerud]

If logging is enabled and you do not see the hit count increasing on the ACL entry, that means it is not being matched.  The only other way then to see what it is actually matching is either run packet tracer, packet capture or click on the Monitor tab at the top of the ASDM > logging > and View realtime logging (I think that is the path ). and then filter on the IP.

[Mike Baker]

Thanks you.  Is there syntax for filtering on the IP?

[Marius Gunnerud]

In the CLI you could run the command show logging | include 192.168.1.20 (for example.)

in the ASDM there is a filter option at the top of the Monitoring page.