ASK THE EXPERTS - TROUBLESHOOTING ASA, PIX AND FWSM

[ciscomoderator]

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to learn how to address and troubleshoot common problems with Adaptive Security Appliances, Private Internet Exchange and Firewall Service Modules with Kureli Sankar.  Kureli is an engineer supporting Cisco's firewall team in Research Triangle Park, North Carolina. Her team supports the Cisco Adaptive Security Appliance, Firewall Services Module, Cisco Security Manager, the Content Security and Control module, and the Zone Based Firewall module in Cisco IOS Software.

Remember to use the rating system to let Kureli know if you have received an adequate response.

Kureli might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through January 14, 2011. Visit this forum often to view responses to your questions and the questions of other community members.

[edoig.admin]

My group does log analyis on a large organizations PIX/ASA logs. Currently we recieve a dump of the daily logs and put them into our system for our analysis. I'm working on a script to process these logs into a CSV or other readable format for investigators. Is there a standard script or tool that can be used to process these logs? We have log analysis tools, but we want to convert them to CSV or other format so that they can be manipulated more easily by non-techies.

Thanks!

[Kureli Sankar]

I believe kiwi syslog has an option to export the logs in .csv format.

http://www.kiwisyslog.com/help/syslogwebaccess/index.html?export_to_csv.htm

I remember responding to your post a while ago: https://supportforums.cisco.com/message/3251283

Let me see if I can find out a way to convert these that you receive from linux to CSV format.

-Kureli

[edoig.admin]

we are getting the raw dump from a syslog server and just pushing that to a linux share... no specific syslog app... I want to change it to a CSV file that has fields for dst address, dst ip, src pt, src ip, etc... but since the logs are specific on the type of message, can't do a simple script... my question was, are there any other solution for converting them to csv or other easily readable format...

if i did the kiwi method, it'd parse first few lines (date, message type) then probably dump rest into on field.

[Kureli Sankar]

Tim,

Pls. let me know if there are any specific syslog messages in particular that you are interested in seeing the source interface, source ip, source port, dest interface, dest ip, dest port.

Since all these messages have unique text in them it will be hard for one particular script to spit out the format as a .csv format.

Are you interested only in 302014 and 302015 and 302016 built and teardown messages?

If so you can use shell script to do what you like to do. Let me know and I shall send a sample.

-KS

[edoig.admin]

The things we care most about are builds and teardowns, but for our purpose, we also care about deny's, icmp's, etc...

so i did a count of each message type for a single day and got what is pasted below... my thinking was to create a script that captured most of the data in the fields not italicized, then throw the data from the others into another field (or wear appropriate)... (if you want to talk offline, please message me.)

Count      Log Type      LogFormat
18132395     %ASA-6-302015: Built {inbound|outbound} UDP connection number for interface_name:real_address/real_port (mapped_address/mapped_port) to interface_name:real_address/real_port (mapped_address/mapped_port) [(user)]
18123239     %ASA-6-302016: Teardown UDP connection number for interface:real-address/real-port to interface:real-address/real-port duration hh:mm:ss bytes bytes [(user)]
9098811      %ASA-6-302014: Teardown TCP connection id for interface:real-address/real-port to interface:real-address/real-port duration hh:mm:ss bytes bytes [reason] [(user)]     
9097915      %ASA-6-302013: Built {inbound|outbound} TCP connection_id for interface:real-address/real-port (mapped-address/mapped-port) to interface:real-address/real-port (mapped-address/mapped-port) [(user)]     
4017138      %ASA-4-106023: Deny protocol src [interface_name:source_address/source_port] dst interface_name:dest_address/dest_port [type {string}, code {code}] by access_group acl_ID     
2646225      %ASA-6-305012: Teardown {dynamic|static} {TCP|UDP|ICMP} translation from interface_name [(acl-name)]:real_address/{real_port|real_ICMP_ID}to interface_name:mapped_address/{mapped_port|mapped_ICMP_ID} duration time     
2645583      %ASA-6-305011: Built {dynamic|static} {TCP|UDP|ICMP} translation from interface_name:real_address/real_port to interface_name:mapped_address/mapped_port     
768037      %ASA-6-302020: Built {in | out}bound ICMP connection for faddr {faddr | icmp_seq_num} gaddr {gaddr | cmp_type} laddr laddr     
767977      %ASA-6-302021: Teardown ICMP connection for faddr {faddr | icmp_seq_num} gaddr {gaddr | cmp_type} laddr laddr     
468749      %ASA-6-106015: Deny TCP (no connection) from IP_address/port to IP_address/port flags tcp_flags on interface interface_name.     
141597      %ASA-3-305006: {outbound static|identity|portmap|regular) translation creation failed for protocol src interface_name:source_address/source_port dst interface_name:dest_address/dest_port     
9917      %ASA-4-733100: Object drop rate rate_ID exceeded. Current burst rate is rate_val per second, max configured rate is rate_val; Current average rate is rate_val per second, max configured rate is rate_val; Cumulative total count is total_cnt
6095      %ASA-3-305005: No translation group found for protocol src interface_name: source_address/source_port dst interface_name: dest_address/dest_port     
1267      %ASA-6-106100: access-list acl_ID {permitted | denied | est-allowed} protocol interface_name/source_address(source_port) - interface_name/dest_address(dest_port) hit-cnt number ({first hit | number-second interval}) hash codes     
219      %ASA-6-314001: Pre-allocated RTSP UDP backconnection for src_intf:src_IP to dst_intf:dst_IP/dst_port.     
164      %ASA-5-111008: User user executed the command string     
143      %ASA-6-302010: connections in use, connections most used     
138      %ASA-4-313005: No matching connection for ICMP error message: icmp_msg_info on interface_name interface. Original IP payload: embedded_frame_info icmp_msg_info = icmp src src_interface_name:src_address dst dest_interface_name:dest_address (type icmp_type, code icmp_code) embedded_frame_info = prot src source_address/source_port dst dest_address/dest_port     
95      %ASA-6-110002: Failed to locate egress interface for protocol from src interface:src IP/src port to dest IP/dest port     
90      %ASA-6-602303: IPSEC: An direction tunnel_type SA (SPI=spi) between local_IP and remote_IP (username) has been created.     
90      %ASA-6-602304: IPSEC: An direction tunnel_type SA (SPI=spi) between local_IP and remote_IP (username) has been deleted.     
88      %ASA-6-303002: FTP connection from src_ifc:src_ip/src_port to dst_ifc:dst_ip/dst_port, user username action file filename     
59      %ASA-4-419002: Received duplicate TCP SYN from in_interface:src_address/src_port to out_interface:dest_address/dest_port with different initial sequence number.     
48      %ASA-5-713041: IKE Initiator: new or rekey Phase 1 or 2, Intf interface_number, IKE Peer IP_address local Proxy Address IP_address, remote Proxy Address IP_address, Crypto map (crypto map tag)     
45      %ASA-5-713049: Security negotiation complete for tunnel_type type (group_name) Initiator/Responder, Inbound SPI = SPI, Outbound SPI = SPI     
45      %ASA-3-713020: No Group found by matching OU(s) from ID payload: OU_value     
28      %ASA-3-313001: Denied ICMP type=number, code=code from IP_address on interface interface_name     
27      %ASA-6-611101: User authentication succeeded: Uname: user     
26      %ASA-1-709003: (Primary) Beginning configuration replication: Sending to mate.     
18      %ASA-6-113004: AAA user aaa_type Successful: server = server_IP_address, User = user     
18      %ASA-6-113008: AAA transaction status ACCEPT: user = user     
11      %ASA-6-315011: SSH session from IP_address on interface interface_name for user user disconnected by SSH server, reason: reason     
9      %ASA-5-502103: User priv level changed: Uname: user From: privilege_level To: privilege_level     
9      %ASA-5-611103: User logged out: Uname: user     
9      %ASA-6-605005: Login permitted from source-address/source-port to interface:destination/service for user “username”     
6      %ASA-5-713050: Connection terminated for peer IP_address. Reason: termination reason Remote Proxy IP_address, Local Proxy IP_address     
5      %ASA-4-313004:Denied ICMP type=icmp_type, from source_address on interface interface_name to dest_address:no matching session     
5      %ASA-5-111007: Begin configuration: IP_address reading from device.
4      %ASA-5-111001: Begin configuration: IP_address writing to device     
4      %ASA-5-111004: IP_address end configuration: {FAILED|OK}     
4      %ASA-5-111005: IP_address end configuration: OK     
4      %ASA-6-611102: User authentication failed: Uname: user     
3      %ASA-4-713903:descriptive_event_string     
3      %ASA-5-713119: PHASE 1 COMPLETED     
3      %ASA-6-713172: Automatic NAT Detection Status: Remote end is|is not behind a NAT device This end is|is not behind a NAT device     
2      %ASA-6-113005: AAA user authentication Rejected: reason = string: server = server_IP_address, User = user     
1      %ASA-6-110003: Routing failed to locate next-hop for protocol from src interface:src IP/src port to dest interface:dest IP/dest port

[edoig.admin]

any thoughts?

[Kureli Sankar]

I can't think of a way to use a script to separate all the fields that you are looking to separate specially all these syslogs have unique messages.

If you can grep for certain syslog messages and then try to separate the 4th column to get all the individual port, ip etc out, it might be easier.  Seems like this might be a lot of work. I am attaching the script that we came up with. Give it a shot.

-Kureli

[gdspa]

We have some problems with a couple of cisco ASA 5510 with stateful failover.

This is the situation.

This host: Primary - Standby Ready
        Active time: 10790719 (sec)
        slot 0: ASA5510 hw/sw rev (1.1/8.2(1)11) status (Up Sys)
          Interface inside (10.11.5.101): Normal
          Interface management (10.12.6.247): Normal
          Interface dmz (10.249.5.2): Normal
          Interface outside (x.x.x.x): Normal
          Interface CircoloAziendale (172.30.1.2): Normal
          Interface Sindacato (172.30.2.3): Normal
          Interface vodafone (10.49.5.2): Normal
          Interface videoconferenza_SalaConsiglio (10.18.5.2): Normal
          Interface GD_guests (10.50.1.52): Normal
        slot 1: empty
Other host: Secondary - Active
        Active time: 6766056 (sec)
        slot 0: ASA5510 hw/sw rev (2.0/8.2(1)11) status (Up Sys)
          Interface inside (10.11.5.100): Normal
          Interface management (10.12.6.246): Normal
          Interface dmz (10.249.5.1): Normal
          Interface outside (x.x.x.y): Normal
          Interface CircoloAziendale (172.30.1.1): Normal
          Interface Sindacato (172.30.2.1): Normal
          Interface vodafone (10.49.5.1): Normal
          Interface videoconferenza_SalaConsiglio (10.18.5.1): Normal
          Interface GD_guests (10.50.1.51): Normal
        slot 1: empty

When I use the primary as active, I have a lot of overruns on the inside interface, withous any other errors.

Now secondary ASA is the active one and we don't have any overrun.

For both firewalls, inside interface speed is 1000Mbps.

Firewalls are different on hardware version, can it cause problems?

[Kureli Sankar]

gdspa,

This should not cause this problem.  Does the switch port show any errors? When the secondary unit is active, I suggest to move the Primary units inside interface to another port on the switch and see if this goes away. Compare the switch port config between these two inside interfaces and make sure they are configured exactly the same way. For 1 GB usually the recommendation is to set it to auto auto on both ends and not to specify the speed.

-Kureli

[gdspa]

I forgot to write that on the switch I don't have any error on the port of the primary firewall.

Speed is configured on auto.

From Cisco docs I read, overruns are caused by too much traffic and not from cable problems. Do you confirm?

[Kureli Sankar]

That is correct.

An Overrun is when an incoming (ingress) packet hits the firewall's NIC, and the rx ring is full.  This is generally caused by elevated CPU, or cpu hogs or infected hosts.

An Underrun is when part of the packet is in the tx ring, and the driver starts transmitting it on the wire, but is unable to get the remaining part of the packet by the time it has finished transmitting the first part.

What doesn't add up is that this doesn't seem to be a problem when the secondary unit is active. That is the reason I suggested to look at the swtichport config for both ports to see if they are any diff.

-Kureli

[harinirina]

Hello,

I would like to ask if it is possible to configure AIP-SSM for redundancy.

We have 2 ASA with AIP-SSM each. the ASA is configured for failover. What should be the configuration of the AIP-SSM so that it can work for failover.

[Kureli Sankar]

Harinirina,

As far as the SSM module is concerned there is no particular failover config for that.  If the module in one ASA fails then that ASA is considered less healthy and it will failover to the other unit and the SSM module in the other unit will do all the scanning per the configuration.

You can read about the failover guidelines here: http://www.cisco.com/en/US/docs/security/asa/asa82/configuration/guide/csc.html#wp1107307

Failover Guidelines

Does not support sessions in Stateful Failover. The CSC SSM does not  maintain connection information, and therefore cannot provide the  failover unit with the required information. The connections that a CSC  SSM is scanning are dropped when the adaptive security appliance in  which the CSC SSM is installed fails. When the standby adaptive security  appliance becomes active, it forwards the scanned traffic to the CSC  SSM and the connections are reset.

-Kureli

[jvardhan29]

Hi

Please explain the input and output in the ASA for QOS policing . i have never been able to understand this and usually end up in making this work by trial and error .i have gone thru the below cisco guide but thought that this platform is good to get answer from experts

http://www.cisco.com/en/US/docs/security/asa/asa72/configuration/guide/mpc.html#wp1116522

also if u can tell what IP Address to use (private or public) while applying an ACL to a class-map (using ASA 8.2 ).

take an example if i want to police the user for a specific website downloads (traffic going from inside to outside) where and how police will be applied .i.e i just want to restrict the downloads but not the HTTP site .now the confusion is that download is also part of browsing that HTTP page,  so how will ASA determine what to police ?