topic At the time of issue did you in Network Security

Cisco ASA 5510 reboot necessary every week

sprintership-il — Tue, 12 Mar 2019 06:33:28 GMT

From time to time , almost every week we have to reboot ASA firewall. Before I manually hit the button, I noticed there is no DNS communication at time we loose internet connection. Logging to CLI ASA can't ping anything to outside word from outside interface. Have already replaced hardware, CISCO TAC checked config and all should be ok. Should be. I am thinking about setting the some sort of syslog to see what is going on.

ASA port is connected to ISP router Cisco 2800. Both port had duplex and speed set to auto. I have changed that manually. What else I can do in order to troubleshoot that?

Do you have a span/monitor

dbellaze — Sun, 06 Sep 2015 05:10:08 GMT

Do you have a span/monitor port capable switch you can use to connect the ASA and the ISP router? This could help give you visibility by mirroring traffic to a device that has tcpdump/wireshark available for analysis.

Below are things I'd look at while the problem is occurring on the ASA.

Can you ping your ISP's side of the connection (2800)?

Do you have a valid arp entry to the ISP's address?

Aside from internet communication do other things to/from/through the ASA appear to work? LAN to DMZ communication? Ping's etc?

Have you checked nat, memory, and cpu resources?

Do you have a static IP or dynamic?

Do interfaces show unusual errors or drops?

Have you conferenced in your ISP and TAC during an outage?

Your ISP should also be able to troubleshoot from their equipment.

Thank You, meantime I have

sprintership-il — Fri, 11 Sep 2015 14:59:51 GMT

Thank You, meantime I have logs from syslog while all was down and could not ping 8.8.8.8.

Do you have a static IP or dynamic? STATIC

Do interfaces show unusual errors or drops? Attached syslog.

I have feeling its either DoS, or some sort of attack. I will try to get answers on all. Thank You.

I have noticed its happening

sprintership-il — Tue, 15 Sep 2015 14:01:04 GMT

I have noticed its happening every time we access NWEA site for testing. All is ok during the days we don't use that site at all.

I found NWEA has some

dbellaze — Thu, 17 Sep 2015 04:06:54 GMT

I found NWEA has some references to DoS on twitter.

https://mobile.twitter.com/NWEA?lang=fil

From the post it sounds like they were attacked in some way so its possible you had/have systems in your network contributing. You mentioned testing as well, maybe a faulty or misconfigured system?

but is that possible their

sprintership-il — Thu, 17 Sep 2015 23:09:58 GMT

but is that possible their problems affecting our network service? I don't want to believe it is possible at all.

What I have done was placing

sprintership-il — Sat, 26 Sep 2015 13:37:32 GMT

What I have done was placing a switch between a router and firewall, mirrored ports and took TCPDUMP. I have a file but another problem is analyzing that file.

The DNS blockout happens at same times, I wonder if there is limit on ASA 5510 for client connection or sessions.

I was able to get tcpdump

sprintership-il — Mon, 28 Sep 2015 00:04:08 GMT

I was able to get tcpdump from the time when I had to reboot ASA again. Looks like right before WAN port "die" there was a lot of https traffic but I don't think I deal with DoS since network was utilized in 48%.

I start taking tcpdump files out of internal network just before they reach ASA and behind both in and out. But this is really drives me crazy since the problem is being since 3 weeks.

What I have noticed based on

sprintership-il — Tue, 29 Sep 2015 15:55:19 GMT

What I have noticed based on tcp dump is a lot https traffic is generated from my side to 23.4.1.138 and 173.194.192.95, but the first one kicks before ASA WAN port dies.

Looks like this is akamai technologies. The question is how that affects and how to stop the traffic?

maybe you're part of the

Peter Koltl — Tue, 29 Sep 2015 18:51:38 GMT

maybe you're part of the problem 🙂

maybe you are one of the attackers? 🙂 some zombies in your network?

At the time of issue did you

Rishabh Seth — Tue, 29 Sep 2015 19:16:59 GMT

At the time of issue did you try pinging internet from the ASA?

Check if the interface is UP or not.

Check ARP entries on ASA and also check if upstream device is passing traffic at the time of issue.

Thanks,

R.Seth

Yes, at that time WAN port on

sprintership-il — Tue, 29 Sep 2015 19:25:51 GMT

Yes, at that time WAN port on ASA looks like is UP - BUT I can't ping anything externally.

Normally, I can ping any ip out of ASA. together with ISP we sat manually speed and duplex on both ASA and router.

ASA > ROUTER (ISP)> switch (ISP)

However, If I unplug the WAN data cable from the router and plug it to the a laptop with my static info I can open any websites without issue.

Hi, You have mentioned that

Rishabh Seth — Tue, 29 Sep 2015 20:02:33 GMT

Hi,

You have mentioned that you can span traffic.

So at the time of issue do you see traffic leaving ASA when :

>> You try to ping public IP from ASA.

>> You try to ping public IP from a host behind ASA.

>> In any of the above cases do you see reply coming back.

It might be possible that there is some issue with upstream device.

And as you have mentioned that when you plug your laptop you do not see the issue. So did you test the connectivity with your laptop directly connected for a week? If not then probably the issue might take some time to occur.

Also try speed and duplex with auto if possible.

Thanks,

R.Seth

well, I cannot use the laptop

sprintership-il — Wed, 30 Sep 2015 12:43:08 GMT

well, I cannot use the laptop connected to my WAN link since this is a production environment. Also, for troubleshooting my time is limited since I have 700 people waiting to be online. 😞 I will be looking for traffic insight of network but I don't think this matters since I see bandwidth utilization for both U/D = 50%

why nobody says to use: show

sprintership-il — Wed, 30 Sep 2015 17:58:25 GMT

why nobody says to use:

show local-host connection tcp 1000 | inc TCP flow count ?????

I think I hit a jock pot and narrowed it down to my problems. So one external IP is not enough to all ports an connections. I may expend that

Make sense?

To check if your PAT pool is

Rishabh Seth — Wed, 30 Sep 2015 18:47:23 GMT

To check if your PAT pool is getting exhausted then you can use command show nat pool and then make a decision to add more IP address for NAT.

Thanks,

R.Seth

damm it, its not the case, I

sprintership-il — Thu, 01 Oct 2015 23:29:25 GMT

damm it, its not the case, I added extra IP as backup of existing one to external,

according to me someone or something from internal network attaching my external IP, rather looking on the syslog.

Is there any way on ASA see what internal IP is messing with external IP?

can you try following at the

Rishabh Seth — Fri, 02 Oct 2015 09:36:05 GMT

can you try following at the time of issue:

>> ping any hostname from ASA and capture traffic to see if the outgoing traffic is getting source translated.

>> Ping your default gateway to ensure upstream device is is reachable.

>> There could be come problem with upstream device on layer two. Try applying capture for ARP at the time of issue and check if you anything unusual.

>> instead of reloading ASA, try bouncing interface connected to upstream device.

Let us know your findings.

Thanks,

R Seth

sorry for a confusion, what

sprintership-il — Fri, 02 Oct 2015 12:04:23 GMT

sorry for a confusion, what you mean "try bouncing interface connected to upstream device" ?

Try shut and no shut of

Rishabh Seth — Fri, 02 Oct 2015 12:12:59 GMT

Try shut and no shut of interface connected to ISP device or unplug and plug tha cable from ASA which is connected to ISP device.

Thanks,

R.Seth