Re: live streaming interrupt by ASA

lamkuokcheng · ‎10-17-2010

Dear all,

i encounter a problem during the live video streaming, the ASA seems denied some packet in a period of time about 1.5 hour.

The problem:

PC A live stream to outside public server IP address, but every 1.5 hour will disconnect, but it will auto-reconnect after about 4 seconds.

Monitor action:

i setup PC B for the ping monitoring.

1. ping google.com

2. ping router

3. ping ASA

4. ping subnet core switch

and the topology is

internet ---> router ---> ASA ----> core switch ---> PC A & PC B

after 1.5 hour, the streaming is disconnect and the ping 1. 2. is timeout for 6 packet at the same time, and i try it about every 1.5 hours, the condition is same , drop 6 packet and resume.

so i guess the problem is ASA and i found the log show as:

Oct 09 2010 13:39:47: %ASA-4-313004: Denied ICMP type=0, from laddr 202.1XX.XX.XX on interface outside to 172.17.XX.XX: no matching session

Oct 09 2010 13:39:47: %ASA-4-313004: Denied ICMP type=0, from laddr 74.125.71.103 on interface outside to 172.17.XX.XX: no matching session

Oct 09 2010 13:39:47: %ASA-4-313004: Denied ICMP type=0, from laddr 202.1XX.XX.XX on interface outside to 172.17.XX.XX: no matching session

Oct 09 2010 13:39:47: %ASA-4-313004: Denied ICMP type=0, from laddr 74.125.71.147 on interface outside to 172.17.XX.XX: no matching session

Oct 09 2010 13:39:47: %ASA-4-313004: Denied ICMP type=0, from laddr 58.64.163.86 on interface outside to 172.17.XX.XX: no matching session

Oct 09 2010 13:39:47: %ASA-4-313004: Denied ICMP type=0, from laddr 192.168.24.254 on interface outside to 172.17.XX.XX: no matching session

Oct 09 2010 13:39:47: %ASA-4-313004: Denied ICMP type=0, from laddr 74.208.112.70 on interface outside to 172.17.XX.XX: no matching session

Oct 09 2010 13:39:48: %ASA-4-313004: Denied ICMP type=0, from laddr 202.1XX.XX.XX on interface outside to 172.17.XX.XX: no matching session

Oct 09 2010 13:39:48: %ASA-4-313004: Denied ICMP type=0, from laddr 74.125.71.106 on interface outside to 172.17.XX.XX: no matching session

Questions:

it seems the ASA denied the back packet because no matching session, it is the ASA auto cancel the current session?

when the ICMP denied by ASA(show as log) the streaming is also disconnect, so is the ASA reset all session at the same time?

any config will cause this condition?

do i need to post any config?if needed i will.

any idea for this?

thank you for any help!!

best regards

Jitendriya Athavale · ‎10-17-2010

when this issue happens

can you please run this command

show conn | in

want to see if the connection entry is actually not thr or is ASA failing to recognise it

also please paste the output of

show run timeout

lamkuokcheng · ‎10-18-2010

Dear jathaval:

i can't reply the issue now, but when the issue happen, i record the sh conn for the streaming session as below:

TCP out 173.XXX.XXX.XXX:1935 in 172.17.XXX.XXX:49180 idle 0:00:00 bytes 15591218 flags UIO <---original session has timed out

TCP out 173.XXX.XXX.XXX:1935in 172.17.XXX.XXX:49175 idle 0:37:47 bytes 848143 flags UIO <------ new session has strat

i have not record the ping session for "sh conn"

and the "sh run timeout" as below:

############

timeout xlate 3:00:00

timeout conn 0:30:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02

timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00

timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00

timeout uauth 0:05:00 absolute

#####################

also the ASA has those inspection enabled

#########################

policy-map type inspect dns migrated_dns_map_1

parameters

message-length maximum 512

policy-map global_policy

class inspection_default

inspect dns migrated_dns_map_1

inspect ftp

inspect h323 h225

inspect h323 ras

inspect netbios

inspect rsh

inspect rtsp

inspect skinny

inspect sqlnet

inspect sunrpc

inspect tftp

inspect sip

inspect xdmcp

"inspect icmp"

!

################

does any relate for the inspection of icmp? for the session not matching?

anyway thank you for your reply

i really need to find out the problem to avoid next interruption.

best regards

Jitendriya Athavale · ‎10-18-2010

what is weird is that the new one is idle for 37 mins, i dont quite understand why that happened, but again in the first place why were 2 connection entries created.

please try the following, apply captures on the outside interface when the issue happens, lets see how the packets are flowing, i am more interested in the port numbers as that is one factor which can tell us which connection is being used and what happened

do you have logs for this traffic when the issue happened, did you see any drops mentioned in the logs

lamkuokcheng · ‎10-21-2010

Dear Jathaval,

Thank you for you reply, the idle is 37min because when the issue happen(on the issue happen day), i tied to change some kind of those parameter to try to fix the problem, and i leave the value since the live streaming event, the original "timeout conn" is 1:00:00.

because the event is over and the equipment and PC is provide by the third party just for the event, i can not buildup the same environment to trigger the same problem.

but i still don't give up to try to reiterate the problem, i buildup a video conference connection via H323 to simulate the environment, connection is OK and last 5~6 hours with no disconnect.

Today, i have start to ping thought each hop of our network in order to check the network connectivity, including the some outside host (like google.com, yahoo.com etc).

At the afternoon, my ping progress has dropping packet (about 10 and the latency is become very high about 1500ms) after about 10 packet drop the ping become normal and i can see the log at ASA as following:

#################

Oct 21 2010 15:48:15: %ASA-4-313004: Denied ICMP type=0, from laddr 64.233.189.104 on interface outside to 17X.XXX.XXX.XXX: no matching session

Oct 21 2010 15:48:25: %ASA-4-313004: Denied ICMP type=0, from laddr 64.233.189.104 on interface outside to 17X.XXX.XXX.XXX: no matching session

Oct 21 2010 15:48:32: %ASA-4-313004: Denied ICMP type=0, from laddr 64.233.189.104 on interface outside to 17X.XXX.XXX.XX: no matching session

Oct 21 2010 15:48:32: %ASA-4-313004: Denied ICMP type=0, from laddr 1XX.XX.XX.XX on interface outside to 17X.XXX.XXX.XX: no matching session

Oct 21 2010 15:48:37: %ASA-4-313004: Denied ICMP type=0, from laddr 1XX.XX.XX.XX on interface outside to 17X.XXX.XXX.XX: no matching session

################

PS: 64.233.189.104 is google and 1XX.XX.XX.XX is our remote site public IP.

the log has exactly same as the live streaming interrupt issue!!

now, i have the question is: Is really the ASA cause the problem? according to above log, it was no doubt the ASA denied the ICMP packet. but how to explain the "no matching session" ? ASA or Router has reset the session?

once again, our network topology is:

Internet ---> router ---> ASA ----> core switch ---> My PC

so ASA detected no matching session and deny the icmp form the outside back to inside, causing my ping is drop.

next layer device is the router, is the router reset the session cause the ASA can not recognize the existing session then denied it?

but how can the router reset or clear a active session? is there any parameter to set it?

any idea?

thanks for help anyway!!

best regards

Jitendriya Athavale · ‎10-21-2010

i am guessing this could be bcoz of the fact that icmp timeout is 2 sec and may be due to delay in the line somewhr the session might be timed out and it might be dropped by the firewall. th ereason i am thinking on these lines is bcoz you said that for pings which succeed the time is too high

can you please check the interface counters on the asa and the router and see if we have any drops or find something wierd like duplex setting being half or something like that

though this does not entiry explain the drops u saw suring the event, but let us try to find the cause for this too

lets try this

apply captures when u see ping drops, lets see if the packets come in late bcoz of which the asa does not have the conn entry anymore

also collect some logs, let us see if the asa makes note of any connection tear down , you can collect buffered logs at level 7 or if u have syslogs that is ideal