topic Re: cisco asa random drop tcp packets send from azure apim in Network Security

cisco asa random drop tcp packets send from azure apim

ctamietto — Fri, 31 Mar 2023 14:00:06 GMT

Hi , I'm in trouble with the communications between Azure API Management and a Cisco ASA Firewall .

At the begin of a get request from apim , randomly some tcp package are dropped from the ASA firewall ( as of picture )

I guess the flow is as follows :

apim send a SYN Packet ( that is dropped because of port number reused , the first)
apim resend a SYN packet after 3 second a couple of times ( that is dropped because retrasmission )
after some minutes the communication resume correctly ( as of picture , the second )

if I try the same get http request repeatedly without using apim but a rest client like postman or a python script all work without errors

why apim cause cisco asa to drop package ?

this is an excerpt from the firewall log

Drop-reason: (tcp-rst-syn-in-win) TCP RST/SYN in window

why the beaviour of apim is different from other rest client ?

any suggestions are welcome

thank you

Re: cisco asa random drop tcp packets send from azure apim

MHM Cisco World — Fri, 31 Mar 2023 14:31:34 GMT

ASA 8.3 Issue: MSS Exceeded - HTTP Clients Cannot Browse to Some Websites - Cisco

Re: cisco asa random drop tcp packets send from azure apim

ctamietto — Fri, 31 Mar 2023 20:41:01 GMT

I've tried but the issue is not the case

the log does not show any MSS Exceeded

instead I get TCP RST/SYN in window

Re: cisco asa random drop tcp packets send from azure apim

MHM Cisco World — Fri, 31 Mar 2023 21:12:10 GMT

show local-host <IP>
it give us some hint why the TCP is failed

also please share this

FW# capture CAPI interface IN match icmp host <IP> host <IP>
FW# capture CAPI interface OUT match icmp host <IP> host <IP>

Re: cisco asa random drop tcp packets send from azure apim

ctamietto — Sun, 02 Apr 2023 15:28:12 GMT

OK , I'll got it

meanwhile I'd like to illustrate some test I've done

The image below is about the tracing of http calls made by a python script (without going through apim and it's working without problem).

I highlighted the start ( SYN ) and end ( FYN ) of TCP sessions (for every HTTP call there is one).

the two images below are about tracing of http call made by apim

In this case there is only one initial connection opening ( SYN ) after which apim reuses ( at least so it seems to me ) the same session to send another http frame .

Now, if for any reason the apim thinks it has lost the tcp session it will try to open another one and at this point I think will happen what you see in the image below

The firewall assumes that an already open session exists and drops the tcp frame.

So , some question ...

the behavior of apim is correct ? is it right that the firewall drop the tcp frames ?

what does mean this error ?

Drop-reason: (tcp-rst-syn-in-win) TCP RST/SYN in window

thanks for any answer

Re: cisco asa random drop tcp packets send from azure apim

MHM Cisco World — Wed, 05 Apr 2023 22:51:48 GMT

I suspect of asymmetric routing.

Can you confirm you have multi path ??

Re: cisco asa random drop tcp packets send from azure apim

ctamietto — Thu, 06 Apr 2023 07:23:11 GMT

Solved ,

It isn't an asa problem .

I try to explain .

the problem was the keep alive timeout of a proxy that receive the http requests from the apim ( going through the firewall )

this is the sequence that leads to the error

apim open a tcp connection ( SYN frame )
proxy answers with a SYN ACK
firewall asa consider the TCP connection open
a couple of minutes and the keep alive timeout of nginx proxy expires
proxy close the connection with a FIN ACK
apim answers with a ACK ( not a FIN ACK ) and consider the connection closed
for the asa firewall the connection is not closed
apim try to reopen the connection ( SYN frame )
the asa firewall consider the connection closed end drop the package
from this point on http calls no longer work

solution : increase the keep alive timeout of the proxy ( greater then the corresponding timeout of apim ) .

in this way the sequence becomes this :

apim open a tcp connection ( SYN frame )
proxy answers with a SYN ACK
firewall asa consider the TCP connection open
nginx does not close the tcp session
the apim's timeout the timeout expires
apim emit a FIN ACK and consider the connection closed
proxy answers with a FIN ACK and consider the connection closed
asa firewall consider the tcp connection closed
apim try to reopen the connection ( SYN frame )
asa firewall accept the request and send the request to the proxy
all is working , apim make http request to the proxy without problems

thanks for your support