ā04-26-2024 04:54 AM
After upgrading ASA to version 9.18.4 we are seeing Azure Traffic Manager probes being dropped (or discarded).
In the logs we see 'TCP access denied by ACL from <traffic manager IP> to Outside Interface /443'
It works fine when reverting back to 9.16.4
Not configuration changed were made
Has anyone seen this happen? What is it in the config that allows the probes?
ā04-26-2024 09:12 AM - edited ā04-26-2024 10:27 AM
It seems there are two discussions for the same issue:
ASA only needs "webvpn" / "enable Outside" (or "http server enable" / "http <subnet> <mask> Outside") in order to respond to SYN packets over TCP/443. Do you have it enabled? Do you have any control-plane ACLs assigned (check with "show run access-group")?
What happens after SYN, when GET request arrives, is a big question. I'm not aware of any filtering by request URL, although who knows...
You can collect "capture cap-asp type asp-drop match ..." for TCP/443 to the firewall IP address. Then display it with "show capture cap-asp [packet-number <N>]". It will show memory address where drop happened, such as "Drop-location: frame 0x0000564694f7624e". TAC will be able to decode it and tell you exactly where drop happened.
ā05-02-2024 03:51 AM
We have webvpn enabled and http server enabled (I assume this is for ASDM). We have ensure that the control plane ACL is replicated from the working ASA.
In the logs we are seeing TCP Reset-O from the Azure Traffic Manager IP addresses.
I've attached a picture of the logs. The ASA seems to allow the probes for a minute or so, then it times out
Can you help with what is happening here please
ā05-02-2024 06:19 AM
Most of the connections timed out after 30 seconds of inactivity, which is a typical TCP SYN timeout, so it looks like TCP 3WHS didn't complete.
Do you reload the ASA and after that probes start working for a minute or so? What do you mean by "control plane ACL is replicated"? Is it a failover pair and you send probes to a standby IP address, or you just mean that control plane ACL is identical between working and non-working ASAs running different versions of code?
Please provide
show run webvpn
show run http
Then start captures:
capture cap-out int OUTSIDE_INET match tcp any host <ASA-IP>
capture cap-asp type asp-drop match any host <ASA-IP>
and periodically collect:
show resource usage all
show asdm sessions
show quota management-session
show asp table socket
show asp table socket stats detail
show asp drop
show processes | grep Unicorn
Then copy captures off the device:
copy /pcap capture:cap-out ftp://.......
copy /pcap capture:cap-asp ftp://.......
no capture cap-out
no capture cap-asp
Then provide collected information.
ā05-09-2024 12:05 AM
Hi tvotna,
Yes that is correct, the issue is happening when I upgrade the Firepower ASA - which has ASA Software, and reload it. It shows as 'Active' on Azure Traffic Manager for 90 seconds or so, and then goes to 'Downgraded'
I just meant we have the same control plane ACLs on the non working ASA as we do the working ASA
The most recent version that seems to work (doesn't have this issue) is cisco-asa-fp2k.9.19.1.SPA
This is the capture we are seeing - I have replaced our external IP address with xxx.xxx.xxx.xxx
We have tried allowing http all, and can connect reach the ASA from the outside network.
This is cap-asp
Many thanks
ā05-09-2024 02:09 AM
As you can see from the capture, the ASA is only responding to a small number of packets, not all. I believe this is why it is showing as 'degraded'
Is there any reason why upgrading the version of code would stop the ASA from responding to all
This is a capture from the working ASA - as you can see here the ASA is responding to the probes from Azure Traffic Manager
Many thanks
ā05-09-2024 06:48 AM
We have very little information to judge and what we have is confusing. From syslog shared earlier it appears that inbound connections to TCP/443 are created. From the non-working capture we see that the ASA doesn't respond to TCP/443 SYNs at all. SYN/ACK is never sent. It sends only TCP RST in response to TCP RST coming from the client. Finally you mentioned "We have tried allowing http all, and can connect reach the ASA from the outside network". This is confusing. I recommend you opening a TAC case and let TAC engineer troubleshoot the issue live on Webex. You'll probably need to schedule maintenance window, reload the ASA with the non-working version so that the engineer can start capture right after device comes online and follow the exact troubleshooting procedure I shared earlier.
The traffic manager may report that device is reachable for 90 seconds or so, but this might not mean the device has ever responded to TCP/443 SYN packets. This possibility needs to be ruled out or confirmed. Then configuration of control plane ACL and http x.x.x.x y.y.y.y needs to be verified too. It is possible to verify this in the low-level firewall ASP tables as well. The commands I shared earlier can help understand what happens with TCP/443 socket and various limiters. Etc.
ā05-10-2024 05:10 AM
the HTTP to ASA is pass through rule before Allow or deny
I draw the rule it must pass
the capture you share meaning that the Azure is send "S" i.e. it try SYNC the TCP connection and ASA not reply
check this points and I waiting your reply
MHM
ā05-02-2024 07:03 AM
So you have control acl'
And you config http 0.0.0.0 0.0.0.0 also ?
If yes the subnet config with http command override the control acl'
I.e. if you permit traffic in control acl and dont specify subnet in http command that not work.
MHM
ā04-26-2024 09:22 AM
it can Thread detection drop the packet try excluded the IP
MHM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide