Problems with two-way UDP connections in firewall (CISCO ASA)

[EHA]

Hi

I have a problem that I`m unable to find a solution for and was hoping someone could help me.

The protocol in play:
SAIA SBUS IP UDP port 5050. A PLC to PLC communication protocol that implement a “guarantied” transmission: more information 26-776_ENG_Manual_TCP-IP-Ethernet.pdf (sbc-support.com)

The setup:
So, I have two process-plants each with a PLC and connected by to firewalls FRP1010 running ASA code (9.17.(1)). And CISCO VTI rote based VPN. So far so god. Everything works as expected. I can ping, use web and prosess traffic is flowing both directions. But when there is a disruption in the connection. Example a reboot for some equipment. Or a drop in the VPN, the SBUS communication does not start op again. Everything else starts up. I can even ping the equipment, but the S-BUS will still not initiate. We used to use policy-based VPN and I don`t recall having this problems then. This started when switched to route based VPN. The main log of the firewall does not indicate that packet are being dropped. Or that anything is out of function.

In the image below you can se a whireshark snip of the communication. We put the VTI interface in shut until the PLC reported communication error. As you can see it starts retry.

When we activate the interface again communication is restored but only one way.  The 10.141.0.0 are able to send data to the frontend at.10.130.0.0 but the front end is trying to reply but its dropped by the firewall

If we look at the conn table we find that the connection is registered and I have a feeling that this might be the core of the problem. Like there is a session mismatch. For when the tunnel Is taken down there is no tear-down of the connection. And the 10.130. will continue to send data. Once the communication Is restored, a “new” connection is established the opposite way and I get a mismatch resulting in communication not being able to flow.

Once I run the: “clear conn”

The communication immediately starts up two way again.  

In addition I se the

|10.130.11.70|1066|10.141.0.162|5050|Built outbound UDP connection 121955 for VTI03-COM4:10.141.0.162/5050 (10.141.0.162/5050) to production:10.130.11.70/1066 (10.130.11.70/1066)

A new connection is being created.

But every time there is a communication interruption, I have to log inn and clear the connections.

Does anybody have any suggestion on how to solve this. I would prefer not to roll back to policy based VPN as its introduces other problems.

In advance thank you

EH 

[Georg Pauwen]

Hello,

I think your analysis of the issue is spot on. Are you running FTD ? You could configure a service policy for just that one service (UDP 5050) and configure a very short timeout, enable 'Reset Connection Upon Timeout' and also 'Detect Dead Connections'.

The link below has instructions for setting these session timeouts...

https://community.cisco.com/t5/security-blogs/session-timeouts-in-cisco-ftd/ba-p/4405166

[MHM Cisco World]

I think you need here EEM, 
where in both side when the VTI is unreachable then clear conn. 
this force the ASA to re-establish new UDP traffic.

[EHA]

Hi,
Thanks for replying. Unfortunately session timeouts are of no use. The lowest settings on my ASA is 1min for UDP. The PLS pushes data more or less every second. I have tried setting the timeout as low as possible. But as mentioned it`never reaches the timeout limit. 

So there is also switches and other equipment behind the firewall. So only relying on the status of the VTI would probly not work as well. Sins f.eks a switch or the firewall itself is rebooted.

A possible solution is to use EEM and drop connections every 4 min. (4min is the time taken before comm. failure alerts har sent out by SMS). But this again will risk missing important parameters that are sent continuously, so not really a good option.

Does the firewall handels this trafikk differently in policy mode? So that might be the solution?

[Georg Pauwen]

Hello,

I had a look at the manual: would it not be possible to set a timeout value somewhere in the PLC itself ?

Conn_Tout: if Conn_Tout (seconds) is exceeded without a telegram being
received via the channel, the channel is closed (unless something is
being received).
● 0 = no timeout check
● x = timeout check every x seconds

With the policy-based VPN, what did you actually configure with regard to the PLC traffic ? Was UDP 5050 traffic encrypted or not ?

[EHA]

I`ll have a look at it with the PLC folks to se what what they come up with. Yes, UDP was defined as interesting trafikk and encrypted. 

[MHM Cisco World]

event manager applet PREEMPT
 event syslog id 622001 occurs 2
 action 1 cli command "clear local-host x.x.x.x"
 output none

syslog id 622001 

622001

Error Message %ASA-6-622001: string tracked route network mask address , distance number , table string , on interface interface-name

Explanation A tracked route has been added to or removed from a routing table, which means that the state of the tracked object has changed from up or down.

• string —Adding or Removing
• network —The network address
• mask —The network mask
• address —The gateway address
• number —The route administrative distance
• string —The routing table name
• interface-name —The interface name as specified by the nameif command#

so we run track then using the track syslog id to trigger the action of clear local-host x.x.x.x << we only clear conn of host  x.x.x.x not all conn. 

you can check clear local-host before decided to apply EEM.

[EHA]

Thanks a lot for your suggestion, I`ll try to implement it over new year, and report back.
Marry Christmas. 

EH 

[MHM Cisco World]

Marry Christmas Friend