cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
198
Views
3
Helpful
5
Replies

SSH Node Timeout on Long-Running Tasks

thein3rovert
Visitor

I'm running into a problem with the Unix/Linux Execute SSH Command node when the remote command takes longer than a few minutes to complete.
Behavior observed:
- The node establishes an SSH session to a remote host and executes a long-running command.
- After ~5–10 minutes of low/no output from the command, the node's SSH session is silently terminated.
- The workflow either times out with no result, or hangs in "running" state indefinitely until the individual command timeout fires (e.g. 3600s).
- On the remote host, the command actually completes normally and produces expected output — but that output never reaches the workflow because the SSH connection is already gone.

Suspicion:
The SSH client used by the Unix/Linux SSH node doesn't appear to send TCP/SSH keep-alives during periods of low command output. Something in the connection path (Cisco-side, or an intermediate device the node's traffic traverses) is treating the idle SSH channel as dead and dropping the connection before the command finishes. If keep-alives were being sent, the connection would be held open regardless of command output timing.
What I'd like to confirm:
1. Does the Unix/Linux SSH node send TCP/SSH keep-alives on active sessions?
2. If so, is the interval configurable, and what's the default?
3. Is there a recommended pattern for running commands that legitimately take 10+ minutes to complete over this node without decoupling into a fire-and-forget model?

5 Replies 5

@thein3rovert I'll check, but I'm not aware of any keep-alive mechanism in the adapter to maintain an active connection while waiting for command output.  Is there any intermediate firewall/loadbalancer/NAT on the connection between your Automation Remote and the Target?

Network Platform Team
Workflows - Technical Marketing Engineer

@Ed Novak - Cisco Workflows TME Thanks for confirming.  We've done the test to rule out the issue.
We deployed a second server on the same subnet as our Automation Remote - no ACL, firewall, load balancer, or NAT anywhere in the path between them.

We ran a  "for i in {1..300}; do echo "heartbeat $i $(date +%T)"; sleep 1; done" script. 

The same script runs cleanly to completion when triggered from an interactive SSH terminal to the exact same target so this is specific to how the SSH node adapter behaves, not the script itself.

This strongly suggests a client-side behavior in the Unix/Linux SSH node adapter — the SSH session is being closed after a fixed duration regardless of activity or path.

Please help look into this 😊

thein3rovert_0-1785407075028.png

thein3rovert_1-1785407115419.png

 

Following up with additional diagnostics I ran directly on the Automation Remote appliance (SSH'd in and  tailed the adapter logs while triggering the workflow).

Confirmed findings:
1. The Unix/Linux SSH Command node has a hardcoded ~180 second (3 min) internal timeout that ignores the "Activity timeout" UI setting. I set the Activity timeout to 1800s, then 3600s — in both cases the adapter logged "Error Message: Activity timeout" at exactly 180 heartbeats (each heartbeat is 1 second):


2026-08-03T17:21:09Z terminal.ssh_command tag=8e969859
2026-08-03T17:24:09Z Error Message: Activity timeout. Last output: heartbeat 180 17:24:13...

2. The container is completely healthy at the moment of failure: no OOM, no restart, no CPU/memory spike. docker stats showed 1.49% CPU / 27 MiB memory during the run.

Please, i need update on this.

@thein3rovert This is clearly a bug in the adapter. I'll get a defect opened for this. I don't have a workaround or an ETA for a fix at this time.  Thank you for bringing this issue to my attention and for your very detailed analysis.

Network Platform Team
Workflows - Technical Marketing Engineer

Thanks again for the fast turnaround, really appreciate you confirming this.