Could not ssh to ASA after upgrading OS from 9.14.3.18 to version 9.1

[loc.nguyen]

Hi,

We upgraded a pair of ASAs from version 9.14.3.18 to version 9.18(4)40. After that, we could not SSH into the ASA. At the moment, we can access the firewall via the console.

I turned on debugging on the ASA while I attempted to SSH. The log file indicated that the Reg-mgr limit was reached. I checked with the command 'show ssh sessions,' and there were no SSH connections.

Could you help troubleshoot this issue?

Below is our log file:

olo-fw1/pri/act# SSH_EXT: flow lookup on vPifNum 3
SSH_EXT: flow lookup for 216.x.x.x:22<--->10.0.99.11:1995
SSH_EXT: flow lookup on vPifNum 2
SSH_EXT: flow lookup for 10.0.99.250:22<--->10.0.99.11:1995
SSH_EXT: found the flow
SSH_EXT: connection on vPif inside
SSH_EXT: Registering external ssh/sshd
SSH_EXT: vcid 0, Process type 2
Reg-mgr limit reached, dropping ssh on vcid: 0
SSH_EXT: Response send len 16
SSH_EXT: Connection close on fd: 991300522
SSH_EXT: io callback performing cleanup for fd: 991300522
SSH_EXT: cleanup event for non-session proc

show ssh sessions

colo-fw1/pri/act# sh asp table socket

Protocol Socket State Local Address Foreign Address
SSL 04d61538 LISTEN 10.0.99.250:443 0.0.0.0:*
SSL 005637e8 LISTEN 216.x.x.x:443 0.0.0.0:*
DTLS 0015d658 LISTEN 216.x.x.x:443 0.0.0.0:*
SSL 022a92f8 ESTAB 216.x.x.x:443 193.37.69.199:43308
SSL 02eca868 ESTAB 216.x.x.x:443 185.190.24.177:60097
SSL 00a96a88 ESTAB 216.x.x.x:443 94.232.45.152:53152
SSL 05246b08 ESTAB 216.x.x.x:443 79.110.62.203:57920
SSL 052470c8 ESTAB 216.x.x.x:443

[Mark Elsen]

         - Contact Cisco's TAC for this issue ,

 M.

[loc.nguyen]

Thanks.

Unfortunately, our contract has expired, and we are in the process of renewing it. It will take some time.

I tried all the suggestions, including upgrading my PuTTY to the latest version and changing the SSH settings.

At the moment, I believe it's a bug in the new OS because the debug shows that all connections are taken while we have no active connections.

"Reg-mgr limit reached, dropping SSH on vcid: 0"

vbnet
Copy code
colo-fw1/pri/act# SSH_EXT: flow lookup on vPifNum 3
SSH_EXT: flow lookup for 216.117.12.36:22<--->10.0.99.39:48838
SSH_EXT: flow lookup on vPifNum 2
SSH_EXT: flow lookup for 10.0.99.250:22<--->10.0.99.39:48838
SSH_EXT: found the flow
SSH_EXT: connection on vPif inside
SSH_EXT: Registering external ssh/sshd
SSH_EXT: vcid 0, Process type 2
Reg-mgr limit reached, dropping SSH on vcid: 0
SSH_EXT: Response sent, len 16
SSH_EXT: Connection closed on fd: 2505856322
SSH_EXT: io callback performing cleanup for fd: 2505856322
SSH_EXT: cleanup event for non-session proc

[Mark Elsen]

 - Do you get any sensible results from this test with nmap  (https://nmap.org/)
                 %  nmap --script ssh2-enum-algos asa-hostname

 M.

[Marius Gunnerud]

TAC would be a logical next step.

Other things to try is to regenerate the general RSA keys, verify that the command ssh <subnet or IP> <interface name> is present.

[Marvin Rhoads]

I have seen this issue when customers are using an old Putty version that will not negotiate a secure key exchange with the newer ASA software. In such cases, a KEX error usually appears in the client side debugging. Upgrading your ssh client software fixes that.

https://community.cisco.com/t5/network-security/asa-upgrade-lost-ssh-access/td-p/5119558

[loc.nguyen]

packet capture shows firewall reset the connection. I did try create new key and upgrade putty, it did not help

colo-fw1/pri/act# show cap capin

6 packets captured

1: 13:37:19.791127 10.0.99.39.49044 > 10.0.99.250.22: S 2888070741:2888070741(0) win 29200 <mss 1460,sackOK,timestamp 485685972 0,nop,wscale 7>
2: 13:37:19.791920 10.0.99.250.22 > 10.0.99.39.49044: S 3554735613:3554735613(0) ack 2888070742 win 28960 <mss 1380,sackOK,timestamp 4172860326 485685972,nop,wscale 7>
3: 13:37:19.792042 10.0.99.39.49044 > 10.0.99.250.22: . ack 3554735614 win 229 <nop,nop,timestamp 485685973 4172860326>
4: 13:37:19.792317 10.0.99.39.49044 > 10.0.99.250.22: P 2888070742:2888070763(21) ack 3554735614 win 229 <nop,nop,timestamp 485685973 4172860326>
5: 13:37:19.792515 10.0.99.250.22 > 10.0.99.39.49044: . ack 2888070763 win 227 <nop,nop,timestamp 4172860327 485685973>
6: 13:37:19.849855 10.0.99.250.22 > 10.0.99.39.49044: R 3554735614:3554735614(0) ack 2888070763 win 227 <nop,nop,timestamp 4172860384 485685973>
6 packets shown
colo-fw1/pri/act#

[johnlloyd_13]

hi,

are you doing SSH from a jump server? the ASA SSH key exchange protocol (DH) might've changed during upgrade

verify in the ASA with a 'show run ssh' and configure a 'weak' DH group as an interim. refer to helpful link below.

you should upgrade your SSH client that supports newer DH group/SSH key exchange as permanent fix.

https://ccnpsecuritywannabe.blogspot.com/2024/04/cisco-asa-firewall-ssh-key-exchange.html