SSH connect via Putty hangs

skele9802274 · ‎12-12-2018

I created a post recently with some issues with connected to switches via SSH. This was solved (but not entirely clear why) by removing the unnecessary configuration of SVI's.

I still have two switches that I am unable to connect to via SSH.

Topology is as follows

Server --> Nexus 3k --> 2960x

SSH from Nexus 3k to Switch works fine, but when attempting SSH from the server to the 2960x Putty just hangs on a blank screen.

While the blank screen is open I can see a connection to the switch

2960#sh users
    Line       User       Host(s)              Idle       Location
* 1 vty 0     matthew.cu idle                 00:00:00 192.168.1.1
   2 vty 1                idle                 00:00:10 192.168.2.149

Interface User Mode Idle Peer Address

If I do s debug on SSH I can see the connection come in, but it seems to stop at the Diffie Hellman Exchange.

Dec 12 16:16:22.594: SSH1: starting SSH control process
Dec 12 16:16:22.594: SSH1: sent protocol version id SSH-2.0-Cisco-1.25
Dec 12 16:16:25.233: SSH1: protocol version id is - SSH-2.0-PuTTY_Release_0.69
Dec 12 16:16:25.233: SSH2 1: kexinit sent: encryption algo = aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
Dec 12 16:16:25.233: SSH2 1: kexinit sent: mac algo = hmac-sha1,hmac-sha1-96
Dec 12 16:16:25.233: SSH2 1: send:packet of length 368 (length also includes padlen of 5)
Dec 12 16:16:25.233: SSH2 1: SSH2_MSG_KEXINIT sent
Dec 12 16:16:25.233: SSH2 1: ssh_receive: 536 bytes received
Dec 12 16:16:25.233: SSH2 1: input: total packet length of 1104 bytes
Dec 12 16:16:25.233: SSH2 1: partial packet length(block size)8 bytes,needed 1096 bytes,maclen 0
Dec 12 16:16:25.233: SSH2 1: ssh_receive: 536 bytes received
Dec 12 16:16:25.233: SSH2 1: partial packet length(block size)8 bytes,needed 1096 bytes,maclen 0
Dec 12 16:16:25.233: SSH2 1: ssh_receive: 32 bytes received
Dec 12 16:16:25.233: SSH2 1: partial packet length(block size)8 bytes,needed 1096 bytes,maclen 0
Dec 12 16:16:25.233: SSH2 1: input: padlength 4 bytes
Dec 12 16:16:25.233: SSH2 1: SSH2_MSG_KEXINIT received
Dec 12 16:16:25.236: SSH2 1: kex: client->server enc:aes256-ctr mac:hmac-sha1
Dec 12 16:16:25.236: SSH2 1: kex: server->client enc:aes256-ctr mac:hmac-sha1
Dec 12 16:16:25.236: SSH2 1: Using kex_algo = diffie-hellman-group-exchange-sha1

I've Zeroized the RSA keys and generated new ones, I have tried different a modulus etc. But as I've previously said SSH works as I can connect from Nexus to 2960.

Connectivity to 2960 from server is fine, as I can ping and the switch can also authenticate to TACACS server.

Any help would be appreciated

Regards

Matt

Peter Paluch · ‎12-12-2018

Hi Matt,

This is a blind shot, but I am thinking if perhaps you can have an MTU issue. You see, I would expect the TCP segments to grow as then carry more and more information during the initial SSH exchange, and I wonder if after some time, they could grow to a size where they don't pass through a link or an interface whose MTU is smaller than the packets.

Would you mind trying pinging the 2960 from the server with packet sizes up to 1500 bytes, and the DF bit set (DF bit is not necessary if the server and the 2960 are in the same IP network but still better to test with it), and see if some pings get lost?
If you ran Wireshark on the server while connecting from PuTTY to the 2960, would you see any TCP retransmissions?
Would a different SSH client on the server work?

Thank you!

Best regards,
Peter

skele9802274 · ‎12-13-2018

Peter

Thank you for your reply.

If I test using ping from the sever with the options you describe, I can ping the swith with a MTU of 1472 but it fails at 1473 as it is required to fragment (as expected witht he additional 28 byte of headers).

If I run wireshark while trying to SSH I do see TCP retranmissions.

I havent had a chance to try another SSH client as of yet.

Regards

Matt

skele9802274 · ‎12-17-2018

Just as an update, I have now tried another SSH client and I still get the symptoms

Regards

Matt

pieterh · ‎12-17-2018

some suggestions:

- post output of "show ip ssh"

- on your host check PUTTY registry SSH key's

HKEY_CURRENT_USER\Software\SimonTatham\PuTTY\SshHostKeys
you can also delete this whole key, you will be prompted again to accept the fingerprints

- in your session check SSH session settings (SSH version and compression -> disable compression)

skele9802274 · ‎12-17-2018

Thank you for your reply,

Pleae find the output of sh ip ssh below

SSH Enabled - version 2.0
Authentication methods:publickey,keyboard-interactive,password
Encryption Algorithms:aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
MAC Algorithms:hmac-sha1,hmac-sha1-96
Authentication timeout: 120 secs; Authentication retries: 3
Minimum expected Diffie Hellman key size : 1024 bits
IOS Keys in SECSH format(ssh-rsa, base64 encoded):
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCaueDiUzUUyprDIYsqagunP/xwIlyMVNKUddQvjxyN
biQrBxCocdgP5XG6pBPLJMEx+QlJptIKfs+m4bK1ECiGKF0x13NycRYeYpBxsPvM86kaTXMFbjQqMoQ9
sDOyfXDN9NeZ6tM2Iu2SAI/FiFzpbNMnClTEIb+Z6nWJgjcEocpuAw05qwdJ1snXYwmOVT7H07BH/Kr1
JGQ5wafbniftq3hjJ58S91Xhu8OxntZM7SQkBsZKT/Vtky2fl2VQUpaHS7tX+n26bsNeBn3h4kbPSLl5
gs9D1dZQA0HR8aH3oobnBrXAoq5wuDe1Oe06PLM0rYMHWceLuFIMP/lIetGt

I have already checked thekeys registry and have also deleted them to force accepting them again. Putty client isnt using compression.

Regards

Matt

Peter Paluch · ‎12-20-2018

Hi Matt,

My apologies for coming back to you with such a delay.

Hmm, the ping size of 1472 bytes seems to be okay - you see, 1472 bytes would be the payload, then we have 8 bytes of ICMP header, and 20 bytes of IPv4 header, totalling 1500 bytes. Quite logically, then, a ping size beyond 1472 bytes with fragmentation prohibited would fail.

Do you think you could actually post a saved Wireshark capture for us to understand how does the SSH session negotiation proceed, and where does it stop? It could perhaps shed some light on why the session ultimately stalls. If you perform that capture, please let the PuTTY or the other client stay and try to connect for a minute or two while capturing the packets, and only then stop the capture.

Thank you!

Best regards,
Peter