Introduction
Some users may receive the following error when connecting to the router from CSM Server via ssh:
Traceback (most recent call last):
File "/usr/local/csm/csmserver/work_units/inventory_work_unit.py", line 78, in start
discover_platform_info(ctx)
File "/usr/local/csm/csmserver/handlers/base.py", line 65, in discover_platform_info
conn.connect()
File "/usr/local/lib/python2.7/site-packages/condoor/connection.py", line 207, in connect
raise ConnectionError("Unable to connect")
ConnectionError: Unable to connect
A number of causes can lead to this error message. An example to work around the error is given below.
Core Issue
One of the causes is mentioned in https://community.cisco.com/t5/xr-os-and-platforms/changing-ios-xr-ssh-server-ciphers/td-p/3034341
The description of CSCvb53125 has to do that the weaker cbc cipher was disabled since Openssh 6.7 due to the known security vulnerabilities:
https://www.kb.cert.org/vuls/id/958563
The cipher scheme has been enhanced in IOS-XR 6.5.1, 6.4.2, 6.2.3, 6.2.25, 6.2.2, and 6.2.1, but older releases are still affected.
An example of the error message that users could see is as follows:
"no matching cipher found. Their offer: aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc"
The other known cause is that DSA crypto key is generated on the router. This can be verified by the following IOS-XR EXEC command 'show crypto key mypubkey dsa'.
Users may run into an error message similar to the one below:
"no matching host key type found. Their offer: ssh-dss"
Users may encounter other ssh error messages when establishing the connection with the router. The solution may vary but will be similar.
When upgrading IOS-XR or changing the IOS-XR crypto key is not a practical option, prior to CSM 4.0, users were able to manually change/add in SSH HEX, CIPHER and HostKeyAlgorithms on the Linux host machine as a workaround.
Since CSM Server 4.0 runs inside a Docker container, changing the host machine’s /etc/ssh/ssh_config & /etc/ssh/sshd_config is no longer sufficient to resolve the SSH issue.
Resolution
The design of CSM Server 4.0 allows users to map any host file to the csm-server container.
Please note that the mapping between the Linux host and csm-server container has been enhanced in CSM Server 4.0.5.
[New Method]
Please issue the following command from the Linux host to test if the new method is supported. If the returned output is empty, please fall back to the section of [Old Method] that is given below.
# docker inspect csm-server | jq '.[0].Mounts[] | select(.Source == "/usr/share/csm/ssh")'
[sudo] password for cisco:
{
"Type": "bind",
"Source": "/usr/share/csm/ssh",
"Destination": "/etc/ssh",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
In the above output, /usr/share/csm/ssh/ssh_config in the Linux host is already mapped to /etc/ssh/ssh_config inside the csm-server container.
Please edit /usr/share/csm/ssh/ssh_config in the Linux host and restart the csm-server container with the command below for the changes to take effect.
# curl -sXPOST http://172.33.32.2/csm/restart | jq
{
"result": "ok",
"data": {}
}
[Old Method]
Below is an example based on Ubuntu 18.04. For other Linux distributions or releases, the ssh configuration may be different but the solution will be the same. Users can edit ssh_config on the Linux host then use the steps outlined below to map the changes to the ssh_config inside the csm-server container.
The example uses the jq tool to nicely format the json file, if you don’t have it installed just omit ‘| jq’ part in below commands.
Please execute all the commands in the Linux host machine (not in the container) except for step 7 after entering into the csm-server container.
1. On Ubuntu 18.04, please edit /etc/ssh/ssh_config.
- To configure the ssh-dss as a host key algorithm, please add the following configuration:
HostKeyAlgorithms=+ssh-dss
- To allow the weaker cbc cipher, please enable the following ciphers:
Ciphers aes128-cbc,3des-cbc
- For other errors such as GSSAPIAuthentication related, users may need to disable GSSAPIAuthentication:
GSSAPIAuthentication no
- After /etc/ssh/ssh_config is edited, please verify that the changes are valid and you are able to ssh to the router manually from the Linux host.
2. Check the current configuration with the following CSM Supervisor API:
curl -s http://172.33.32.2/csm/info | jq
{
"result": "ok",
"data": {
"version": "4.0.0",
"last_version": "4.0.0",
"machine": "qemux86-64",
"image": "devhub-docker.cisco.com/csm-docker/qemux86-64-csm-server",
"custom": false,
"boot": true,
"port": 5000,
"ssl": false,
"watchdog": true,
"wait_boot": 120,
"in_progress": false,
"volumes": {} <======== This is empty now
}
}
3. Configure and map the host file/volume to the container file/volume. In this case we bind /etc/ssh/ssh_config from host to /etc/ssh/ssh_config inside the csm-server container:
$ curl -sX POST http://172.33.32.2/csm/options -d '{"volumes":{"/etc/ssh/ssh_config": { "bind":"/etc/ssh/ssh_config"}}}' | jq
{
"result": "ok",
"data": {}
}
4. Check the configuration again:
$ curl -s http://172.33.32.2/csm/info | jq
{
"result": "ok",
"data": {
"version": "4.0.0",
"last_version": "4.0.0",
"machine": "qemux86-64",
"image": "devhub-docker.cisco.com/csm-docker/qemux86-64-csm-server",
"custom": false,
"boot": true,
"port": 5000,
"ssl": false,
"watchdog": true,
"wait_boot": 120,
"in_progress": false,
"volumes": {
"/etc/ssh/ssh_config": {
"bind": "/etc/ssh/ssh_config"
}
}
}
}
5. Restart CSM Server Container:
$ curl -sXPOST http://172.33.32.2/csm/restart | jq
{
"result": "ok",
"data": {}
}
6. Verify that csm-container has picked up the new config:
$ docker inspect csm-server | jq '.[0].Mounts[] | select(.Source == "/etc/ssh/ssh_config")'
{
"Type": "bind",
"Source": "/home/cisco/custom_ssh_config",
"Destination": "/etc/ssh/ssh_config",
"Mode": "rw",
"RW": true,
"Propagation": "rprivate"
}
7. Check the file inside the csm-server container:
Please execute the following command to enter the csm-server container:
$ docker exec -it csm-server bash
Inside the csm-server container, please issue the following command:
bash-4.4# cat /etc/ssh/ssh_config
Please verify that any of the following configurations that was configured in the first step has become available.
HostKeyAlgorithms=+ssh-dss
Ciphers aes128-cbc,3des-cbc
GSSAPIAuthentication no
Finally, please exit the csm-server container:
bash-4.4# exit
8. This configuration is persistent and stored between CSM server container reloads or host machine reloads.
The config is stored on host in /etc/share/csm/csmserver.json file, however please do not edit this file manually (use the rest call as above).
$ cat /usr/share/csm/csmserver.json
{
"volumes": {
"/etc/ssh/ssh_config": {
"bind": "/etc/ssh/ssh_config"
}
},
"watchdog": true,
"ssl": false,
"wait_boot": 120,
"uuid": "4335567342344e73bde213c9d2070e25",
"boot": true,
"port": 5000
}
