cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1748
Views
25
Helpful
7
Replies

NSO upgrade - SSH Shell Broken/Crippled?

tpomerhn
Cisco Employee
Cisco Employee

Hello all, I was using NSO 5.3.0.1 for almost a year and it was working great. The main usage was to allow other scripts to leverage the parsing/interaction that NSO provides to give a consistent experience no matter the end device. Most of the time, I was doing this by issuing commands to the NSO box over shared-key passwordless ssh, as follows:

 

ssh admin@nso_server_ip -i /opt/admin/.ssh/admin-nso-key -p 2024 -o StrictHostKeyChecking=No 'show devices list'

 

...which would output the true output of the command.

 

I just updated the system to 5.4.1.1 and updated all the NEDs, but now when I issue the above command, I get zero output. It's as if the update to NSO has resulted in a non-shell SSH subsystem that cannot accept the sending of commands over the SSH session. But I need that.

 

If I do the above command but don't include the 'show devices list' portion, I get right on to the CLI of the NSO instance no problem, from which I can manually issue the 'show devices list' command, but that obviously defeats the purpose of this in the first place, to use with scripts.

 

If I use NETCONF to add/del a device, this works exactly as before, so I know that it's just the SSH subsystem and not NSO overall that's changed. But I also don't like the idea of completely revamping my scripts to use NETCONF for every single command (or even if the same functionality is even possible), and when i just tried to downgrade back to 5.3.0.1 (and recompiling NEDs), the system is now throwing errors about a port already being open, so maybe the 5.4.1.1 upgrade installed some things that make a downgrade more problematic. I'm back on 5.4.1.1 now and the problem is still present.

 

I also tried rewriting the command as follows:

echo 'show devices list' | ssh admin@nso_server_ip -i /opt/admin/.ssh/admin-nso-key -p 2024 -o StrictHostKeyChecking=No

 

...which, interestingly enough, does work - but the downside is that it leaves the connection open and never closes it (i.e. the output comes properly but then the session remains stuck open). I've tried using "-t" with SSH and it doesn't help.

 

If someone could perhaps speak to whether or not the SSH subsystem was changed between 5.3.0.1 and 5.4.1.1, and if there are workarounds/new things one must do to leverage the SSH CLI interface in a different way, I would greatly appreciate it... otherwise, I'm looking at having to reinstall the whole NSO VM to get back to 5.3.0.1 and resume my normal operation (or, using NETCONF for everything, which I really really really really don't want to do!)...

 

Thanks!

1 Accepted Solution

Accepted Solutions

jobengts
Cisco Employee
Cisco Employee

This functionality is broken in the 5.4.1.1 release, with no known workarounds unfortunately. A fix has been identified and will be included in the next suitable release. Sorry for the inconvenience.

View solution in original post

7 Replies 7

jobengts
Cisco Employee
Cisco Employee

This functionality is broken in the 5.4.1.1 release, with no known workarounds unfortunately. A fix has been identified and will be included in the next suitable release. Sorry for the inconvenience.

have you tried doing

echo 'show device list\nexit'

to close the session?

maybe it works while waiting for the fix to NSO.

 

Stefano

Sorry, that won't work either. It comes back with "show device" and then a line saying "syntax error: unknown argument" - my guess is that passing escape characters to the echo inside of the ssh session doesn't work. Thanks for the suggestion, though.

 

I suppose I can see if there's a fix soon, in the meantime maybe I'll look to 5.4.1 or something to see if there's a retro fix. 5.4.1.1 was released on the internal servers on 1 October so I hope there's a new revision soon, that's almost two months ago. The docs for 5.5 are already posted internally.

 

Incidentally, is there a bug repo that anyone knows of inside or outside of Cisco? There's no apparent "Release Notes" for NSO that I can easily find, which is where I would normally go to track things like this (by CDETS bug ID or otherwise), but if there's a place to find this kind of thing I'd really love to know.

Hi,

yes there is a file called CHANGES in the top level of the NSO install where you see all changes and bugs and issues fixed. Similarly in each NED or function pack at the top level of the archive.

 

Stefano

FYI, a downgrade to 5.4.1 did not work. I didn't attempt 5.4.0 or any of its maintenance releases, but I did go for 5.3.3 and that works (after recompiling the NEDs, of course).

I did test my answer locally with ncs 5.3 and 5.4.1.1 two versions I had handy and got identical results. What |I did not do was set up pki, just used passwords, I'm wondering if there isn't an issue with shell and quotes. here is my terminal log

snovello@SNOVELLO-M-J0VB work % echo 'show devices list' | ssh -p2024 admin@localhost
Pseudo-terminal will not be allocated because stdin is not a terminal.
admin@localhost's password:
NAME  ADDRESS       DESCRIPTION  NED ID                ADMIN STATE
------------------------------------------------------------------
POD   10.48.59.234  -            cisco-apicdc-gen-3.7  unlocked
nx0   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
nx1   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
nx2   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
nx3   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
nx4   127.0.0.1     -            cisco-nx-cli-5.15     unlocked

^C%                                                                                                                 snovello@SNOVELLO-M-J0VB work % echo 'show devices list\nexit' | ssh -p2024 admin@localhost
Pseudo-terminal will not be allocated because stdin is not a terminal.
admin@localhost's password:
NAME  ADDRESS       DESCRIPTION  NED ID                ADMIN STATE
------------------------------------------------------------------
POD   10.48.59.234  -            cisco-apicdc-gen-3.7  unlocked
nx0   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
nx1   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
nx2   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
nx3   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
nx4   127.0.0.1     -            cisco-nx-cli-5.15     unlocked
snovello@SNOVELLO-M-J0VB work % 

it shows how ssh hangs when you don't  have the exit - and also shows how you get your prompt back when the exit is present.

Hmm I think my issue might have been using the keyword/command "device" instead of "devices" :). Thanks for the follow up. At least I'm able to continue to use my existing scripts now with a downgrade to 5.3.3, but if i have to go to 5.4 and this is still broken, I will experiment with the echo. I guess I could also look into leveraging NETCONF as well, which is more programmatic anyways.

 

Also, it would be great is the contents of those "CHANGES" or "README" files was online somewhere and easily searchable, I don't think it's really feasible to expect folks to download the file to see if a bug they've been encountering is fixed. Also, bugs affecting a particular release that are new probably aren't in their own CHANGES/README files. 

 

Thanks again!

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the NSO Developer community: