Hi and thank you for replying

amgds · ‎12-17-2015

Hi all,

I'd installed CSM 3.0 to manage XR SMUs.

The installation was a success but software cannot connect with nodes but in server bash i don't have any problem to connect.

All python nodes were installed without errors.

I did a tcpdump in server and no packets was sent to the node. I don't use any jumpserver to connect to the nodes.

There are some problem in code or some miss config ?

The error message when "Check availability" in Edit Host is "Unable to reach the specified host or you are not an authorized user".

Thanks.

Eddie Chami · ‎12-18-2015

Your using telnet or SSH? Have you tried to switch between either?

amgds · ‎12-18-2015

None of then works.

I'd tried both without success.

No error logs generated.

Eddie Chami · ‎12-18-2015

drop me an email we can do a webex and i'll take a look.

MAURY MALONE · ‎12-29-2016

What ended up resolving this?

I'm having the same symptoms with an install of CSM 3.3.

The CLI only shows a single error regardless of trying to use 'telnet' or 'SSH'.

"Exception AttributeError: "'Connection' object has no attribute 'logger'" in <bound method Connection.__del__ of <condoor.Connection object at 0x7fd77b113390>> ignored"

Other than this things seem to be working fine on the server.

Running new Vagrant box of Ubuntu 14.04 and only installed modules required by this CSM 3.3. Chose not to run in virtualenv setup.

Thanks!

smailmilak · ‎02-04-2017

I have the same issue on two csm instances!

I hope that Eddie can help us out.

smailmilak · ‎02-05-2017

CSM 3.3 in my office is working with "telnet" but not "ssh".

CSM 3.3 on customers site is not working at all. I need this tool because the customer has a bunch of A9K and next week we will get 5 more. I have tried 3.2 (same issue as on 3.3) and 3.0 (internal error).

I would like to run this tool for the upcoming upgrades but this issue is getting me mad.

Here are the logs from the working (telnet only, though) CSM 3.3. SSH is not working because of this

'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -2 -p 22 cisco@10.5.10.168'

smail@smail-virtual-machine:/usr/local/csm/csmserver$ sudo ./csmserver start
starting CSM Server.....
LDAP authentication is not supported because it has not been installed.
smail@smail-virtual-machine:/usr/local/csm/csmserver$ [2017-02-05 22:05:00 +0000] [15064] [INFO] Starting gunicorn 19.2.1
[2017-02-05 22:05:00 +0000] [15064] [INFO] Listening at: http://0.0.0.0:5000 (15064)
[2017-02-05 22:05:00 +0000] [15064] [INFO] Using worker: sync
[2017-02-05 22:05:00 +0000] [15070] [INFO] Booting worker with pid: 15070
[2017-02-05 22:05:01 +0000] [15075] [INFO] Booting worker with pid: 15075
[2017-02-05 22:05:01 +0000] [15076] [INFO] Booting worker with pid: 15076
[2017-02-05 22:05:01 +0000] [15077] [INFO] Booting worker with pid: 15077
2017-02-05 22:06:13,025 INFO: [igwapolje]: Connecting to telnet://cisco@10.5.10.168:23 using generic driver
2017-02-05 22:06:13,025 DEBUG: [igwapolje]: [CTRL] Restarting from hop: 0
2017-02-05 22:06:13,026 DEBUG: [igwapolje]: [CTRL] [1] 10.5.10.168: Connecting. Attempt (1/1)
2017-02-05 22:06:13,026 DEBUG: [igwapolje]: [CTRL] [1] 10.5.10.168: Connecting to target device
2017-02-05 22:06:13,026 DEBUG: [igwapolje]: [TELNET]: Executing command: 'telnet 10.5.10.168 23'
2017-02-05 22:06:13,033 DEBUG: [igwapolje]: [TELNET]: Terminal window size: (24, 80)
2017-02-05 22:06:13,034 DEBUG: [igwapolje]: [TELNET]: Terminal window size changed to: (24, 160)
2017-02-05 22:06:13,034 DEBUG: [igwapolje]: [TELNET-CONNECT] FSM Started
2017-02-05 22:06:13,039 DEBUG: [igwapolje]: [TELNET-CONNECT] E=0,S=0,T=300,RT=0.00
2017-02-05 22:06:13,039 DEBUG: [igwapolje]: [TELNET-CONNECT] No action
2017-02-05 22:06:13,039 DEBUG: [igwapolje]: [TELNET-CONNECT] NS=1,NT=20
2017-02-05 22:06:13,261 DEBUG: [igwapolje]: [TELNET-CONNECT] E=3,S=1,T=20,RT=0.22
2017-02-05 22:06:13,261 DEBUG: [igwapolje]: [TELNET-CONNECT] A=save_pattern
2017-02-05 22:06:13,261 DEBUG: [igwapolje]: [TELNET-CONNECT] NS=-1,NT=20
2017-02-05 22:06:13,261 DEBUG: [igwapolje]: [TELNET-CONNECT] FSM finished at E=3,S=-1
2017-02-05 22:06:13,262 DEBUG: [igwapolje]: [TELNET]: EXPECTED_PROMPT=[\w\-]+[#>]
2017-02-05 22:06:15,264 DEBUG: [igwapolje]: [TELNET-AUTH] FSM Started
2017-02-05 22:06:15,265 DEBUG: [igwapolje]: [TELNET-AUTH] INIT_PATTERN=([U|u]sername:\\s|login:\\s?)
2017-02-05 22:06:15,265 DEBUG: [igwapolje]: [TELNET-AUTH] E=0,S=0,T=300,RT=0.00
2017-02-05 22:06:15,266 DEBUG: [igwapolje]: [TELNET-AUTH] A=send_username
2017-02-05 22:06:15,316 DEBUG: [igwapolje]: [TELNET-AUTH] NS=1,NT=10
2017-02-05 22:06:15,430 DEBUG: [igwapolje]: [TELNET-AUTH] E=1,S=1,T=10,RT=0.11
2017-02-05 22:06:15,430 DEBUG: [igwapolje]: [TELNET-AUTH] A=send_pass
2017-02-05 22:06:15,481 DEBUG: [igwapolje]: [TELNET-AUTH] NS=2,NT=20
2017-02-05 22:06:15,620 DEBUG: [igwapolje]: [TELNET-AUTH] E=2,S=2,T=20,RT=0.14
2017-02-05 22:06:15,620 DEBUG: [igwapolje]: [TELNET-AUTH] No action
2017-02-05 22:06:15,621 DEBUG: [igwapolje]: [TELNET-AUTH] NS=-1,NT=20
2017-02-05 22:06:15,621 DEBUG: [igwapolje]: [TELNET-AUTH] FSM finished at E=2,S=-1
2017-02-05 22:06:17,352 DEBUG: [igwapolje]: [TELNET]: Detecting prompt. Attempt (1/10)
2017-02-05 22:06:20,813 DEBUG: [igwapolje]: [TELNET]: LD=0,MP=4
2017-02-05 22:06:20,813 DEBUG: [igwapolje]: [TELNET]: Detected prompt: 'RP/0/0/CPU0:IGW_APOLJE#'
2017-02-05 22:06:20,814 DEBUG: [igwapolje]: [TELNET]: Compiled prompt: ''(\r\n|\n\r)RP\\/0\\/0\\/CPU0\\:IGW\\_APOLJE\\#'
2017-02-05 22:06:20,940 DEBUG: [igwapolje]: [CTRL] [1] 10.5.10.168: Connected successfully
2017-02-05 22:06:20,940 DEBUG: [igwapolje]: [CTRL] Connected target device
2017-02-05 22:06:20,940 INFO: [igwapolje]: Connected to telnet://cisco@10.5.10.168:23
2017-02-05 22:06:20,941 DEBUG: [igwapolje]: Sending command: 'terminal len 0'
2017-02-05 22:06:21,120 DEBUG: [igwapolje]: Waiting for prompt
2017-02-05 22:06:21,120 DEBUG: [igwapolje]: [WAIT-4-PROMPT] FSM Started
2017-02-05 22:06:21,240 DEBUG: [igwapolje]: [WAIT-4-PROMPT] E=4,S=0,T=60,RT=0.12
2017-02-05 22:06:21,240 DEBUG: [igwapolje]: [WAIT-4-PROMPT] A=_expected_prompt
2017-02-05 22:06:21,240 DEBUG: [igwapolje]: [CTRL] [1] 10.5.10.168: Updated target prompt: RP/0/0/CPU0:IGW_APOLJE#
2017-02-05 22:06:21,241 DEBUG: [igwapolje]: Mode: global
2017-02-05 22:06:21,241 DEBUG: [igwapolje]: Hostname detecting not implemented for generic driver
2017-02-05 22:06:21,241 DEBUG: [igwapolje]: [WAIT-4-PROMPT] NS=-1,NT=60
2017-02-05 22:06:21,241 DEBUG: [igwapolje]: [WAIT-4-PROMPT] FSM finished at E=4,S=-1
2017-02-05 22:06:21,242 INFO: [igwapolje]: Command executed successfully: 'terminal len 0'
2017-02-05 22:06:35,577 INFO: [igwapolje]: Connecting to ssh://cisco@10.5.10.168:22 using generic driver
2017-02-05 22:06:35,577 DEBUG: [igwapolje]: [CTRL] Restarting from hop: 0
2017-02-05 22:06:35,577 DEBUG: [igwapolje]: [CTRL] [1] 10.5.10.168: Connecting. Attempt (1/1)
2017-02-05 22:06:35,577 DEBUG: [igwapolje]: [CTRL] [1] 10.5.10.168: Connecting to target device
2017-02-05 22:06:35,578 DEBUG: [igwapolje]: [SSH]: Executing command: 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -2 -p 22 cisco@10.5.10.168'
2017-02-05 22:06:35,586 DEBUG: [igwapolje]: [SSH]: Terminal window size: (24, 80)
2017-02-05 22:06:35,587 DEBUG: [igwapolje]: [SSH]: Terminal window size changed to: (24, 160)
2017-02-05 22:06:35,587 DEBUG: [igwapolje]: [SSH-CONNECT] FSM Started
2017-02-05 22:06:35,662 ERROR: [igwapolje]: [CTRL] Error during connecting to device: Session closed unexpectedly

Nitin Pabbi · ‎02-05-2017

Below Error :-

2017-02-05 22:06:35,578 DEBUG: [igwapolje]: [SSH]: Executing command: 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -2 -p 22 cisco@10.5.10.168'

Is an indication that SSH is doing the strick check of host to allow the connection. Because CSM is script based and interactive session to accept the keys is not possible hence I would suggest you to change ssh / config file and explicitly allow your internal network for accepting the keys.

If you want to disable strict host key checking permanently in ssh ( since you have internal known users), you can use ssh configuration (i.e., ~/.ssh/config or /etc/ssh/ssh_config).

To turn off host key checking for all hosts you connect to:

Host *
StrictHostKeyChecking no

To avoid host key verification, and not use known_hosts file for 10.5.10.* subnet:

Host 10.5.10.*
StrictHostKeyChecking no
UserKnownHostsFile=/dev/null

Otherwise for single host you can also try below trick ex:-

add the following to either ~/.ssh/config or /etc/ssh/ssh_config.

Host 10.5.10.168
StrictHostKeyChecking no

Please check if this help you to solve the issue.

Thanks

Nitin Pabbi

smailmilak · ‎02-05-2017

Thanks,

I did edit the ssh_config file but it was still not working, but I have tried to connect to XRv from the Ubuntu VM and got this error:

Unable to negotiate with legacyhost: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1

A quick research showed me that I had to add this line:

Host somehost.example.org KexAlgorithms +diffie-hellman-group1-sha1

SSH is now working fine but the main issue is that csm on second VM (at customers site) is not working at all.

I get this error while testing reachability via telnet and ssh.

"Exception AttributeError: "'Connection' object has no attribute 'logger'" in <bound method Connection.__del__ of <condoor.Connection object at 0x7fd77b113390>> ignored"

Under /usr/local/csm_data I could not find any logs. I would really appreciate if somebody could help me out.

Should I try with Centos?

alextang · ‎02-06-2017

For the SSH key exchange issue, another way to resolve it is by manually issuing SSH to10.5.10.168 on the Linux server which hosts CSM. This will cause SSH to prompt for key exchange, answering 'Yes' will update the known_hosts file.

>>>I get this error while testing reachability via telnet and ssh.

"Exception AttributeError: "'Connection' object has no attribute 'logger'" in <bound method Connection.__del__ of <condoor.Connection object at 0x7fd77b113390>> ignored"

Can you send the complete trace? It should appear in Tools - System Logs after testing reachability if there is an exception.

smailmilak · ‎02-06-2017

Hi and thank you for replying. I have tried telnet and ssh (check reachability) and it this case no logs had been collected.

Only when I click on "Retrieve Latest Software" we get some logs.

Here is the trace:

Traceback (most recent call last):
File "/usr/local/csm/csmserver/work_units/inventory_work_unit.py", line 68, in start
handler_class = get_inventory_handler_class(ctx)
File "/usr/local/csm/csmserver/handlers/loader.py", line 55, in get_inventory_handler_class
discover_platform_info(ctx)
File "/usr/local/csm/csmserver/handlers/loader.py", line 33, in discover_platform_info
conn = condoor.Connection(name=ctx.hostname, urls=ctx.host_urls, log_level=logging.CRITICAL)
File "/usr/local/lib/python2.7/dist-packages/condoor/__init__.py", line 159, in __init__
nodes[index].append(make_hop_info_from_url(url))
File "/usr/local/lib/python2.7/dist-packages/condoor/hopinfo.py", line 81, in make_hop_info_from_url
parsed.port,
File "/usr/lib/python2.7/urlparse.py", line 113, in port
port = int(port, 10)
ValueError: invalid literal for int() with base 10: 'PASSWORD REMOVED BY ME'

alextang · ‎02-06-2017

What is the value given to the port number?

Is this CSM version download from CCO? Is it v3.3?

Can you provide me a MySQL dump? You can use this command

mysqldump csmdb -u root -p > backup.sql

After that, please attach backup.sql here.

alextang · ‎02-06-2017

The reason you see the exception is because the port number contains non-numeric value. It seems that you have 'PASSWORD REMOVED BY ME' as the device port number. Why is that?

MAURY MALONE · ‎02-06-2017

It looks to be related to CSM and how it stores/uses passwords with special characters.

I had the same trace as smailmilak but the 'PASSWORD REMOVED BY ME' was referencing my password not the port number. So I'm sure smailmilak was just scrubbing the trace logs before posting.

I noticed that the special character that my password ends with was missing from the trace. Tried another account that doesn't have a special character in it and everything started to work.

So a work around is now in place for me.

alextang · ‎02-06-2017

What is the special character in the password?

CSM 3.0 connectivity to nodes