12-20-2021 08:30 AM
I'm trying to get pyATS to work, but it won't connect to the devices in any way.
There are several vIOSs deployed in the EVE-NG lab and connection refused occurs when pyATS try to connect to my devices.
The hostname matches the hostname in the testbed.yaml file. Also I try to create new environment, but have the same result.
If you try to connect manually via ssh client it works, python/nornir scripts also work. But pyATS show me "ssh: connect to host port 22: Connection refused" for all atempts for all devices.
12-20-2021 11:53 AM - edited 12-20-2021 12:06 PM
Hello @vsfomin,
Here's something you can try to get more information about why SSH is failing...
Go to your YAML testbed file, select a device, and modify it as follows:
Replace
protocol: ssh
with
command: "ssh -v <device-ip-addr> -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null"
Replace <device-ip-addr> with the IP address of the device that was modified in the YAML testbed file.
Save your YAML testbed file changes, then bring up a Python interactive shell session and enter the following:
from pyats.topology import loader tb_file = <testbed-file-name> tb = loader.load(tb_file) dev = tb.devices["<device-name>"] dev.connect()
Replace <testbed-file-name> with the name of your YAML testbed file.
Replace <device-name> with the name of the device that you modified.
The output from the dev.connect() command should provide you with verbose output from the SSH -v command and provide a clue regarding why the connection request is being refused.
Please post the output from your "dev.connect()" attempt here if you require additional assistance working through this issue.
fjm
@ittybittypacket
12-20-2021 10:21 PM - edited 12-20-2021 10:41 PM
This is output what I have. I get this result whether I change protocol: ssh to command: ssh -v... or not.
Traceback (most recent call last): File "/home/vsfomin/python-scripts/pyats/test.py", line 3, in <module> tb = loader.load(tb_file) File "/home/vsfomin/.local/lib/python3.6/site-packages/pyats/topology/loader/__init__.py", line 44, in load return TestbedFileLoader(**init_kwargs).load(loadable, **kwargs) File "src/pyats/utils/yaml/loader.py", line 116, in pyats.utils.yaml.loader.Loader.load File "src/pyats/topology/loader/base.py", line 170, in pyats.topology.loader.base.TestbedFileLoader.load_arbitrary File "src/pyats/utils/yaml/loader.py", line 247, in pyats.utils.yaml.loader.Loader.load_arbitrary pyats.utils.yaml.exceptions.LoadError: Content of 'testbed1.yml' failed to load into a dict. Got: testbed1.yml
My testbed file is:
devices: SW-PD-01: connections: cli: ip: 10.0.240.171 command: "ssh -v 10.0.240.171 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null" credentials: default: password: secretpassword username: admin enable: password: '%ASK{}' os: iosxe type: iosxe SW-PD-02: connections: cli: ip: 10.0.240.172 command: "ssh -v 10.0.240.172 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null" credentials: default: password: secretpassword username: admin enable: password: '%ASK{}' os: iosxe type: iosxe SW-PD-03: connections: cli: ip: 10.0.240.173 command: "ssh -v 10.0.240.173 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null" credentials: default: password: secretpassword username: admin enable: password: cisco os: iosxe type: iosxe
my script test.py which I start to test if it works or not:
from pyats.topology import loader tb_file = "testbed1.yml" tb = loader.load(tb_file) dev = tb.devices["SW-PD-01"] dev.connect()
No any access-list on these devices.
12-20-2021 11:34 PM
Hello @vsfomin,
Thanks for the feedback.
I suspect there is an issue with the YAML file, but the issue is not obvious because I copied and pasted the information you provided into a file, launched a Python shell, and imported the testbed file without an issue -- no errors similar to what you show above.
The shell did ask for "enable passwords" for two of the devices, but did NOT generate errors, even when I attempted to connect to one of the devices.
Here's the output showing the Python session:
(tvenv) [fjm@myServer .tmp]$ python Python 3.8.0 (default, Apr 28 2020, 13:27:51) [GCC 9.3.1 20200408 (Red Hat 9.3.1-2)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> >>> from pyats.topology import loader >>> tb_file = "testbed1.yml" Enter enable password for device SW-PD-01: Enter enable password for device SW-PD-02: >>> dev = tb.devices["SW-PD-01"] >>> dev.connect() 2021-12-21 01:03:51,334: %UNICON-INFO: +++ SW-PD-01 logfile /tmp/SW-PD-01-cli-20211221T010351333.log +++ 2021-12-21 01:03:51,335: %UNICON-INFO: +++ Unicon plugin iosxe +++ OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 58: Applying options for * debug1: Connecting to 10.0.240.171 [10.0.240.171] port 22. 2021-12-21 01:03:51,349: %UNICON-INFO: +++ connection to spawn: ssh -l admin -v 10.0.240.171 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null, id: 139941292532256 +++ 2021-12-21 01:03:51,349: %UNICON-INFO: connection to SW-PD-01
This line in your output tells us that the YAML file wasn't loaded properly:
pyats.utils.yaml.exceptions.LoadError: Content of 'testbed1.yml' failed to load into a dict.
After loading the YAML file successfully, you should be able to "ask" Python to display the dictionary that is built from the data: tb.__dict__
Here's the dictionary that was built when I loaded your YAML file:
>>> tb.__dict__ {'name': 'testbed1', 'alias': 'testbed1', 'servers': AttrDict({}), 'custom': AttrDict({}), 'credentials': Credentials(NestedAttrDict({})), 'devices': TopologyDict({'SW-PD-01': <Device SW-PD-01 at 0x7f46a24794f0>, 'SW-PD-02': <Device SW-PD-02 at 0x7f46a2479400>, 'SW-PD-03': <Device SW-PD-03 at 0x7f46a2479700>}), 'tacacs': AttrDict({'login_prompt': 'login:', 'password_prompt': 'Password:', 'username': 'vsuri001'}), 'passwords': AttrDict({'tacacs': 'lab', 'enable': 'lab', 'line': 'lab', 'linux': 'lab'}), 'clean': AttrDict({}), 'testbed_file': 'testbed1.yml', 'ipv4_cache': <genie.conf.base.utils.IPv4InterfaceCache object at 0x7f46a2bce2e0>, 'ipv6_cache': <genie.conf.base.utils.IPv6InterfaceCache object at 0x7f46a2bb0790>, 'mac_cache': <genie.conf.base.utils.MACCache object at 0x7f46a2bce7c0>, 'raw_config': {'devices': {'SW-PD-01': {'connections': {'cli': {'ip': '10.0.240.171', 'command': 'ssh -v 10.0.240.171 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null'}}, 'credentials': {'default': {'password': 'secretpassword', 'username': 'admin'}, 'enable': {'password': '%ASK{}'}}, 'os': 'iosxe', 'type': 'iosxe'}, 'SW-PD-02': {'connections': {'cli': {'ip': '10.0.240.172', 'command': 'ssh -v 10.0.240.172 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null'}}, 'credentials': {'default': {'password': 'secretpassword', 'username': 'admin'}, 'enable': {'password': '%ASK{}'}}, 'os': 'iosxe', 'type': 'iosxe'}, 'SW-PD-03': {'connections': {'cli': {'ip': '10.0.240.173', 'command': 'ssh -v 10.0.240.173 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null'}}, 'credentials': {'default': {'password': 'secretpassword', 'username': 'admin'}, 'enable': {'password': 'cisco'}}, 'os': 'iosxe', 'type': 'iosxe'}}, 'testbed': {'testbed_file': 'testbed1.yml', 'name': 'testbed1'}, 'topology': {}}} >>>
You won't be able to attach to any devices until you can successfully load the YAML file.
What version of pyATS are you using? Would you please provide the output from this command? pyats version check
I'm running an older version of pyATS -- 21.6, so perhaps there's an issue w/ the version that you're using.
There is a Webex Space where you can get help directly from the development team.
Here's the URL to join the Webex space and raise your question.
12-20-2021 11:51 PM - edited 12-21-2021 12:08 AM
$ pyats version check
You are currently running pyATS version: 21.8.1
Python: 3.6.8 [64bit]
Package Version
---------------------------- -------
genie 21.8.2
genie.libs.clean 21.8
genie.libs.conf 21.8
genie.libs.filetransferutils 21.8
genie.libs.health 21.8.1
genie.libs.ops 21.8
genie.libs.parser 21.8
genie.libs.robot 21.8
genie.libs.sdk 21.8
genie.telemetry 21.8
genie.trafficgen 21.8.1
pyats 21.8.1
pyats.aereport 21.8.1
pyats.aetest 21.8.1
pyats.async 21.8.1
pyats.connections 21.8.1
pyats.contrib 21.8
pyats.datastructures 21.8.1
pyats.easypy 21.8.1
pyats.kleenex 21.8.1
pyats.log 21.8.1
pyats.reporter 21.8.1
pyats.results 21.8.1
pyats.robot 21.8.1
pyats.tcl 21.8.1
pyats.topology 21.8.1
pyats.utils 21.8.1
unicon 21.8
unicon.plugins 21.8
I've try to create new env and install pyats 21.6, copied testbed file and test.py script to new directory
2021-12-21 03:00:16,974: %UNICON-INFO: connection to SW-PD-01
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: Connecting to 10.0.240.171 [10.0.240.171] port 22.
debug1: connect to address 10.0.240.171 port 22: Connection refused
ssh: connect to host 10.0.240.171 port 22: Connection refused
Traceback (most recent call last):
File "src/unicon/statemachine/statemachine.py", line 727, in unicon.statemachine.statemachine.StateMachine.go_to
File "src/unicon/statemachine/statetransition.py", line 478, in unicon.statemachine.statetransition.AnyStateTransition.do_transitions
File "src/unicon/eal/dialogs.py", line 459, in unicon.eal.dialogs.Dialog.process
File "src/unicon/eal/dialog_processor.py", line 321, in unicon.eal.dialog_processor.SimpleDialogProcessor.process
File "src/unicon/eal/dialog_processor.py", line 235, in unicon.eal.dialog_processor.SimpleDialogProcessor.expect_eval_statements
File "/home/vsfomin/.local/lib/python3.6/site-packages/unicon/plugins/iosxe/cat9k/statements.py", line 29, in wrapper
return func(spawn)
File "/home/vsfomin/.local/lib/python3.6/site-packages/unicon/plugins/generic/statements.py", line 41, in connection_refused_handler
raise Exception('Connection refused to device %s' % (str(spawn)))
Exception: Connection refused to device spawn: ssh -l admin -v 10.0.240.171 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null, id: 140179752518880
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "src/unicon/bases/connection.py", line 757, in unicon.bases.connection.Connection.connect
File "src/unicon/bases/routers/connection_provider.py", line 190, in unicon.bases.routers.connection_provider.BaseSingleRpConnectionProvider.connect
File "src/unicon/bases/routers/connection_provider.py", line 217, in unicon.bases.routers.connection_provider.BaseSingleRpConnectionProvider.establish_connection
File "src/unicon/statemachine/statemachine.py", line 730, in unicon.statemachine.statemachine.StateMachine.go_to
unicon.core.errors.StateMachineError: Failed while bringing device to "any" state
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/vsfomin/python-scripts/pyats-21.6/test.py", line 5, in <module>
dev.connect()
File "src/pyats/connections/manager.py", line 453, in pyats.connections.manager.ConnectionManager.connect
File "src/unicon/bases/connection.py", line 763, in unicon.bases.connection.Connection.connect
unicon.core.errors.ConnectionError: failed to connect to SW-PD-01
Failed while bringing device to "any" state
12-21-2021 06:16 AM
Hello @vsfomin,
I'm glad to see that the older environment (21.6) loaded the testbed file and now produces the same result that you originally saw.
I checked w/ the folks in the Webex pyATS Community space regarding the failure to properly load the YAML file into the 21.8.1 environment, and the suggestion came back that you may have kicked off the Python session from a directory that did not include the YAML file.
To avoid an issue like this, I always call out the full pathname to the testbed file.
Back to the login failure...
Time to figure out why the device is not allowing you in.
Please turn on debugging for AAA Authentication and SSH at the device to determine why the device is not accepting the login.
debug aaa authentication
debug ip ssh detail
terminal monitor
Here's what I received on a successful login to an IOS-based device with the two debug commands on plus terminal monitor:
Dec 21 08:52:32.368 EST: SSH1: starting SSH control process
Dec 21 08:52:32.368 EST: SSH1: sent protocol version id SSH-2.0-Cisco-1.25
Dec 21 08:52:32.368 EST: SSH1: protocol version id is - SSH-2.0-Sun_SSH_1.1.8
Dec 21 08:52:32.368 EST: SSH2 1: Server certificate trustpoint not found. Skipping hostkey algo = x509v3-ssh-rsa
Dec 21 08:52:32.368 EST: SSH2 1: kexinit sent: hostkey algo = ssh-rsa
Dec 21 08:52:32.368 EST: SSH2 1: kexinit sent: encryption algo = aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
Dec 21 08:52:32.368 EST: SSH2 1: kexinit sent: mac algo = hmac-sha1,hmac-sha1-96
Dec 21 08:52:32.368 EST: SSH2 1: SSH2_MSG_KEXINIT sent
Dec 21 08:52:32.376 EST: SSH2 1: SSH2_MSG_KEXINIT received
Dec 21 08:52:32.376 EST: SSH2 1: kex: client->server enc:aes256-ctr mac:hmac-sha1
Dec 21 08:52:32.376 EST: SSH2 1: kex: server->client enc:aes256-ctr mac:hmac-sha1
Dec 21 08:52:32.376 EST: SSH2 1: Using hostkey algo = ssh-rsa
Dec 21 08:52:32.376 EST: SSH2 1: Using kex_algo = diffie-hellman-group-exchange-sha1
Dec 21 08:52:32.576 EST: SSH2 1: SSH2_MSG_KEX_DH_GEX_REQUEST received
Dec 21 08:52:32.576 EST: SSH2 1: Range sent by client is - 1024 < 4096 < 8192
Dec 21 08:52:32.576 EST: SSH2 1: Modulus size established : 4096 bits
Dec 21 08:52:32.952 EST: SSH2 1: expecting SSH2_MSG_KEX_DH_GEX_INIT
Dec 21 08:52:32.952 EST: SSH2 1: SSH2_MSG_KEXDH_INIT received
Dec 21 08:52:33.412 EST: SSH2: kex_derive_keys complete
Dec 21 08:52:33.412 EST: SSH2 1: SSH2_MSG_NEWKEYS sent
Dec 21 08:52:33.412 EST: SSH2 1: waiting for SSH2_MSG_NEWKEYS
Dec 21 08:52:33.420 EST: SSH2 1: SSH2_MSG_NEWKEYS received
Dec 21 08:52:33.620 EST: SSH2 1: Authentications that can continue = publickey,keyboard-interactive,password
Dec 21 08:52:33.620 EST: SSH2 1: Using method = none
Dec 21 08:52:33.620 EST: SSH2 1: Authentications that can continue = publickey,keyboard-interactive,password
Dec 21 08:52:33.784 EST: SSH2 1: Using method = keyboard-interactive
Dec 21 08:52:33.784 EST: AAA/BIND(0000002C): Bind i/f
Dec 21 08:52:33.784 EST: AAA/AUTHEN/LOGIN (0000002C): Pick method list 'default'
Dec 21 08:52:45.717 EST: SSH2 1: authentication successful for vsuri001
Dec 21 08:52:45.717 EST: SSH2 1: channel open request
Dec 21 08:52:45.717 EST: SSH2 1: pty-req request
Dec 21 08:52:45.717 EST: SSH2 1: setting TTY - requested: height 45, width 142; set: height 45, width 142
Dec 21 08:52:45.717 EST: SSH2 1: shell request
Dec 21 08:52:45.717 EST: SSH2 1: shell message received
Dec 21 08:52:45.717 EST: SSH2 1: starting shell for vty
router01-lab-03#show debug
General OS:
AAA Authentication debugging is on
SSH:
ssh detail messages debugging is on
Hopefully something obvious will pop up to tell you why you're being rejected.
fjm
@ittybittypacket
12-20-2021 12:04 PM
Hi @vsfomin,
One additional thought occurred to me after replying -- are there access lists on the devices that are preventing you from logging in from the host that you're running Python on?
fjm
@ittybittypacket
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide