netconf device sync fromm get Protocol error

[sm000x]

Hi,

I have a strange issue and I cannot figure out what was wrong.

I have a netconf device but when I do sync-from I got error:
admin@ncs> request devices device zrdm60gcsmf01 sync-from
result false
info Failed to connect to device zrdm60gcsmf01: Protocol error
[ok][2024-02-18 13:33:20]
admin@ncs> exit

and the trace log size is 0:
-rw-rw-r--. 1 sm000x sm000x 0 Feb 18 13:33 netconf-zrdm60gcsmf01.trace

However, I am able to use
ssh -s sm000x@zrdm60gcsmf01-ip-addess -p 22 netconf

to invoke the netconf (Although it takes longer than usual time)

The device type is netconf:
admin@ncs> show configuration devices device zrdm60gcsmf01
address x.x.x.x;
port 22;
authgroup SMF;
device-type {
netconf {
ned-id smf-nc-1.0;
}
}
state {
admin-state unlocked;
}

I am able to do sync-from to other device with the same authgroup and ned-id:
dmin@ncs> request devices device z68bcsmf01 sync-from
result true
[ok][2024-02-18 13:49:42]

admin@ncs> show configuration devices device z68bcsmf01
address x.x.x.x;
port 22;
authgroup SMF;
device-type {
netconf {
ned-id smf-nc-1.0;
}
}
state {
admin-state unlocked;
}

Does anyone have same expierence?
What does "Protocol error" mean?

THX
sm000x

[sm000x]

Hi, cohult:

Thank you so very much for your input.
I will contact the device manager to investigate your suggestion.

THX
sm000x

[cohult]

Hi @sm000x,
As an example, the device manager (vendor A) can, for their OpenSSH server, separate the NETCONF port from the CLI port, with NETCONF on its IANA assigned port 830. Example device /etc/ssh/sshd_config:

Port 22
Port 830
...
Match LocalPort 830
    PasswordAuthentication yes
    AuthenticationMethods publickey password
    ForceCommand /path/to/bin/netconf-subsys
    ...
Match all
PasswordAuthentication no
AuthenticationMethods publickey keyboard-interactive
ForceCommand /path/to/bin/confd_cli
...
Subsystem	netconf    /path/to/bin/netconf-subsys

Another option is to run two sshd instances serving either the CLI or NETCONF on separate ports.

[sm000x]

Hi, cohult:

Thank you so much for this information. I will forward to the device vendor and the device manager.

Thank you
sm000x

[sm000x]

Hi,

If the keyboard-interactive authentication type is enabled for the device SSH server, try disabling it.
sm000x: I do not know how to check and how do disable it.

Here is the -vv ssh:

[root@mtnjdslncs06 ~]# ssh -vv sm000x@166.193.252.44
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 58: Applying options for *
debug2: resolving "166.193.252.44" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 166.193.252.44 [166.193.252.44] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.8
debug1: match: OpenSSH_8.8 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 166.193.252.44:22 as 'sm000x'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-ed25519,ecdsa-sha2-nistp256,ssh-rsa
debug2: ciphers ctos: aes256-ctr,aes192-ctr,aes128-ctr
debug2: ciphers stoc: aes256-ctr,aes192-ctr,aes128-ctr
debug2: MACs ctos: hmac-sha2-512,hmac-sha2-256
debug2: MACs stoc: hmac-sha2-512,hmac-sha2-256
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: kex: curve25519-sha256@libssh.org need=32 dh_need=32
debug1: kex: curve25519-sha256@libssh.org need=32 dh_need=32
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:yv7ZAEJqIeK6M/iDvyJrRLhsa7JXTKN4THFkWISsbWI
The authenticity of host '166.193.252.44 (166.193.252.44)' can't be established.
ECDSA key fingerprint is SHA256:yv7ZAEJqIeK6M/iDvyJrRLhsa7JXTKN4THFkWISsbWI.
ECDSA key fingerprint is MD5:97:96:a6:de:eb:bc:c5:8f:cc:ea:d9:fb:c3:ca:97:98.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '166.193.252.44' (ECDSA) to the list of known hosts.
debug2: set_newkeys: mode 1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 4294967296 blocks
debug2: key: /root/.ssh/id_rsa ((nil))
debug2: key: /root/.ssh/id_dsa ((nil))
debug2: key: /root/.ssh/id_ecdsa ((nil))
debug2: key: /root/.ssh/id_ed25519 ((nil))
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ed25519
debug2: we did not send a packet, disable method
debug1: Next authentication method: password
sm000x@166.193.252.44's password:


Thx
sm000x

[Nabsch]

Hello,

Regarding the trace, maybe you need to enable them.

devices device <device_name> trace raw
commit dry-run
commit
devices device <device_name> connect

Can you also execute the following  command and post the output

devices device <device_name> connect | detail debug
#sync-from but if it's fails for the connect it will also  fail for the sync-from
devices device <device_name> sync-from | detail debug

Regarding the other devices  , are they using the same version ? Same config ssh/netconf config ?

[sm000x]

Hi, Nabsch:

admin@ncs% set devices device zrdm60gcsmf01 trace raw
[ok][2024-02-20 17:54:32]

[edit]
admin@ncs% commit dry-run
cli {
local-node {
data devices {
device zrdm60gcsmf01 {
- trace pretty;
+ trace raw;
}
}
}
}
[ok][2024-02-20 17:54:38]

[edit]
admin@ncs% commit
Commit complete.
[ok][2024-02-20 17:54:50]

[edit]
admin@ncs% request devices device zrdm60gcsmf01 connect
result false
info Failed to connect to device zrdm60gcsmf01: Protocol error
[ok][2024-02-20 17:55:19]

[edit]
admin@ncs%

[sm000x@mtnjdslncs06 logs]$ ls -l *zrdm60gcsmf01*
-rw-rw-r--. 1 sm000x sm000x 0 Feb 18 13:33 netconf-zrdm60gcsmf01.trace
[sm000x@mtnjdslncs06 logs]

THX
sm000x

[sm000x]

Hi, Nabsch:

admin@ncs> request devices device zrdm60gcsmf01 connect | details debug
2024-02-20T18:01:08.448 device zrdm60gcsmf01: connect...
2024-02-20T18:01:08.448 device zrdm60gcsmf01: SSH connecting to admin@zrdm60gcsmf01
2024-02-20T18:01:15.046 device zrdm60gcsmf01: connect: error (6.598 s)
result false
info Failed to connect to device zrdm60gcsmf01: Protocol error
[ok][2024-02-20 18:01:15]


admin@ncs> show configuration devices device zrdm60gcsmf01
address x.x.x.x;
port 22;
authgroup SMF;
device-type {
netconf {
ned-id smf-nc-1.0;
}
}
trace raw;
state {
admin-state unlocked;
}
[ok][2024-02-20 18:02:19]
admin@ncs> show configuration devices authgroups group SMF
umap admin {
remote-name sm000x;
remote-password $9$n1M6TW3Rwz6ZBB9BsLs3zBDo43EO8QJS2QQ2vLdxzTo=;
}
[ok][2024-02-20 18:02:31]
admin@ncs>

For other device with same auth group:
admin@ncs> show configuration devices device z68bcsmf01
address y.y.y.y;
port 22;
authgroup SMF;
device-type {
netconf {
ned-id smf-nc-1.0;
}
}
state {
admin-state unlocked;
}

admin@ncs> request devices device z68bcsmf01 connect | details debug
2024-02-20T18:07:53.924 device z68bcsmf01: connect...
2024-02-20T18:07:53.924 device z68bcsmf01: SSH connecting to admin@z68bcsmf01
2024-02-20T18:07:55.465 device z68bcsmf01: reuse SSH connection to admin@z68bcsmf01
2024-02-20T18:07:56.502 device z68bcsmf01: connect: ok (2.578 s)
result true
info (admin) Connected to z68bcsmf01 - y.y.y.y
[ok][2024-02-20 18:07:56]

THX
sm000x

[Nabsch]

Regarding the other devices  , are they using the same version ( device os version) ? Same config ssh/netconf config ( i mean device config) ?

[sm000x]

Hi, Nabsch:

Regarding the other devices , are they using the same version ( device os version) ? Same config ssh/netconf config ( i mean device config) ?
sm000x: No, the other device is older OS version. The device in question is the newer version. Therefore, ssh/netconf might or might not be the same.

THX
sm000x

[Nabsch]

Can you switch to netconf ned & check if the connect is working ? ( IF you have no services on it otherwise create a new device that use the previous ip 

devices device <device-name> device-type netconf ned-id netconf

[sm000x]

Hi, Nabsch:

admin@ncs% set devices device zrdm60gcsmf01 device-type netconf ned-id netconf
[ok][2024-02-21 08:41:14]

[edit]
admin@ncs% commit
Commit complete.
[ok][2024-02-21 08:41:17]

[edit]
admin@ncs% exit
[ok][2024-02-21 08:41:20]
admin@ncs> request devices device zrdm60gcsmf01 ssh fetch-host-keys
result unchanged
fingerprint {
algorithm ssh-ed25519
value 77:a5:d4:e5:d0:1c:ca:18:d4:e0:36:f4:d6:7b:b0:b1
}
fingerprint {
algorithm ecdsa-256
value a0:a9:c8:4c:37:37:de:a5:5d:fa:0a:ca:f7:70:71:df
}
fingerprint {
algorithm ssh-rsa
value 1f:bb:4a:95:93:9e:aa:46:f3:44:d4:6c:d1:ac:65:1d
}
[ok][2024-02-21 08:41:33]
admin@ncs> request devices device zrdm60gcsmf01 connect | details debug
2024-02-21T08:41:46.288 device zrdm60gcsmf01: connect...
2024-02-21T08:41:46.288 device zrdm60gcsmf01: SSH connecting to admin@zrdm60gcsmf01
2024-02-21T08:41:52.837 device zrdm60gcsmf01: connect: error (6.548 s)
result false
info Failed to connect to device zrdm60gcsmf01: Protocol error
[ok][2024-02-21 08:41:52]
admin@ncs>

THX
sm000x