New CMX on-prem won't connect to 9800-CL Wireless Controller

[klopez138]

Currently testing the new 9800-CL controllers and we've gotten an eval of the CMX on-prem appliance to replace our existing MSEs. I'm able to get CMX connected to our production AireOS controllers but there appears to be an SSH issue when trying to add the test 9800-CL controllers to CMX. I've tested SSH to other switches and routers from the CMX and it has no problems so I'm thinking there is a config that I'm missing on the 9800.There is full IP connectivity between the vWLC and the CMX as they are both at the same site behind the same firewall with no ACLs in between.

Screen caps and config examples are below so if there are any wireless savvy folks out there with experience with 9800s and CMX who may be able to help, it would be much appreciated. FYI, hostnames, IPs, usernames, and passwords have been obfuscated.

SSL errors in the 9800 controller logs:

Mar 19 17:40:53.976: %NMSP_SYSLOG-3-NMSP_SSL_ERROR_DISCONNECT: Chassis 1 R0/0: nmspd: nmspd TLS disconnection: [TLS local: XXX.XXX.35.33, remote: XXX.XXX.35.52] SSL accept failed with SSL error (code: 5, error:00000000:lib(0):func(0):reason(0)), closing connection
Mar 19 17:40:53.976: %NMSP_SYSLOG-5-NMSP_SSL_NOTICE: Chassis 1 R0/0: nmspd: nmspd TLS notice: [TLS local: XXX.XXX.35.33, remote: XXX.XXX.35.52] closing CMX connection over TLS protocol

[Rasika Nayanajith]

Hi 

I would check C9800 got proper configuration for SSH. Pls see below screen

If you can SSH to 9800, then you should be able to simply add it on CMX, refer below

https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/214729-configuring-and-troubleshooting-cmx-conn.html 

Then you should be able to see NMSP status by "show nmsp status" on your C9800.

HTH

Rasika

*** Pls rate all useful responses ***

[klopez138]

I should've mentioned that SSH to the vWLC from other devices works. I'm using smartcard authentication and password. It's only an issue when trying from the CMX CLI. my SSH config is below.

line vty 0 4
session-timeout 10
access-class 1 in vrf-also
exec-timeout 25 0
privilege level 15
authorization exec AAA
login authentication AAA
length 0
transport input ssh
line vty 5 15
session-timeout 10
access-class 1 in vrf-also
exec-timeout 25 0
privilege level 15
authorization commands 1 AAA
authorization commands 15 AAA
authorization exec AAA
login authentication AAA
transport input ssh

show ip ssh
SSH Enabled - version 2.0
Authentication methods:publickey,password,keyboard-interactive
Authentication Publickey Algorithms:x509v3-ssh-rsa
Hostkey Algorithms:x509v3-ssh-rsa,ssh-rsa
Encryption Algorithms:aes256-ctr,aes192-ctr,aes128-ctr
MAC Algorithms:hmac-sha1
KEX Algorithms:diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
Authentication timeout: 60 secs; Authentication retries: 3
Minimum expected Diffie Hellman key size : 2048 bits
IOS Keys in SECSH format(ssh-rsa, base64 encoded): SSH-RSA

[Rasika Nayanajith]

You can try manual config & see if that works

Step1: Get mac address &  SHA2 key has from your CMX

[cmxadmin@cmx ~]$ cmxctl config authinfo get

+-------------------+------------------------------------------+------------------------------------------------------------------+
| macAddress | keyHashString | sha2KeyHashString |
+-------------------+------------------------------------------+------------------------------------------------------------------+
| 00:0c:29:89:5b:26 | 6b045e761b112a3ff394c37ad0ffcc0c2750215f | 2faf935028816daf405b6d90aa8c112f5b25ea5a4f227767b90fa92099e3a6ba |
+-------------------+------------------------------------------+------------------------------------------------------------------+

Step2 : Configure NMSP on your 9800. 

nmsp enable
aaa attribute list NMSP_LIST
attribute type password 2FAF935028816DAF405B6D90AA8C112F5B25EA5A4F227767B90FA92099E3A6BA
username 000c29895b26 mac aaa attribute list NMSP_LIST

This post also talks about NMSP configs

https://mrncciew.com/2014/09/25/what-is-nmsp/ 

HTH

Rasika

*** Pls rate all useful responses ***

[klopez138]

Thanks for replying. I've already tried the configuration that you're recommending and it appears to be an ssh issue. When I try to add the controller in the CMX, it says "Unable to do SSH to the controller". See the screen cap. So this is what is leading me to believe that it's got something to do with the SSH config on the 9800. 

[klopez138]

FYI All, 

           I've opened a TAC case and it appears that there is a bug that we're hitting. Ill be sure to update this thread with the outcome. Thanks to all that chimed in to help. 

-Kevin

[cae_technology]

What was the bug/outcome?

[klopez138 · Accepted solution]

It ended up being an issue that we're using smartcard login for our cisco routing and switching gear. We had to change the WLC config to allow ssh-rsa and x509 algorithm and on the CMX under /etc/ssh_conf we changed the like HostKeyAlgorithms and it started working.