Can't add IWAN devices to APIC-EM

[MOSAADNVGL]

I'm trying to configure IWAN devices in APIC-EM and every time it fails giving me this message:

Failed to add the Device. Inventory service adding device has failed.. Failed to add device x.x.x.x

All the details are correct as we have devices in Prime infrastructure with the same snmp, CLI credentials

[rnaggi]

Mohamed ,

Could you pls

(A) let me know the APIC-EM Version

(B) device (router ) IOS-XE version

(C) Router ISR4x or ASR

(D) could you also check under Inventory UI , If the device is shown and if shown what is the status ? Is the status Partial Collection failure.

[MOSAADNVGL]

a. APIC-EM Version 1.2.0.1594.

b. IOS XE ver is csr1000v-universalk9.03.16.01a.S.155-3.S1a-ext.SPA.bin

c. The MC is CSR1000v and note that this also happens with our Border routers ISR4k.

d. Device was there and inventory was in progress since a week so I have deleted it before posting this question.

[MOSAADNVGL]

Now the status of the inventory is Managed after the removal

[rnaggi]

Could you try adding the device when the device is not listed under Inventory.Before adding , you could also check by logging to the device ( telnet or ssh ) from APIC-EM Cluster to make sure that you are able to access the device on network.

[MOSAADNVGL]

I'm sure that APIC-EM can reach the devices with no problems. Now I have deleted all 3 (MC, 2 BR) from inventory and gone directly to the IWAN HUB site configurations.

Added the 3 devices as per the attached picture and it is loading then I get the following:

Device x.x.x.x detected with partial information. Please verify that the CLI credentials or selected protocol is valid.

[rnaggi]

Are u using Telnet or SSH ?

delete the device from inventory UI and try out below.

Step 1 :make sure you have the license (advance enterprise , security-k9) and XE version ( XE 3.16.2 and above )

Step 2 : Make sure the router has these config .

Sample Config on router ::

Config term

username YYYY priv 15 passwd XXXX

snmp-server community public RO

snmp-server community private RW

line vty 0 15

login local

password XXXX

transport input telnet ssh

transport output telnet ssh

Step 3 : Login to APIC-EM Cluster using ssh grapevine@x.x.x.x and then login to the device for verification ( telnet or ssh whichever way you are adding devices ) --- > telnet to BR1 router

hope this helps.

[MOSAADNVGL]

I can do so but we are running 3.16.01 and all 3 devices the MC, 2 BRs, is it mandatory to upgrade the IOS XE to 3.16.2?

[rnaggi]

XE 3.16.2 is the supported version. It would be good to have the supported version to avoid issues.

did you try to login to these devices from APIC-EM cluster ?

I will suggest using 3.16.2 version on the routers. These images are verified. If you face the same issue with XE 3.16.2 , We can dig in more.

[MOSAADNVGL]

I will plan the IOS XE upgrade to 3.16.2 this weekend. I have tried the ssh from APIC-EM console to all devices and it works without any issue.

[MOSAADNVGL]

I have now completed the IOS XE update on all devices the Master controller (CSR1000v) and the two border routers ISR 4451 to version 3.16.3 as per your recommendations.

I have tried to add the devices as before in the IWAN wizard but it keeps spinning and then I get the following error.

[rnaggi]

I will be online for next 45 mins. Let me know if you face any issue

[MOSAADNVGL]

I'm using Radius for ssh and I have tried to login with that account in the AD and it works with level 15 privilege to the CLI.

What is RCA ? I don't know much about Linux, I need more details please.

[rnaggi]

RCA :: collects the log files into a TAR.gz format ( works like a winzip file :: all the log files are compressed and stored in a single file )

Here are the steps to collect the log files ::

step 1: Login to your APIC-EM cluster ssh grapevine@x.x.x.x

step 2 : from the console / shell ,type in

rca

password : yyyyy

Step 3: this process will start collecting all the log files.

Step4 : when the Log collection will complete, it should show you a successful mssg and a *.tar.gz file.

Step 5: upload the *.tar.gz file to your ftp or tftp server and share it with me.

[rnaggi]

After the upgrade on IOS-XE devices,

make sure that

Step 1 : Inventory UI deosnt have the Device listed. May be the device is still in inventory dbase with old IOS-XE information.

Pls delete the device from inventory UI

Step 2 : Make sure the router has these config .

Sample Config on router ::

Config term

username YYYY priv 15 passwd XXXX

snmp-server community public RO

snmp-server community private RW

line vty 0 15

login local

password XXXX

transport input telnet ssh

transport output telnet ssh

After step 2 , still you are having issues, then we should collect the logs and i can send it to dev team to take a look.

RCA log file can be collected by ssh to APIC-EM cluster using grapevine user --- > execute RCA from console --> type in the password --- > /tmp/*.gz file will be created. Send me the file and will analyze it for you.