LMS 4.0 Issues

[Martin Smid]

Hi,

I was looking for some main topic which would have a list of issues so far found in LMS 4.0. Here are few first minor bumps on the road.

1) Missing Navigation Panel on several pages

2) Device configuration not being backed up even though all devices are supposed to be managed (LMS didn't even try to download the config. In the configuration overview there was just "Total 1" instead of 2.

I would like to add more and more issues into this thread or if one of the moderators could create a separate folder just for this kind of topics I think it would make the search and life much easier.

P.S.: Any hints or fixes are appreciated

Regards,

              Martin

[Nael Mohammad]

Just tested it out on our lab server the navigation pages appear to work fine and using Firefox 4.0 RC2. Verify that you did not have any installation issues during the setup and that you have added LMS server to your trusted sites in IE.

Post the installation logs if adding the lms server to your trusted site or using a different browser does not display the navigation headers. Logs should start with C:\Ciscoworks_install_date.logs or /var/tmp directory.

For problem 2:

Goto Inventory --> Job Browsers --> Device Credential Verification --> Create Job for devices in question.

Goto Inventory --> Job Browsers --> Inventory Collection ---> Create job for both polling and collection.

Get the logs from CSCOpx/log dcmaclient.log and dcmaservice.log along with the IP address or hostname of test device.

[Joe Clarke]

Thanks for the suggestion, Martin.  However, there is a difference between known issues, and localized problems.  I think you're seeing the latter, and those should be analyzed as new issues.  For future issues, please start new threads, so we can properly distinguish individual problems.

For the first issue, how did you reproduce this exactly?  Is it something you can reliably reproduce?  If so, try clearing your browser cache, and see if that corrects the problem.

For the second issue, more information is required.  What device type is not being archived?  If you go to Configuration > Configuration Archive > Summary, what do you see?  If you go to the Inventory > Device Status dashboard, what do you see in the Collection Summary portlet?

[Martin Smid]

Hi,

Thanks for the replies. The issue with disappearing navigation menu is fine now. (even though I didn't do anything, so I guess it was related to the browser).

The other issue is a bit strange. Devices are now flagged as managed, but LMS can't seem to be able to use any credentials. I mean it reports telnet credentials as incorrect. I have tried several working logins and I have tried to change the protocol usage as well (SSH, Telnet) and nothing seems to work. I remember that we had an issue like this back with 3.2 and it was related to the TACACS prompts. The strange thing is that our Cisco ACS server doesn't even pick up any login attempts. When I use Putty from the LMS server it works all fine (SSH, Telnet ... several different credentials).

I will try to check the TACACS prompts on the LMS server.

If anyone has any suggestions, please share them with me.

Regards,

          Martin

[Joe Clarke]

Well, if the username/password prompts are unknown to LMS, it won't even try to login.  It will just stare at the prompt.  You can add any prompt customization to NMSROOT/objects/cmf/data/TacacsPrompts.ini.

That said, if SSH is not working, I don't think this is your problem.  I would first go to Inventory > Device Administration > Add / Import / Manage Devices, and edit credentials to make absolutely sure you've entered the correct values.  Do not assume that just because • characters are there that the credentials are entered correctly.

If things still fail.  Go back to Reports > Inventory > Management Status > Credential Verification and run a report for telnet credentials.  At the same time, start a sniffer trace filtering on tcp/23 traffic to the test device.  Check the trace after the job fails to see what LMS is seeing and sending.

[Martin Smid]

Hi,

I have sorted out the TACACS prompts (or at least I would think so ). Here is what we have got in the TacacsPrompts.ini file

[TELNET]
USERNAME_PROMPT=username: ,Username: ,
PASSWORD_PROMPT=password: ,Password: ,

So here is what is happening right now. The archiving job fails and the reported reason is "Command failed". (See the attached screenshot "Job result")

I have done some digging and sometimes it might be an issue with job timeouts. The job timeouts have been extended. (See attached screenshots "Job timeout" and "Telnet timeout")

And what is really strange is that if I go into our ACS server and check AAA Accounting I can see the "copy vlan.dat" command being executed by the LMS server. The devices, which are currently in the LMS 4.0 are 3560, 3750 and also 6509.

Also, how does it work with support from Cisco? Do we need to use the forum or is there an official source for support (something like TAC for evaluation period)?

Many thanks for the help,

                                              Martin

[Martin Smid]

Hi,

So after packet sniffing and reconstructing the Telnet stream we can see, instead of pile of packets, the actual commands sent to the device and the replies sent to LMS server.

The IP addresses have been replaced and the hostname as well. You can see X --> Y and vice versa. So the first line says that device sent to LMS server info which you can see on the next line and so forth. The whole data stream has been cut to this most important part and in the end the device says that it cannot write into LMS' TFTP server. So at least we know what the problem is. Any reason why? ( I have checked the IPs before I replaced them and the device really is targeting LMS4.0 server)

Regards,

                 Martin

[Joe Clarke]

A permission denied error implies there was a problem writing to the NMSROOT/tftpboot directory.  LMS should have created the file in that directory prior to copying it, but check to make sure casuser has full control, and check the output of "netstat -a -n -o -b" to make sure it is crmtftp bound to udp/69.

[Martin Smid]

Hi Joe,

Please see the attached screenshot of the permissions. Nobody touched them so I guess it is setup the way LMS did during the install. Casuser is a member of casusers group and as you can see that has full control. In the top you can see crmtftp.exe is on UDP:69

Regards,

                 Martin

[Joe Clarke]

Yeah, this looks okay.  I wonder if perhaps the problem is being caused by some host-base IPS or firewall software.  Gather a new sniffer trace filtering on all UDP traffic between the device and the server when doing a config collection.  It looks like the first udp/69 packet is working, but the subsequent data stream may not be.

[Martin Smid]

Hi,

Sorry for the delay in reply, I got overwhelmed with lots of stuff in the last week. I have got the TFTP packet sniffing results. And LMS doesn't seem to send any packets back. See for yourselfs in the screenshot. We have also came across some issues with job statuses (another screenshot), that was "fixed" by rebooting the server (not just daemon manager service .. we did that and that didn't help).

The HW spec is based on Windows 2003 Server LMS4.0 license for 1500 devices (we currently have 3.2 for 1500 devices). The reason for this is that we wanted to move to LMS 4.0 asap, but as you can see we are kinda struggling

The AntiVirus is switched off on the LMS server and there is nothing on the network (security-wise) what could cause the issues. LMS 3.2 is doing just fine (using the same path and traversing the same devices as 4.0)

Regards,

                  Martin

[Joe Clarke]

This indicates that udp/69 is blocked between the device and the server.  Run the packet capture from the LMS server itself (LMS includes a built-in packet capture utility in Device Center).  If you see the udp/69 packets arriving on the LMS server, check to see if Windows Firewall is enabled.  If so, disable it.  If Windows Firewall is disabled, check for other security software on the server (IPS-like software).  Disable those suites and see if TFTP works.

[Martin Smid]

Hi,

I have looked for the packet capture utility and I can't find it anywhere, not in the user guides, not in the site map. All I found in the user guides is that LMS has packet capture utility in the Troubleshooting Tools.

Could you please tell me where I can find it?

Many thanks,

                    Martin

[yjdabear]

It's under Inventory > Tools > Device Center, according to page 372 of http://www.cisco.com/en/US/docs/net_mgmt/ciscoworks_lan_management_solution/4.0/user/guide/monitoring_troubleshooting/UserGuideforMNT.pdf

[Martin Smid]

I found the packet capture. To be honest I did not expect that an icon of a star is actually a menu button. Nevertheless here is the packet capture result.

LMS is basically saying that it cannot save the file. Anti virus software is disabled on the server (3.2 runs with it enabled no problem) and there is no security device in the way between LMS server and the device.

Martin