topic Re: Updates From MC Not Taking On NIDS in Network Security

Updates From MC Not Taking On NIDS

ryan.brennan — Sun, 10 Mar 2019 09:13:22 GMT

We have 2 4235 NIDS devices that are run by a Security Monitor VMS server. Up until version S128 we were able to upgrade both devices through the MC, however this function has stopped working. The subsequent updates (S129 thru S135) appear to have worked, and even running a version report on the MC shows both sensors at S135. However when you manually telnet into each of the two sensors and do a show ver, both sensors are still back on the old S128 code. Any suggestions as to what has failed in the interim, or what service/process could have stopped/failed to stop updates between the Sec Mon and the sensors from happening?

Re: Updates From MC Not Taking On NIDS

scothrel — Wed, 05 Jan 2005 02:25:51 GMT

Could you post what version of MC you are running?

Thnx

Re: Updates From MC Not Taking On NIDS

jason.santamaria — Wed, 05 Jan 2005 15:50:03 GMT

I have exactly the same symptom. I have one 4235 one 4210 that I have been trying to get updated via IDS MC. While they appear to update to the latest signatures in the MC, telnetting to them or accessing them via IDM shows they are stuck on S123. I have a 4235 in a remote location that seems will show being updated to S91, but has no updates installed when I telnet to it or accessing it via IDM.

IDS MC Versions below:

Apache 1.3.27 12-26-2003 11:02:16 2 ENABLED

Auto Update Server 1.1 11-30-2004 10:58:03 none ENABLED

Client Application Manager 3.0 11-19-2003 16:17:51 none ENABLED

CWCS SQL Components 7.1.3 11-19-2003 16:17:51 none ENABLED

CiscoWorks Common Services with SP2 2.2 12-26-2003 11:02:16 2 ENABLED

Cisco Common Services Help 1.1 11-19-2003 16:17:51 none ENABLED

CWCS Foundation 2.2 12-26-2003 11:02:16 2 ENABLED

CWCS java2 engine 1.2 11-19-2003 16:17:51 none ENABLED

CWCS Web Desktop 2.2 12-26-2003 11:02:16 2 ENABLED

CWCS Utilities 1.1 12-26-2003 11:02:16 1 ENABLED

Database package 4.2 11-19-2003 16:17:52 none ENABLED

CiscoWorks Process Management package 3.5 11-19-2003 16:17:52 none ENABLED

CWCS Event Distribution System 3.2 11-19-2003 16:17:52 none ENABLED

Event Services Software 2.0 12-26-2003 11:02:16 1 ENABLED

Argent Grid classes 1.29 11-19-2003 16:17:52 none ENABLED

IDS MC/Security Monitor Common Framework 1.2 11-30-2004 11:23:49 1 ENABLED

IDS MC 1.2 11-30-2004 11:23:49 1 ENABLED

Security Monitor 1.2 11-30-2004 11:23:49 1 ENABLED

IpSecPole 1.22 11-19-2003 16:17:52 none ENABLED

Java SDK 1.3.1 12-26-2003 11:02:17 1 ENABLED

Jscape widget classes 1.1 11-19-2003 16:17:52 none ENABLED

JChart package 4.0.0.J 4.0 11-19-2003 16:17:52 none ENABLED

JDOM 1.0.7 11-19-2003 16:17:52 none ENABLED

Sun JRE Standard Extensions 1.0 11-19-2003 16:17:52 none ENABLED

Objectspace JGL classes 3.1 11-19-2003 16:17:52 none ENABLED

Jscape powersearch classes 1.1 11-19-2003 16:17:52 none ENABLED

Java Runtime Environment 1.2.2 2.2 11-19-2003 16:17:52 none ENABLED

Job and Resource Management Services 2.1 11-19-2003 16:17:52 none ENABLED

JRUN Servlet Engine 2.3.3 11-19-2003 16:17:52 none ENABLED

Log4j 1.01.03 11-19-2003 16:17:52 none ENABLED

LotusXSL for Java classes 0.16 11-19-2003 16:17:52 3 ENABLED

Application Administration Server 1.1 12-26-2003 11:02:17 2 ENABLED

CWCS Core 1.1 12-26-2003 11:02:17 2 ENABLED

NMCS Network Management Common Services 2.2 12-26-2003 11:02:17 1 ENABLED

nsdb 1.43 11-30-2004 09:30:59 none ENABLED

Perl package 5.00502.1 11-19-2003 16:17:52 none ENABLED

Management Center for Firewalls 1.1 11-30-2004 10:25:51 3 ENABLED

Java Plug-in 1.4.1_02 1.4 12-26-2003 11:02:17 1 ENABLED

Cisco Secure Post Office 1.0196 11-30-2004 09:30:59 none ENABLED

CWCS Help 2.2 12-26-2003 11:02:17 1 ENABLED

Java SNMP 2.6 11-19-2003 16:17:52 none ENABLED

Secure Shell Services 2.2 12-26-2003 11:02:17 1 ENABLED

Java Runtime Environment 1.3.1 11-19-2003 16:17:52 none ENABLED

Syslog, TFTP and RSH services 2.2 11-19-2003 16:17:52 none ENABLED

Sun JFC (Swing) Components 1.1 11-19-2003 16:17:52 none ENABLED

TomCat 3.3 12-26-2003 11:02:17 2 ENABLED

VisiBroker Orb 4.1 11-19-2003 16:17:52 none ENABLED

Web Server package 3.4 12-26-2003 11:02:17 1 ENABLED

Xalan 2.2 11-19-2003 16:17:52 none ENABLED

Xerces 1.5.1 11-19-2003 16:17:52 none ENABLED

IBM XML parser for Java classes 2.0.11 11-19-2003 16:17:52 none ENABLED

RunTime System package 3.2.2 12-26-2003 11:02:17 2 ENABLED

Re: Updates From MC Not Taking On NIDS

jason.santamaria — Wed, 05 Jan 2005 15:56:06 GMT

I have exactly the same symptom. I have one 4235 one 4210 that I have been trying to get updated via IDS MC. While they appear to update to the latest signatures in the MC, telnetting to them or accessing them via IDM shows they are stuck on S123. I have a 4235 in a remote location that will show being updated to S91, but has no updates installed when I telnet to it or accessing it via IDM.

IDS MC Versions below:

Apache 1.3.27 12-26-2003 11:02:16 2 ENABLED

Auto Update Server 1.1 11-30-2004 10:58:03 none ENABLED

Client Application Manager 3.0 11-19-2003 16:17:51 none ENABLED

CWCS SQL Components 7.1.3 11-19-2003 16:17:51 none ENABLED

CiscoWorks Common Services with SP2 2.2 12-26-2003 11:02:16 2 ENABLED

Cisco Common Services Help 1.1 11-19-2003 16:17:51 none ENABLED

CWCS Foundation 2.2 12-26-2003 11:02:16 2 ENABLED

CWCS java2 engine 1.2 11-19-2003 16:17:51 none ENABLED

CWCS Web Desktop 2.2 12-26-2003 11:02:16 2 ENABLED

CWCS Utilities 1.1 12-26-2003 11:02:16 1 ENABLED

Database package 4.2 11-19-2003 16:17:52 none ENABLED

CiscoWorks Process Management package 3.5 11-19-2003 16:17:52 none ENABLED

CWCS Event Distribution System 3.2 11-19-2003 16:17:52 none ENABLED

Event Services Software 2.0 12-26-2003 11:02:16 1 ENABLED

Argent Grid classes 1.29 11-19-2003 16:17:52 none ENABLED

IDS MC/Security Monitor Common Framework 1.2 11-30-2004 11:23:49 1 ENABLED

IDS MC 1.2 11-30-2004 11:23:49 1 ENABLED

Security Monitor 1.2 11-30-2004 11:23:49 1 ENABLED

IpSecPole 1.22 11-19-2003 16:17:52 none ENABLED

Java SDK 1.3.1 12-26-2003 11:02:17 1 ENABLED

Jscape widget classes 1.1 11-19-2003 16:17:52 none ENABLED

JChart package 4.0.0.J 4.0 11-19-2003 16:17:52 none ENABLED

JDOM 1.0.7 11-19-2003 16:17:52 none ENABLED

Sun JRE Standard Extensions 1.0 11-19-2003 16:17:52 none ENABLED

Objectspace JGL classes 3.1 11-19-2003 16:17:52 none ENABLED

Jscape powersearch classes 1.1 11-19-2003 16:17:52 none ENABLED

Java Runtime Environment 1.2.2 2.2 11-19-2003 16:17:52 none ENABLED

Job and Resource Management Services 2.1 11-19-2003 16:17:52 none ENABLED

JRUN Servlet Engine 2.3.3 11-19-2003 16:17:52 none ENABLED

Log4j 1.01.03 11-19-2003 16:17:52 none ENABLED

LotusXSL for Java classes 0.16 11-19-2003 16:17:52 3 ENABLED

Application Administration Server 1.1 12-26-2003 11:02:17 2 ENABLED

CWCS Core 1.1 12-26-2003 11:02:17 2 ENABLED

NMCS Network Management Common Services 2.2 12-26-2003 11:02:17 1 ENABLED

nsdb 1.43 11-30-2004 09:30:59 none ENABLED

Perl package 5.00502.1 11-19-2003 16:17:52 none ENABLED

Management Center for Firewalls 1.1 11-30-2004 10:25:51 3 ENABLED

Java Plug-in 1.4.1_02 1.4 12-26-2003 11:02:17 1 ENABLED

Cisco Secure Post Office 1.0196 11-30-2004 09:30:59 none ENABLED

CWCS Help 2.2 12-26-2003 11:02:17 1 ENABLED

Java SNMP 2.6 11-19-2003 16:17:52 none ENABLED

Secure Shell Services 2.2 12-26-2003 11:02:17 1 ENABLED

Java Runtime Environment 1.3.1 11-19-2003 16:17:52 none ENABLED

Syslog, TFTP and RSH services 2.2 11-19-2003 16:17:52 none ENABLED

Sun JFC (Swing) Components 1.1 11-19-2003 16:17:52 none ENABLED

TomCat 3.3 12-26-2003 11:02:17 2 ENABLED

VisiBroker Orb 4.1 11-19-2003 16:17:52 none ENABLED

Web Server package 3.4 12-26-2003 11:02:17 1 ENABLED

Xalan 2.2 11-19-2003 16:17:52 none ENABLED

Xerces 1.5.1 11-19-2003 16:17:52 none ENABLED

IBM XML parser for Java classes 2.0.11 11-19-2003 16:17:52 none ENABLED

RunTime System package 3.2.2 12-26-2003 11:02:17 2 ENABLED

Re: Updates From MC Not Taking On NIDS

ryan.brennan — Wed, 05 Jan 2005 17:24:37 GMT

I'm running version 1.2.

Thx.

Re: Updates From MC Not Taking On NIDS

scothrel — Wed, 05 Jan 2005 22:40:10 GMT

We have people looking into this from the sensor side and I will also forward the case to the MC folks in case they know something about MC issues. Could the folks on this thread let me know what version of the sensor software they have running?

We are also rechecking the last couple of signature updates to ensure compatibility with MC. This will take us a couple of days, as we had just moved our testing system to the MC 2.0 version.

Scott Cothrell

Re: Updates From MC Not Taking On NIDS

ryan.brennan — Thu, 06 Jan 2005 02:02:36 GMT

Per the 'show ver' on one of our sensors:

Cisco Systems Intrusion Detection Sensor, Version 4.1(4)S128

OS Version 2.4.18-5smpbigphys

Platform: IDS-4235

The S128 code is the last time we were to successfully push out the upgrade to our 2 sensors from the MC. We've installed S129 thru S136 (today) and it doesn not go through. When you run a sensor version report from the MC it says that both sensors are up to date. But it is my understanding that when this report is run it does not query the device to find it's version. It simply checks it's own database to see what sensors it has updated, and what release they were updated to.

Re: Updates From MC Not Taking On NIDS

scothrel — Fri, 07 Jan 2005 17:51:03 GMT

Ryan,

We re-verified signatures updates from S126 to S136 with no errors on MC ver. 1.2.3. After talking with the sensor and MC folks, we have two suggestions.

First, you can upgrade to 4.1.4f patch level (see http://www.cisco.com/cgi-bin/tablebuild.pl/ids-patches for information). The patch contains modifications to lower memory usage during updates. It is possible that sensors with lower total memory (4210 for example) and/or a lot of signatures enabled can run out of memory during a signature update.

Second, the MC folks suggest that "to troubleshoot the problem [from the MC side] they'll probably have to enable debugging to look at the CLI logs, so the best thing for these customers is to open a TAC case, so that a TAC engineer can walk them through the debug process..."

Re: Updates From MC Not Taking On NIDS

scothrel — Fri, 07 Jan 2005 17:54:28 GMT

Jason, see my reply further down in the thread. Same suggestions apply. Also, wrt the sensor showing S91. S91 was installed by the update to 4.1.4 from 4.1.3, the full package name is 4.1.4S91. So you get S91 included.

Re: Updates From MC Not Taking On NIDS

jason.santamaria — Fri, 07 Jan 2005 18:09:59 GMT

I have opened a TAC Case, and sent in the mc debugs for analysis. Waiting to hear back on Monday.

Re: Updates From MC Not Taking On NIDS

ryan.brennan — Fri, 07 Jan 2005 20:56:35 GMT

Thanks to Jason and Scott for the efforts thus far. I'll wait to see what comes of the TAC case. I'm upgrading to the 4.1(4)f and see what happens. Thanks again.

Re: Updates From MC Not Taking On NIDS

twilcox — Fri, 07 Jan 2005 20:57:33 GMT

We are having the exact same problem. But I think ours broke around S132 sig. update.

Queries from IDSMC to the NIDS work but pushes to the NIDS for new sig. updates don't work. Security Monitor is fine. The IDSMC actually recognizes the new sig updates if I do a query which ordinarily won't work if the MC itself hasn't been updated.

So I it's a bit baffling to say the least.

Re: Updates From MC Not Taking On NIDS

rpmanning — Mon, 10 Jan 2005 15:39:27 GMT

Any news on the TAC explanation? We had this issue, but it seemed that overwriting the S125 with two signature upgrades 'fixed' the problem. I have a before and after sh ver on what did not and what did work.

sh ver (not working)

* IDS-sig-4.1-4-S125 16:05:23 UTC Sat Nov 06 2004

IDS-sig-4.1-4-S134.rpm.pkg 15:52:10 UTC Wed Dec 20 2004

(but the signatures did not show in the sensor's config.)

!!!!!!!!!!!!!!!!!!!!

sh ver (this works)

* IDS-sig-4.1-4-S134 15:52:10 UTC Wed Dec 29 2004

IDS-sig-4.1-4-S135.rpm.pkg 18:49:04 UTC Tue Jan 04 2005

Re: Updates From MC Not Taking On NIDS

jason.santamaria — Mon, 10 Jan 2005 17:08:59 GMT

I am still working with TAC on the problem. I verified that I can updated the signatures on the NIDS directly, so it appears to be an MC problem rather than a sensor problem. As part of the process, I applied a couple of patches to the IDS MC, and now it no longer incorrectly reports patch levels. On the negative side, it doesn't push the updates out to my sensors either.

Re: Updates From MC Not Taking On NIDS

twilcox — Thu, 13 Jan 2005 22:01:57 GMT

Jason, did you receive any tip from TAC on fixing this issue; we're having the same problem as well. I upgrades all our IDS devices to the latest patch and our MC version is 1.2.3 , VMS 2.2 with all updated patches installed. We can update our NIDS directly and the MC seems to be able to query the latest version but it won't push updates out to the devices.

Re: Updates From MC Not Taking On NIDS

jason.santamaria — Fri, 14 Jan 2005 13:38:57 GMT

I got a couple of suggestions yesterday, but they did not work for me. I replaced the expired Cert on my MC, but that did not fix it. Also, my MC is not Dual homed, so the second suggestion really is not applicable. On the off chance that they work for you, I have pasted them below...

ADDITIONAL POSSIBLE CAUSES

CERT EXPIRED:

1. Stop CiscoWorks Daemon Manager (from Windows Services control panel) 2. Open DOS window and cd to $BASE\cscopx\mdc\apache\conf\ssl 3. create temp directory and copy all file to temp directory, there should be 4 files: openssl.conf, server.cert, server.csr, and server.key.

4. run "keytool -printcert -file server.cert" and capture output. This should show something like below

D:\Program Files\CSCOpx\MDC\Apache\conf\ssl>keytool -printcert -file server.cert

........

Valid from: Mon Oct 07 23:22:20 CDT 2002 until: Tue Oct 07 23:22:20 CDT 2003 Certificate fingerprints:

......

Note the expiration of "Tue Oct 07 23:22:20 CDT 2003"

5. run "..\..\gencert" you should see something like....

D:\Program Files\CSCOpx\MDC\Apache\conf\ssl>..\..\gencert

Loading 'screen' into random state - done Generating a 1024 bit RSA private key .................................................++++++

.....++++++

writing new private key to

'd:\PROGRA~1\CSCOpx\MDC\Apache\conf\ssl\server.key'

-----

unable to get 'req_attributes' section

problems making Certificate Request

Loading 'screen' into random state - done Signature ok .....

Getting Private key

6. run "keytool -printcert -file server.cert" again. Now you should see something like...

D:\Program Files\CSCOpx\MDC\Apache\conf\ssl>keytool -printcert -file server.cert

......

Valid from: Thu Nov 13 13:37:29 CST 2003 until: Fri Nov 12 13:37:29 CST 2004 Certificate fingerprints:

......

Note the expiration date "Fri Nov 12 13:37:29 CST 2004" (of course you will see a slightly different date, but at the least,a year ahead)

7. restart CiscoWorks Daemon Manager.

8. Now try upgradinng sensor

Another POSSIBLE ISSUE

MULTI HOMED VMS BOX

#######

Multi-Homed Machines

A multi-homed machine is a machine that has multiple NIC cards, each configured with different IP addresses. To run CiscoWorks Common Services on a multi-homed machine, there are two requirements.

* First, all IP addresses must be configured in DNS.

* Second, because of restrictions with CORBA, only one IP address can be used by the client / browser to access the server. You must select one IP address as the external address, with which the client will login to the CiscoWorks server.

To select an IP address, modify the gatekeeper file located in NMSROOT\lib\vbroker\gatekeeper.cfg. Replace every instance of external-IP-address with the external IP address you choose, and remove the "#" character, from the following:

* #vbroker.gatekeeper.backcompat.callback.host=external-IP-address

* #vbroker.se.exterior.host=external-IP-address

* #vbroker.se.iiop_tp.host=external-IP-address

* #vbroker.se.interior.host=external-IP-address

After modifying the gatekeeper file, restart the Daemon Manager by entering

net start crmdmgtd.

#######################

Re: Updates From MC Not Taking On NIDS

twilcox — Tue, 18 Jan 2005 21:35:06 GMT

Scott,

any news on this; we've had a TAC case open for about a month now and so far no results. Is the course of action to upgrade to MC 2.01 right now?

We are on MC 1.2.3 and can't do any upgrades; all our patches are currently. What happpened? Please advise.

Best regards,

Tom

Re: Updates From MC Not Taking On NIDS

ryan.brennan — Wed, 19 Jan 2005 00:58:45 GMT

I tried the new certificate route that was suggested here in this thread, as well as elsewhere. This did not work.

When querying the sensor it detects the version it's on (S128), however when you go to update it to the latest sig (S137) it responds as if you've already upgraded it and will not proceed.

Any idea on how to roll back an MC's database that houses the signature version table of the sensors it knows of?

Re: Updates From MC Not Taking On NIDS

scothrel — Wed, 19 Jan 2005 03:24:53 GMT

Tom,

Sorry to say that I come at this from the sensor side, MC is not a product I know much about. From what I have heard, upgrading to 2.01 is a good idea, as this looks to be an MC issue. It is my understanding that 2.01 does a better job of keeping the MC in sync with the sensor.

Brainstorming this: You might try setting up a 2.01 MC in parallel with your normal system and importing one of your "problem children sensors" into it to see what you get.

I'll have to reiterate, though, that I'm not an MC guy and I have no idea if what I proposed will screw things up more than they already are. YMMV.

I would also suggest that everyone reading this post, who has a similar situation, should open a TAC case. A flood of similar complaints should clue the TAC into the problem being more than a unique case.

Scott

Re: Updates From MC Not Taking On NIDS

bernhard — Wed, 19 Jan 2005 14:38:07 GMT

Here are some things to check that cause signature updates to fail.

1. Did the ip address of the IDSMC change since install?

The IDSMC has to tell the sensor where the update package lives.

Here's what to do when the IP address changes:

Stop daemon manager

Modify the following files:

CSCOpx/MDC/etc/ids/xml/SystemConfig.xml

Change value

Copy this file to CSCOpx/MDC/Tomcat/vms/ids-config/web-inf/classes/com/cisco/nm/mdc/ids/common/SystemConfig.xml

Copy this file to CSCOpx/MDC/Tomcat/vms/ids-monitor/web-inf/classes/com/cisco/nm/mdc/ids/common/SystemConfig.xml

CSCOpx/PostOffice/etc/routes

Change Host name and IP address

Restart daemon manager

NOTE: If you have any sensors in the system, you will need to modify each and every sensor. The Configuration->Settings->Communications->Remote Hosts page will need to be modified so that your new IP address is listed, instead of your old IP address.

2: Can the sensor/IDSM2 contact the IDSMC using HTTPS?

If the sensor is between a firewall and the VMS server, the address cannot be NAT'ed. The IDSMC ssh's into the devices and issues the upgrade command with the ip address of the VMS box. It uses the ipaddress of the VMS box from VMS perspective (this is a bug). Make sure the sensor is not NAT'ed. Also make sure that HTTPS traffic is enabled between the sensor and VMS. You can diagnose this problem by trying the upgrade manually.

First let's log onto the sensor.

type show settings

make sure all processes are running.

Then type

config t

tls trusted-host ip-address 10.10.111.222 port 443

upgrade https:///vms/sensorupdate/IDS-sig-4.1-1-S137.rpm.pkg

Use the IDS MC ip address instead of 10.10.111.222.

Use what ever the pkg name is if it is not as listed above.

You'll have to answer yes to some of the above commands.

3. Check the time on the sensor and VMS server

Another thing that can cause the certificate to be valid is the time.

Make sure the time set on the sensor is reasonably close to the time on the VMS server.

Certificates are bad and the hosts are not trusted if the time is off.