cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1336
Views
0
Helpful
16
Replies

DFM upgrade to 2.0.10 - email notification stopped working

ibc-tcs
Level 1
Level 1

After we completed the DFM upgrade from 2.0.6 to 2.0.10 on our LMS 2.6 bundle, the email notifications just stopped working. Fault History still shows the events or alerts are coming in but no emails are sent out.

Appreciate any help.

1 Accepted Solution

Accepted Solutions

Nevermind again. I found a bug. I can create a test patch for you to try out. You will need to open a TAC service request to get it, though. Have your engineer contact me directly for the patch.

View solution in original post

16 Replies 16

Joe Clarke
Cisco Employee
Cisco Employee

Please post a screenshot of your Notification Group configuration, the details of a new event which has been generated after the upgrade, and the output of the pdshow command.

I have attached the Notification Group config and one sample event (OperationallyDown port) on my test switch. I am not too sure about where to get the output of the pdshow, I just attached the running services. I apologize if that's not the info you needed. Please let me know how to get it.

Thanks again.

I finally was able to get the pdshow output. Pls see attached. Thanks.

I don't see any errors per se, but I cannot confirm that Event Set A has the OperationallyDown even selected nor that the device 10.109.128.254 is selected for notifications. Verify that those things are true. If so, post the NOSServer.log and NMSROOT\log\dfmLogs\NOS\nos.log.

I verified and both are true. Hoping to find out more info, I enabled "debug" in logging and then simulated the OperationallyDown event. Maybe you can find other details there. Also, I manually tested sending email by selecting the "Notify" in the Alert and Activities window. I did get the email so it appears only the auto email is not working.

Attached are the logs. Thanks again.

I saw this once before where the EPM database becomes out of sync with itself. New events never trigger notifications because the event IDs are not what is expected. Try reinitializing the EPM database alone. You will lose all your current events, but notification and fault history will remain:

NMSROOT\bin\perl NMSROOT\bin\dbRestoreOrig.pl dsn=dfmEpm dmprefix=EPM

Here's what I did as advised:

Stopped the Ciscowork Daemon Manager

C:\Program Files\CSCOpx\bin>net stop crmdmgtd

The CiscoWorks Daemon Manager service is stopping...............................

.........................................................

The CiscoWorks Daemon Manager service was stopped successfully.

Re-initialize EPM

C:\Program Files\CSCOpx\bin>perl dbRestoreOrig.pl dsn=dfmEpm dmprefix=EPM

WARNING: Existing contents of dfmEpm database will be lost.

Do you want to continue [y/n]?y

dfmEpm database initialization is completed.

C:\Program Files\CSCOpx\bin>net start crmdmgtd

The CiscoWorks Daemon Manager service is starting.

The CiscoWorks Daemon Manager service was started successfully.

Re-booted the server...

Simulated the event. The Alert and Activities panel logged the alert/event but same problem - no auto email..

I presume you left debugging on? If so, please post the new nos.log.

No not really. Anyway I re-enabled debug and simulated the event again. You can look at the attached NOS log beginng at 11-March-2008-15:58:05.

Thanks.

This doesn't help. I need to see the log from when NOSServer is first started. Keep debugging enabled, then stop dmgtd. Clear out nos.log, then restart. Then simulate the event, and post the new nos.log.

OK. Here's the NOServer log.

I will send the other nos log after the clear out and restart.

After the log was cleared and the server restarted, a strange thing happended - I suddenly got 3 email notifications(?).

To verify, I simulated the event again but no email came.... I thought for a momemnt it's working again.

Anyway, here's the log.

According to this, email should now be working:

11-Mar-2008|16:41:16.921|DEBUG|NOS|Email Notify Thread:Pooled Thread:1|EmailUtility|createAndSendMail()|.|

Mail was sent successfully to gromero@ibc.ca

I suggest you check your SMTP server (10.108.51.42) to make sure it is relaying this mail. It may also be getting filtered in SPAM. The contents should be:

EVENT ID = 00000RW

ALERT ID = 00000RT

TIME = Tue 11-Mar-2008 15:59:12 EDT

STATUS = Active

SEVERITY = Critical

MANAGED OBJECT = 10.109.128.254

MANAGED OBJECT TYPE = Switches and Hubs

EVENT DESCRIPTION = OperationallyDown::Component=PORT-10.109.128.254/0.2 [Fa0/2];Type=ETHERNETCSMACD;OperStatus=DOWN;InterfaceCode=CODEUNKNOWN;DuplexMode=FULLDUPLEX;AdminStatus=UP;LastChangedAt=11-Mar-2008 3:59:12 PM;IsFlapping=false;MaxSpeed=100000000;Mode=NORMAL;

CUSTOMER IDENTIFICATION = IBC

CUSTOMER REVISION = 3.0

Nevermind. I just reread what you typed. When did you simulate this latest event?

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco