ESA Upgrade 8.5 Reporting problem

[Tim Schneider]

Hello Community,

After upgrading our ESA from 8.0.x to 8.5x yesterday, our reporting malfunctions. It constantly shows "Data ist currently unavailable" and the GUI is slow as hell, with almost no CPU load (~6%).

I tried to rebuild the DB by using DELETEDB on the CLI, following this:

Choose the operation you want to perform:
- DELETEDB - Reinitialize the reporting database.
- DISABLE - Disable the reporting system.
[]> deletedb

This command will delete all reporting data and cannot be aborted. In some
instances it may take several minutes to complete. Please do not attempt a
system restart until the command has returned. Are you sure you want to
continue? [N]> y

Reseting reporting data......

And nothing happens. The SSH window isnt closing, but "Reseting reporting data...." is still there and nothing happens. Reporting is still not working.

I attached a screenshot of the GUI.

Any ideas how to resolve this?

[Mathew Huynh]

Hello Tim,

Thanks for the screenshot for the reporting concern.

From what I can tell; it looks like after you deletedb the data should have resetted to the time-frame where you have did the reset db.

Was the reset db done recently or on Feb 25th as per the screenshot.

But judging from the screenshot; possibly the reporting service on the back-end (process) may not be properly working or so if the reset db did not correct it.

Are you able to open a TAC case with us so that we can have a look for you and see what's happening here.

Thank you,

Matthew

[Tim Schneider]

Hello Matthew,

The DELETEDB command was made today, just about an hour before I created this thread.

I checked logs and the reporting log is "empty", meaning it didnt start a new instance after the upgrade-reboot.

These are the latest entries for reportd log:

Tue Mar  3 18:12:37 2015 Info: Begin Logfile
Tue Mar  3 18:12:37 2015 Info: Version: 8.0.1-023 SN: A4BADBE0B832-8Y7ST4J
Tue Mar  3 18:12:37 2015 Info: Time offset from UTC: 3600 seconds
Tue Mar  3 18:12:37 2015 Info: System is coming up.
Tue Mar  3 18:12:37 2015 Info: reportd is currently not enabled
Tue Mar  3 18:23:45 2015 Info: System is shutting down.
Tue Mar  3 18:23:45 2015 Info: End Logfile

And this is Reporting Query Logs:

Tue Mar  3 18:12:33 2015 Info: Begin Logfile
Tue Mar  3 18:12:33 2015 Info: Version: 8.0.1-023 SN: A4BADBE0B832-8Y7ST4J
Tue Mar  3 18:12:33 2015 Info: Time offset from UTC: 3600 seconds
Tue Mar  3 18:12:33 2015 Info: System is coming up.
Tue Mar  3 18:12:33 2015 Info: reportd is currently not enabled
Tue Mar  3 18:23:35 2015 Info: System is shutting down.
Tue Mar  3 18:23:35 2015 Info: End Logfile

As one can see, no new files have been started after the upgrade.

I assume it is as you said, the process might be malfunctioning. How can I fix that? Is there a (hidden) command on the ESA to show the status of the processes? I know that the WSA has something like this...

Currently I'm sadly unable to open a TAC case since I don't have the privilige to open one (internal rule).

When I turn off the reporting via "disable", the GUI is fast again when accessing the "Monitor" part.

Any help is much appreciated!

[Mathew Huynh · Accepted solution]

Hey Tim,

Sorry for not getting back to you sooner (From Australia so we're literally night and day different :D)

As per your follow up;

From what i can see the reportd process does seem to be playing up.

Unfortunately the ESA's processes to restart or restore them would require TAC access via the remote tunnel (service level access) to log in.

Else the alternative method would be to reboot the box.

From what I can see, the logs are showing 8.0.1; from the first post you stated you upgraded to 8.5+

Was this device also upgraded to 8.5+ ?

Regards,

Matthew

[Tim Schneider]

Hello Matthew,

No sweat, the ESA is working correctly, only reporting is borked.

I started the upgrade on March 3rd, right where the logging stops.

I disabled reporting prior to the upgrade to free memory, and proceeded with the upgrade.

After reboot I noticed the GUI was slow as hell and that, after enabling reporting, it didnt report like it should.

I will try to schedule a reboot ASAP, if this doesn't help I will talk to my superior to get the "go" for the TAC procedure.

Can you elaborate for me how the TAC tunneling is done? Is it enough if we perform a NAT statement which leads to the ESA?

Thanks in advance!

tim

[Ken Stieers]

You don't have to create a new NAT.

When you open the tunnel, the box creates an outbound connection to a system at Cisco TAC, and they connect to that connection and are required to authenticate.  IIRC its on port 25 and is encrypted.

Depending upon the version of ESA you're running you either set the password or it generates a key that you have to give TAC or they can't get in.

You can open/close that connection at your discretion.

[Tim Schneider]

Ah okay, thanks for the quick reply. That may be a big help for argumentation with my superior!

[Mathew Huynh]

Hey Tim,

Sorry for my late response as the time-zone difference is causing a delay for me to get back to you :)

As per your enquiry;

Typically the tunnel connection will go to 'upgrades.ironport.com' hostname which the tunnel server resides as well from memory.

It will use whichever port you decide to specify to establish connection, however it will request to use SSH protocol (for encrypted connection).

Once it establishes, we will be able to remotely SSH into your device via the tunnel connection and there the TAC engineer should review your system's reporting process and move forward to correct the process if they are being problematic.

I hope this helps.

Thanks,

Matthew

[Ken Stieers]

Are you using Intenet Explorer?? 10 or 11? In Compatibilit mode?

Go to System Administration/General Settings.... check the box.