cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2311
Views
0
Helpful
11
Replies

WBRS updates fails - reverting to factory defaults

esuljevic
Level 1
Level 1

Hi,

For the past two weeks we are experiencing problems with wbrs updates, as they keep failing and wbrs engine update keeps rolling back and reverting to factory due to unexpected failure.

Updates are set to 8h for wbrs and 3 h to all others. Updates keep failing even when device is not under load, like in 00:00 where are no users online. Also same things are happening with our virtual S100V, and updates of wbrs fails even machine is not under load, because no users are redirected to this machine.

What is interesting is that wbrs seems to fail when wbrs starts to applying component updates, because this operation takes to long, and after that wbrs fails.

Sample of update logs:

Wed May 25 05:55:42 2016 Info: wbrs beginning download of remote file "http://updates.ironport.com/wbrs/3.0.0/categories/v2/1424380693"
Wed May 25 05:55:42 2016 Info: wbrs released download lock
Wed May 25 05:55:42 2016 Info: wbrs successfully downloaded file "wbrs/3.0.0/categories/v2/1424380693"
Wed May 25 05:55:42 2016 Info: wbrs started applying files
Wed May 25 05:55:42 2016 Info: wbrs started applying files
Wed May 25 05:55:42 2016 Info: wbrs applying engine update
Wed May 25 05:56:34 2016 Info: wbrs applying component updates
Wed May 25 07:41:49 2016 Info: Starting scheduled update

Wed May 25 07:41:51 2016 Info: Acquired server manifest, starting update 79
Wed May 25 07:41:51 2016 Info: Server manifest specified an update for sophos
Wed May 25 07:41:51 2016 Info: sophos was signalled to start a new update
Wed May 25 07:41:51 2016 Info: sophos processing files from the server manifest
Wed May 25 07:41:51 2016 Info: sophos started downloading files
Wed May 25 07:41:51 2016 Info: sophos waiting on download lock
Wed May 25 07:41:51 2016 Info: sophos acquired download lock
Wed May 25 07:41:51 2016 Info: sophos beginning download of remote file "http://updates.ironport.com/sophos/ide/1464149401"
Wed May 25 07:41:51 2016 Info: Server manifest specified an update for merlin
Wed May 25 07:41:51 2016 Info: merlin was signalled to start a new update
Wed May 25 07:41:51 2016 Info: merlin processing files from the server manifest
Wed May 25 07:41:52 2016 Info: merlin started downloading files
Wed May 25 07:41:52 2016 Info: merlin waiting on download lock
Wed May 25 07:41:52 2016 Info: merlin acquired download lock
Wed May 25 07:41:52 2016 Info: merlin beginning download of remote file "http://updates.ironport.com/merlin/masters-2.1/18076"
Wed May 25 07:41:52 2016 Info: Scheduled next update to occur at Wed May 25 10:41:52 2016
Wed May 25 07:42:17 2016 Info: sophos released download lock
Wed May 25 07:42:17 2016 Info: sophos successfully downloaded file "sophos/ide/1464149401"
Wed May 25 07:42:17 2016 Info: sophos started applying files
Wed May 25 07:42:17 2016 Info: sophos applying file "sophos/ide/1464149401"
Wed May 25 07:42:18 2016 Info: sophos restarting sophos
Wed May 25 07:42:18 2016 Info: sophos verifying applied files
Wed May 25 07:42:18 2016 Info: sophos updating the client manifest
Wed May 25 07:42:18 2016 Info: sophos update completed
Wed May 25 07:42:18 2016 Info: sophos waiting for new updates
Wed May 25 07:42:43 2016 Info: merlin released download lock
Wed May 25 07:42:43 2016 Info: merlin successfully downloaded file "merlin/masters-2.1/18076"
Wed May 25 07:42:43 2016 Info: merlin started applying files
Wed May 25 07:42:43 2016 Info: merlin applying file "merlin/masters-2.1/18076"
Wed May 25 07:42:43 2016 Info: merlin restarting merlin
Wed May 25 07:42:43 2016 Info: merlin verifying applied files
Wed May 25 07:43:43 2016 Info: merlin Starting heath monitoring
Wed May 25 07:43:43 2016 Info: merlin Initiating health check
Wed May 25 07:43:44 2016 Info: merlin Healthy
Wed May 25 07:43:54 2016 Info: merlin Initiating health check
Wed May 25 07:43:55 2016 Info: merlin Healthy
Wed May 25 07:44:05 2016 Info: merlin Initiating health check
Wed May 25 07:44:06 2016 Info: merlin Healthy
Wed May 25 07:44:16 2016 Info: merlin Initiating health check
Wed May 25 07:44:17 2016 Info: merlin Healthy
Wed May 25 07:44:27 2016 Info: merlin Initiating health check
Wed May 25 07:44:27 2016 Info: merlin Healthy
Wed May 25 07:44:37 2016 Info: merlin Initiating health check
Wed May 25 07:44:38 2016 Info: merlin Healthy
Wed May 25 07:44:48 2016 Info: merlin Initiating health check
Wed May 25 07:44:49 2016 Info: merlin Healthy
Wed May 25 07:44:59 2016 Info: merlin Initiating health check
Wed May 25 07:45:00 2016 Info: merlin Healthy
Wed May 25 07:45:10 2016 Info: merlin Initiating health check
Wed May 25 07:45:11 2016 Info: merlin Healthy
Wed May 25 07:45:21 2016 Info: merlin Initiating health check
Wed May 25 07:45:21 2016 Info: merlin Healthy
Wed May 25 07:45:31 2016 Info: merlin Initiating health check
Wed May 25 07:45:32 2016 Info: merlin Healthy
Wed May 25 07:45:42 2016 Info: merlin Initiating health check
Wed May 25 07:45:43 2016 Info: merlin Healthy
Wed May 25 07:45:53 2016 Info: merlin Initiating health check
Wed May 25 07:45:54 2016 Info: merlin Healthy
Wed May 25 07:46:04 2016 Info: merlin Initiating health check
Wed May 25 07:46:04 2016 Info: merlin Healthy
Wed May 25 07:46:14 2016 Info: merlin Initiating health check
Wed May 25 07:46:15 2016 Info: merlin Healthy
Wed May 25 07:46:25 2016 Info: merlin Initiating health check
Wed May 25 07:46:27 2016 Info: merlin Healthy
Wed May 25 07:46:37 2016 Info: merlin Initiating health check
Wed May 25 07:46:37 2016 Info: merlin Healthy
Wed May 25 07:46:47 2016 Info: merlin Initiating health check
Wed May 25 07:46:48 2016 Info: merlin Healthy
Wed May 25 07:46:58 2016 Info: merlin Initiating health check
Wed May 25 07:46:59 2016 Info: merlin Healthy
Wed May 25 07:47:09 2016 Info: merlin Initiating health check
Wed May 25 07:47:10 2016 Info: merlin Healthy
Wed May 25 07:47:20 2016 Info: merlin Initiating health check
Wed May 25 07:47:21 2016 Info: merlin Healthy
Wed May 25 07:47:31 2016 Info: merlin Initiating health check
Wed May 25 07:47:32 2016 Info: merlin Healthy
Wed May 25 07:47:42 2016 Info: merlin Initiating health check
Wed May 25 07:47:42 2016 Info: merlin Healthy
Wed May 25 07:47:52 2016 Info: merlin Initiating health check
Wed May 25 07:47:53 2016 Info: merlin Healthy
Wed May 25 07:48:03 2016 Info: merlin Initiating health check
Wed May 25 07:48:04 2016 Info: merlin Healthy
Wed May 25 07:48:14 2016 Info: merlin Initiating health check
Wed May 25 07:48:15 2016 Info: merlin Healthy
Wed May 25 07:48:25 2016 Info: merlin Initiating health check
Wed May 25 07:48:26 2016 Info: merlin Healthy
Wed May 25 07:48:36 2016 Info: merlin Initiating health check
Wed May 25 07:48:37 2016 Info: merlin Healthy
Wed May 25 07:48:47 2016 Info: merlin updating the client manifest
Wed May 25 07:48:47 2016 Info: merlin update completed
Wed May 25 07:48:47 2016 Info: merlin waiting for new updates
Wed May 25 09:56:45 2016 Info: wbrs rolling back engine update
Wed May 25 09:56:46 2016 Info: wbrs Reverting to factory due to unexpected update failure
Wed May 25 09:56:46 2016 Info: wbrs restarting wbrsd
Wed May 25 09:56:51 2016 Info: Received remote command to reset the client manifest
Wed May 25 09:56:51 2016 Info: internal_cert updater shutdown complete
Wed May 25 09:56:51 2016 Info: wbrs stopped the update handler
Wed May 25 09:56:51 2016 Info: wbrs updater shutdown complete
Wed May 25 09:56:51 2016 Info: merlin updater shutdown complete
Wed May 25 09:56:51 2016 Info: timezones updater shutdown complete
Wed May 25 09:56:51 2016 Info: sophos updater shutdown complete
Wed May 25 09:56:51 2016 Info: firestone updater shutdown complete
Wed May 25 09:56:51 2016 Info: cato updater shutdown complete
Wed May 25 09:56:51 2016 Info: mcafee updater shutdown complete
Wed May 25 09:56:51 2016 Info: amp updater shutdown complete
Wed May 25 09:56:51 2016 Info: trusted_root updater shutdown complete
Wed May 25 09:56:51 2016 Info: openssh_key updater shutdown complete
Wed May 25 09:56:51 2016 Info: trafmon updater shutdown complete
Wed May 25 09:56:51 2016 Info: avc updater shutdown complete
Wed May 25 09:56:51 2016 Info: Stopped the updates handler; no new updates will occur
Wed May 25 09:56:51 2016 Info: Loaded client manifest
Wed May 25 09:56:51 2016 Info: internal_cert waiting for new updates
Wed May 25 09:56:51 2016 Info: wbrs waiting for new updates

We opened support case, we were pointed to this bug (https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuy64732), bu I dont see any correlation to disk utilization because this problem is evident even when no load on device or virtual machine.

Anybody with similar problems?

PS: We are running 9.0.1-162 (WSA S170 and WSAv), but problem was also on 7.7.0-761(we upgraded to 9.0.1-162).

Thanks,

Edin

11 Replies 11

Atazazuddin Shaikh
Cisco Employee
Cisco Employee

Hi Edin

Thanks for reaching out, and I agreed this is not the behavior documented in defect # CSCuy64732.  can you please create a TAC case so we can investigate this issue and provide resolution.

Thanks

Zack

Hi,

We opened support case #639040711 on May 19.

Thanks for your reply.

Edin

Hi,

From 31.05.2016 we are not seeing update failures from WBRS. It seems that problem was in some of the updates that were causing problem, although Cisco support did not confirmed that, except that they had similar problems with other customers in same time period.

Also we were directed to defect: CSCuy48239, which I cant see because of privileges, and Support says that it has to do with performance issues.

So far, updates are running fine. Just to note, that we had similar update problems last year also...

Hi,

Our S170 started updating fine on the 31.05.16 also. The funny thing was that support RMA our box and the replacement had the same problem with no load on it. It wasn't until the replacement had the same problem that they really look at the issue again. Still now support can't tell us the root cause and said it was performance even that the replacement box had no traffic or load on it.

Thank you for adding the issue on the forum as this helped me push cisco on fixing the issue.

It would be nice if someone could tells us more about the defect as I also can't access it.

Hi Paul,


Thanks for your reply. We are having same problems starting yesterday (16.06.2016).

Also I found this Bug CSCuy27601. It describes symptoms that we are having, but there is no workaround or anything usable(also we are running 9.0.1-162) .Severity of bug is set to "Enhancement", which is strange....


Are you experiencing same problem again?

Thanks,

Hi,

I have been watching our last update for the last few hours and yes it has now failed with the below error,

Thu Jun 16 13:51:25 2016 Info: wbrs Reverting to factory due to unexpected update failure

I have emailed support as my case is still open.

Hi,

Seems like we are hitting same error/bug/problem. My case is also still open, and I am waiting for their response.

Thanks for sharing your info.

Regards,

Hi,

Latest info is that we are having same problems, starting from today, 28.06.2016. Case is still open, hope this will be resolved in near future.

Regards,

pmjoleary
Level 1
Level 1

Hi, 

We are having the same problem with our S170. The wbrs have not updated since 14 May. same message update log wbrs Reverting to factory due to unexpected update failure.

Support have said the database file is corrupt and they have RMA the box as they can't fix the issue. Which is a pain, because as far as I aware I am going to lose all my reporting data.

we are running 8.5.3-069, 

Hi,

Thanks for your reply. I think that problem is in updates, that are in some way overloading update engine, which results in updates failing after wbrs start to apply updates. Problem was also evident on our virtual S100V, which we installed on 18 May. Starting from yesterday afternoon, on our virtual WSAv, two consecutive updates have finished with success, so will be watching for progress.

S170 last update was on 24 May, after that every update was unsuccessful, so wbrs keeps reverting to factory defaults.

Because of failing wbrs updates and reverting to factory defaults, sites are not categorized and lots of sites fall into uncategorized sites, which are by default set to blocked by our access policy.

We are waiting for cisco support, for further actions.

Hi,

We also block uncategorized sites so it causing us a nightmare. our call is 680369843 if anyone wants to compare the service requests.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: