WAE512 File Manager failed / Poor DRE results on one of our WAE devices

[joepena2012]

Hello all,

during a routine check of the WAE statistics, i detected poor results on one of our WAE's.

The reduction there is around 10% overall. Normally it is around 60% in our environment.

With a closer view to the device, i detected the following errors on syslog server occuring once a minute:

2010 Sep  3 07:38:54 nhkwa222 so_dre: %WAAS-RE-3-690432: (480264:2) FileMgr failed in writing data-segment start at id=141978, error=0xfffff
2010 Sep  3 07:38:54 nhkwa222 so_dre: %WAAS-RE-3-690464: (506935:2) RE File Manager error: filemgr_state::do_io (segments) call pwrite(44, 0
7734400): No space left on device
2010 Sep  3 07:38:54 nhkwa222 so_dre: %WAAS-RE-3-690432: (507266:2) FileMgr failed in writing data-segment start at id=141847, error=0xfffff
2010 Sep  3 07:38:54 nhkwa222 so_dre: %WAAS-RE-3-690464: (533772:2) RE File Manager error: filemgr_state::do_io (segments) call pwrite(44, 0
8297216): No space left on device
2010 Sep  3 07:38:54 nhkwa222 so_dre: %WAAS-RE-3-690432: (534071:2) FileMgr failed in writing data-segment start at id=141703, error=0xfffff
2010 Sep  3 07:39:54 nhkwa222 so_dre: %WAAS-RE-3-690464: (588960:2) RE File Manager error: filemgr_state::do_io (segments) call pwrite(44, 0
6319616): No space left on device
2010 Sep  3 07:39:54 nhkwa222 so_dre: %WAAS-RE-3-690432: (589308:2) FileMgr failed in writing data-segment start at id=141978, error=0xfffff
2010 Sep  3 07:39:54 nhkwa222 so_dre: %WAAS-RE-3-690464: (615997:2) RE File Manager error: filemgr_state::do_io (segments) call pwrite(44, 0
7734400): No space left on device
2010 Sep  3 07:39:54 nhkwa222 so_dre: %WAAS-RE-3-690432: (616301:2) FileMgr failed in writing data-segment start at id=141847, error=0xfffff
2010 Sep  3 07:39:54 nhkwa222 so_dre: %WAAS-RE-3-690464: (643513:2) RE File Manager error: filemgr_state::do_io (segments) call pwrite(44, 0
8297216): No space left on device
2010 Sep  3 07:39:54 nhkwa222 so_dre: %WAAS-RE-3-690432: (643821:2) FileMgr failed in writing data-segment start at id=141703, error=0xfffff

Apart from these two indications the WAE looks o.k., especially on CM we cannot see any problems with this device.

--> the first logfile messages, indicating a problem were these:

a222.hk.festo.net 2010 Aug  5 05: so_dre: %WAAS-RE-3-690464: (398362:18) RE File Manager error: filemgr_state::do_io (segments) call pwrite(
6, 526319616): No space left on device
a222.hk.festo.net 2010 Aug  5 05: so_dre: %WAAS-RE-3-690432: (454098:18) FileMgr failed in writing data-segment start at id=141978, error=0x
ll
a222.hk.festo.net 2010 Aug  5 05: so_dre: %WAAS-RE-3-690435: (454140:18) RE cache disk full, enter bypass mode

[Thomas Jardin · Accepted solution]

Hello Dieter,

The "out-of-space" error log is usually seen if more memory was added to the WAE.  When you add memory, it changes the size of the DRE cache, but it doesn't automatically change the DRE partition.  To repartition the disks, enter this command,

disk delete-data-partition

Then reboot the box.  Unfortunately, this will remove all cached data, but it is necessary to resolve the issue.

Regards,

Tom Jardin

[joepena2012]

Hi Tom,

You were right. I checked the "sh disk detail" and saw only 60GB available for DRE cache (cache size with 1GB RAM on WAE 512)

But we have 2GB RAM on WAE 512.

After i did the "disk delete-data-partition" followed by a reboot, the partitions were rebuilt and we got a dre cache size of 80GB

Additional we have no more syslog messages, indicating a problem with this device.

Your recommendation fixed the problem.

thank you very much

Dieter