Depending on the version running (I see several people reported this as an issue), a good approach would be to upgrade the software to 15.x and above.
This introduces better memory handling and limits to the process itself. Your appliance is potentially experiencing symptoms reported on CSCwk14053 which includes workarounds that can improve the overall memory consumption behaviour but it can only be applied once the upgrade is complete.
