There's alot of details here that might be needed to troubleshoot this issue...
But here's a basic issue that you should address: A device can only generally use 1 default gateway at a time.
Now that's not true in some situations, but it might be the issue here. Your 'server' is already configured to access the internet via one path. Now your trying to access it via a different path. The traffic will get to the server, but the server may only have 1 default gateway configured. So the traffic goes back out that gateway which is a different path then the way the request came in.
This can create asymmetric data paths, which firewalls do not like at all.
One solution would be to PAT the incoming traffic from one of the internet connections to a local LAN IP, if the traffic is destined for the server. That would enable routing and everything to work correctly.