I believe that Joseph has identified an important issue about address space being used. Another potential issue is the type of device used for the Internet connection. The drawing shows something that looks like a switch. And that would be ok. But if the device were something that did stateful inspection (as many firewalls like ASA do) then the asymmetric path would be a problem.
The drawing in the original post was quite simple. It showed a host connected through a device that appears to be a switch to 2 ISP with traffic going through one and returning via the other. Now we are finding that the situation is more complex than that. There are different IPs and there is an ASA firewall. We really need a better understanding of the environment to be able to give good answers.
But based on the incomplete information that we have so far I would suggest these points:
- if we have an IP packet whose source address is a Green address and we send it out through the Red ISP then it should come back to us from the Green ISP. This assumes that the Red ISP accepted the packet with a source address that was not one of its networks. Some ISPs have restrictive policies about what source addresses they will accept (one of the motivations here would be to prevent address spoofing), some do not have restrictive policies, and some are willing to negotiate about what they will accept. We do not know which category the Red and Green ISP fall into.
- for devices like ASA when a packet goes out one interface it expects the response to come through that same interface (this is one of the essential aspects of stateful inspection). It is not clear in the drawing where the ASA is and whether both ISP might connect through the same ASA interface or connect through different interfaces.