What you suggest sounds fairly typical for configuration of Policy Based Routing. You would start by configuring access lists that identify the traffic. So probably you want one access list that matches the source address for traffic coming from VLAN 10 and 20 and another access list that matches the source address for tarffic coming from VLAN 30 and 40.
Then you would configure a route map. In the first instance of the route map you would match the first access list and would set the ip next hop to use ISP 1. In the next instance of the route map you would match the secnod access list and would set the ip next hop to use ISP 2.
Then you would assign the route map to interfaces. PBR route maps are assigned on interfaces where traffic arrives. So if you have 4 VLANs connecting via router interfaces then you would need the route map assigned on the 4 interfaces.
Failover with PBR can get a bit complex. Sometimes PBR will automatically fail over (if it knows that the next hop is not reachable). But sometimes (especially on Ethernet interfaces) the next hop may not work but the router does not know this. So if you want failover you probably will want to include the verify availability on the set next hop command. This will use tracking to verify if the next hop is working or not.
HTH
Rick
HTH
Rick