Depending on your network, there are at least few ways of doing this, none that I know of are all that pretty and it depends on your network structure as to which way would be best. Assuming the /24 that you want to egress through router A is directly connected to router and and the other three /24s are directly connected to Router B, you can add a weight to the incoming advertisements from your external peers. The weight is only local to the router, so once traffic arrives on that router, it will use that as an exit point, unless the exit point is down.
The second way to do this is to use policy based routing. You create a route-map which matches on prefix you want to route out of router A, and then set the next-hop to the next-hop to router A. You then match your other three prefixes, (or just omit the match statment for everything else) and set the next hop to your path to router B. You need to apply this on the inbound interface where the traffic is coming from as a policy-map. Depending on how your network is built, you have to be careful not to create a routing loop as you are manually manipulating traffic.
If the prefix that is to go out router A is located on one router C and the prefixes to go our router B are on router D, you can also change the local preference on your iBGP advertisement out to routers C and D. To do this you would apply a route-map outbound from router A to router C which sets to the local preference to say 200. Because iBGP routes are not passed on to any other iBGP neighbors, all of the other routers will still have a local preference of 100 for the outbound destinations . You could then also apply a route-map outbound on the iBGP session from router B to router D doing the same thing. This would make Router D prefer router B as its exit point, and Router C would prefer router A as its exit point. All other routers would be uneffected by the change.
Let me know if this helps or more information about how the network is configured.
