Jay,
you probably have configured stickyness.
This is a mechanism that ensure that a single client always uses the same server.
This is often required by website where you have a 'basket' with items in it.
The basket only exist on a single server, so if you are loadbalanced between servers, you end up losing items.
The negative effect of stickyness is that it forces all connections from that single client to go the same server.
If you have clients behing a proxy, all connections appear as coming from the same source ip.
Therefore the loadbalancer will send all connections to the same server.
Same if you start with 2 servers only, after adding more servers, if you do not clear the sticky table, the CSS will continue the old information and forward all requests to the first 2 servers.
The sticky feature is configured with 'advanced-balance' command.
First step will be to clear the sticky table.
Be aware this should not be done during peek hours since users may be redirected to a new server and lose the information they were working on.
If you have advanced-balance srcip, you should also study the possibility to try 'advanced-balance arrowpoint-cookie'.
Loadbalancing is a huge topic.
If you really have no experience, it will be difficult to know what to do.
Like troubleshooting BGP with no experience with any routing protocol.
Gilles.