the CSM will loadbalance the SYN [unless you have a Layer 7 vserver - you can see the level with sho mod csm x vserver detail].
Moreover, a connection failure indicates a server did not respond to a SYN or responded with a RESET.
So, I would sugges to verify the level of your vserver [most probably not l7] and then check your servers.
Make sure they sent traffic back to the CSM and not directly to the client.
Make sure the server is alive.
Regards,
Gilles.