1.Hard code the speeds on the LD interfaces and corresponding switches.
2.Make sure LD has correct default route statement.
3. If servers or vips are on different subnet than LD, make sure you have alias ip address command.
4.If all else fails, open a TAC case and be prepared to capture simultaneous sniffer traces on front end vlan and back end vlan. Usually, latency issues are difficult to troubleshoot.
I hope this helps!
Gonzalo