Hi nygenxny123,
the easiest way is indeed to control which traffic is processed by the ACE via the ACL.
However you could do it this way (being SFARM1 your existing serverfarm)
class-map match-all MYCL
2 match virtual-address 10.20.30.40 tcp eq www
class-map type generic match-any C-SRC-IP
2 match source-address 1.1.1.1 255.255.255.255
3 match source-address 2.2.2.2 255.255.255.255
4 match source-address 3.3.3.3 255.255.255.255
policy-map type loadbalance generic first-match PM-SRC-IP
class C-SRC-IP
serverfarm SFARM1
class class-default
drop
policy-map multi-match MMPOL
class MYCL
loadbalance vip inservice
loadbalance policy PM-SRC-IP
loadbalance vip icmp-reply active
Still I would recommend to use the ACL.
Hope this helps,
Alessandro