Michael,
As you might aware you can do this by url or language or header-field-group IE or Netscape etc...
I haven't come across any such option on the CSS that can do 'restrictive' load balancing among the servers based on the source ip addresses.
The other option is the GSS which can do this but its not at Server level, rather a Site level load balancing box by issuing alternate(or in some other fashion) DNS A-Records of either site. Here you can restrict the audience I mean you can define who should get an A-record of a particular site...
Hope this would help...
thanks