You are missing nat exemption (nonat) between your inside network and the vpn pool.
Lets say hypothetically you give vpn clients ips 172.16.4.1-254/24 and on the inside of the ASA you have 192.168.1.0/24 and 192.168.10.0/24. You would need nat exemption as follows for this:
access-list nonat permit ip 192.168.1.0 255.255.255.0 172.16.4.0 255.255.255.0
access-list nonat permit ip 192.168.10.0 255.255.255.0 172.16.4.0 255.255.255.0
nat (inside) 0 access-list nonat
-heather