AnyConnect can use a variety of transports. Are you using IPSecV2? DTLS? Or just TLS?
I have AnyConnect deployed on some ISRs and the performance with DTLS enabled (UDP) is pretty decent. I also played around with IPSecV2, but didn't notice any performance gain.
Beyond the above also check for MTU issues, this is a nice write up: https://supportforums.cisco.com/document/12318216/anyconnect-mtu
In short, check your MTU settings and if appropriate enable DTLS to start.