Latency increased 800ms when crypto map tunnel establishes

[JustJeffG]

I'm using CSRs to build tunnels between cloud instances.  On router A, before my tunnel is established, I can ping my default gateway and out to the Internet without issue.  Gateway pings are 1ms as expected and pings to google are 10ms.  Once my VPN is established to another CSR, pings to my default gateway goes to 800ms and I'm losing packets.  If I drop the tunnel, my pings return back to 1ms.  Below is my config.

crypto isakmp policy 1
encr aes 256
authentication pre-share
group 5
crypto isakmp key thisismycryptokey address 2.2.2.2 
!
crypto ipsec transform-set P2-AES256 esp-aes 256 esp-sha-hmac
mode tunnel
crypto map TUNNEL local-address GigabitEthernet1
crypto map TUNNEL 10 ipsec-isakmp
set peer 2.2.2.2
set transform-set P2-AES256
set pfs group5
match address crypto-map

!

interface GigabitEthernet1

ip address 1.1.1.1 255.255.255.0

crypto map TUNNEL

[balaji.bandi]

how is your routing are you using tunnel route to ping public internetl ? or do you have split tunnel ?

[JustJeffG]

I'm only routing private addresses across the tunnel.  Pings to my default gateway jump to 800ms when the vpn comes up, if I take down the vpn, my pings return to normal response times.