Looks like a routing protocol issue (they run rip by default) between the two inside interfaces /networks.
Basically if your tunnel is interrupted (even for a split-second) your tunnel will try to re-key, which may fail and depending on routing protocol timers, or the settings on your concentrators, they may not learn about the routes on each side for X period.
So even if the tunnel stays up, as you said, the routes may be unavailble until they are learned again.
I think what you should do is make sure you are manually specifying the networks on each side, with network lists or subnets/wildcard masks (as apposed to "network autodiscovery").
Off hours try to kick the tunnel (by shutting
down the public interface of the vpn for say 5 seconds) now count how long pings fail from the moment of enabling public interface. (with manual networks and "network autodiscovery"
You should also check under
Configuration | Policy Management | Traffic Management | Security Associations | Modify
"ESP/IKE-3DES-MD5" or what ever you use,
your Data Lifetime and Time Lifetime (they should b
e generous, 10,000 and 28,800 seconds.
Another BIG question is are you using NAT outside the
concentrators (is the IP configured on the public interfaces, the ACTUAL ip address your concentrators use on the internet) NAT can cause lots of
"packet out of order" or "duplicate first packet
detected" logs. (in the Filterable Event Log)
This activity prevent tunnel re-key/re-estab
