Well you need to find out if the tunnel goes down indeed or if it just stops passing traffic.
when the issue arises get these outputs:
"sh cry isa sa"
clear the ipsec sa counters:
"clear crypto ipsec sa counters"
and get the following output:
"sh cry ipsec sa peer X.X.X.X"
Issue the above command a couple of times to see how the counters are incrementing.
If just the encaps increments and the decaps stay at 0 for that particular SA then the issue is at the remote end on the other hand if it is the opposite then the issue is on the ASA and we might be duplicating ASP entries.
captures would be helpful too so we can see what it's going on with this traffic.