I'm afraid that is just the way it is. Both ends judge independently whether the traffic is interesting, and both ends have their independent timeout. The only workarounds are to flag all traffic as interesting at your end, or to make your idle timeout very long.
I have asked our supplier to make a feature request to Cisco to be able to configure different idle timeouts depending on the direction of the call, but so far the request has fallen on deaf ears. All I can suggest is that you ask your supplier to do the same, and if enough people request it, Cisco might listen.
Kevin Dorrell
Luxembourg