I would still be inclined to say it is QoS related. And notoriously hard to diagnose. First of all I would verify if your RTP stream has the proper DSCP marking of ef. The easiest way to do that is to call into a Jabber softphone which has wireshark running on it.
Please remember to rate useful posts, by clicking on the stars below.