One-way voice can occur in Media Gateway Control Protocol (MGCP) gateways if the source interface for signaling and media packets is not specified. You can bind the MGCP media to the source interface if you issue the mgcp bind media source-interface interface-id command and then the mgcp bind control source-interface interface-id command. Reset the MGCP gateway in Cisco CallManager after you issue the commands.
If the mgcp bind command is not enabled, the IP layer still provides the best local address.
If its already there, then you should collect the detailed callmanager traces as well as the following debugs from the gateway for a call with one way audio
debug mgcp packets
show call active voice brief
By pressing the '?' on the IP phone you can verify the Tx and Rx packets , if they are incrementing properly. If Rx packets are not increasing then the issue could be with a firewall between the MGCP gateway and cucm or a routing issue. However, as the issue is intermittent, the requested traces / debugs might be needed for exact conclusion.
HTH
Manish