I had a similar situation. Even without a firewall, do you have any ACL's on the switches or any other link. If we look at the call flow it might make more sense.
Phone calles Call Manager to initiate call - Call manager calls the end phone. Once the call is initiated it hands it off and the two devices communicate directly.
If you have any ACL's that allow the phones to talk to the call manager, you will also need to allow all other subnets that have voice traffic.
When I was trying to troubleshoot the issue I was having, I really began to understand the flow of the conversation. I also experienced one way communications.
Phone connects to call manager, call manager dials number, hands off communications and one phone could hear the other, but an ACL was only blocking the traffic from the remote site into the main site, but the main site had free access to the remote site. I hope this helps.