It is possible to configure direct ip dialing for a specific situation however it is not well documented, tested or supported. As an alternative to this approach, most voip providers allow you to call other in-network numbers using their service without additional charge. Using this solution from a voip provider is much easier, more reliable and much less trouble. I would check the voip provider used by the ata's you wish to call in this manner to see if they match the voip provider you are using.
For direct ip dialing, you can put an element in a dial plan to call a distant user_id at an ip_address:port_number when you dial a certain character string such as #9. Something like this:
|<#9:user_id>S0<:@ip_address:port>|
To make this work you must be prepared to make certain setting changes in the SPA, setting changes in the various ATA router(s) and you need to be able to troubleshoot network addressing and audio problems. You should be able to capture the sip dialog to analyze problems.