Hi Jamie,
It is not currently possible to configure the Caller ID that is displayed for incoming FXO calls but if your FXO Service Provider delivers Caller ID that is what should get displayed on your incoming calls. If not, you might need to open a case with the Small Business Support Center.
Regards,
Jeff