In your situation, I'd think about using one of the new Cisco 7970 phones - not for the conferencing capability, but the external inputs.
I'm assuming if the venue in question is capable of holding hundreds of people, there's already a sound system in place, amplifiers, wireless mics, etc. The 7970 has standard-sized hookups for an external microphone and an external speaker. A reasonably competent sound engineer should be able to tie these into your PA system so that the speaker's existing mic, wireless or otherwise, also feeds into the phone. This would also enable you to pipe the phone-side attendees' audio into the PA speakers, although you would probably want to set up something so you could easily mute it if you hooked it up that way.
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_ipphon/english/ipp7970/admin/3_3_3/headins.htm#1069275