Hello Joseph,
Sorry for the delay in response.
To control access to ASR5000 via telnet, other than configuring an ACL, there is a way to disable telnetd by configuring local context.
For example:
config
context local
..
no server telnetd
..
#exit
System Administration Guide of the relevant version will give you detailed information in this regard.
Here is the latest system admin guide (for SW version 17): http://www.cisco.com/c/dam/en/us/td/docs/wireless/asr_5000/17-0/PDF/17-ASR5000-Sys-Admin.pdf
You can find other guides here: http://www.cisco.com/c/en/us/support/wireless/asr-5000-series/products-installation-and-configuration-guides-list.html
Hope this helps..
Regards
Aneesh