This example shows how to allow telnet from an internal network, ssh from any but deny anyone else while logging all activity
ip access-list extended TerminalAccess
permit tcp host 10.0.0.2 any eq telnet log
permit tcp any any eq 22 log
deny tcp any any log
!
line vty 0 4
access-class TerminalAccess in
You could also use the line "transport input none"
The best option is to have some secure means to remotely manage the device.