When you configure aaa it establishes a default authentication for anyone who is logging into the router. You can establish additional authentication methods to supplement this.
If you do want users who connect to vty (and perhaps console) to be authenticated via TACACS and want other users to not be authenticated then I would suggest that you configure the default method to be none and configure an additional method to use TACACS and assign it to the vty (and console if you want).
The config would look something like this:
aaa new-model
aaa authentication login default none
aaa authentication login vtyauth group tacacs+ line
aaa authentication enable default group tacacs+ enable
line console 0
login authentication vtyauth
line vty 0 4
login authentication vtyauth
some details may vary depending on the IOS version that you are running.
I have used an approach like this at a customer site and it works well for us. I hope it work as well for you.
HTH
Rick
HTH
Rick