Let's assume your management ip is 1.1.1.1
You can create an acl like
access-list 100 permit any host 1.1.1.1 eq 22
Then apply it on your lines like
Line vty 0 15
access-class 100 in
Not sure if it works on 2960xr. I know there was a bug with extended acl which has been corrected for routers.
If that doesn't work then you need to apply an acl on your L3 interfaces to deny inbound ssh and allow it just on your management.
Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question