You could, but it's a major pain in the butt. Essentially what you would need to do is go to User > Roles > Agent... go to the Relationships tab and select Actions. What you'll be looking for is the Resource Type, Routing Queue and its name, Transfer Activities. You'll select that and remove it from that list. This will remove it for everybody, but you can add it back to the rest of the agents, but you have to do it agent-by-agent.
The problem is that the Actions for the agent are greyed out, otherwise you could've easily removed just that Action for that one specific agent. We can't do this is because of the integration process. When you import users, they're automatically assigned the Agent role and you cannot remove it.
