You probably don't want to take the port out of the hunt. This would cause problems if they wanted to use failover and really isn't necessary.
You do have a couple options...
CFA a CTI route point and then use a call routing rule to route anything forwarded from that extension to the directory handler -- or slap and extension on directory handler and skip the call routing rule.
Or you can have callers hit a normal call handler and give them the option to enter a number (like 4) to get to the directory handler. You can also record the greating telling them that they can enter an extension during the greeting if you want that functionality. This is how most every company does it.
Hope this helps...
Thanks,
Keith