How does that port's usage compare to the other ports in the same Etherchannel?
Realize that if you have a very bandwidth usage intensive flow, it will only use one link, and Etherchannel doesn't take into consideration port usage. I.e. You can be "unlucky" such that you might have just one or more busy flows that "land" on a particular port, and if it's the same two hosts, especially w/o including 6500 hashing using UDP/TCP ports numbers, the same transmission port will be used over and over.
What might be done, is trying a different hashing algorithm, but improvement is not guaranteed as much depends on the attributes of your traffic.
As to using hold-queue 4K, I forget whether a 6500 really uses that (as much, on switches, is done all with special hardware). You could try it. Increasing queue depth often mitigates drops due to transient congestion (like with microbursts), but will not mitigate sustained congestion. It also can introduce additional queuing latency.
