Heads Up :
The post you are writing will appear in a public forum. Please ensure all content is appropriate for public consumption. Review the employee guidelines for the community here.
ACL's are exact. They either do or don't work. You have a pattern that points to a slowdown of some resource on your network (seeing that it's not bandwidth related). Do a 'show proc cpu' during the slowdown period and compare with the same for norma...
What is the capacity of your connection to your upstream provider? You may be hitting the ceiling during the peak times and seeing that you have proxy server(s), they may have significant impact on your total capacity, depending on prefetch options, ...