The echo cancellers can take a second or two to converge at the beginning of a call where a reflection is present. Five seconds is unusually long, but users have been known to exaggerate.
If this happens on all calls you may have a problem, but if it only occurs on calls to a certain number, or very infrequently, it is normal.
