What type of audio and PST connection(PRI, SIP, etc.) are you using. If CCA, I would open a request with telco, otherwise TAC.
Many times telco can find the call path and try rerouting over a different underlying carrier. Whether it helps or not, is a toss up.
You can also start looking at edge audio if you think you would be able to provide a better (internet based) connection.
I typically do not panic about a on off quality issue. However if its multiple separate meetings and occurrences, sounds like something telco and TAC would need to look at.