You can verify it: Just plug off your provider router and put in an own server. Probably, you will recognise the same issue.

We are running into the same issue.Slow Internet download, good upload. Good down an upload on local servers. It's very certain a problem of fragmentation. We had put a server outside firewall and can reproduce the issue, same here: slow download, go...