cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements
1423
Views
0
Helpful
3
Replies
ac513
Beginner

AMP/Secure Endpoint interrupting SharePoint Online browser downloads

 

I have a long story which I believe culminates in a pretty huge bug with AMP/Secure Endpoint, and I'm amazed how it's not been reported en masse (that I've seen), let alone resolved.

 

Since 2020, we have been struggling with an issue where our university's machines have experienced interrupted browser-based downloads from one single location:  SharePoint Online. If we download a file that is 2, 3, 4 or more GB in size, the download will go 25% to 50% of the way before being interrupted and failing, regardless of transfer speed, OS, device type, or location on our network. Any SPO service endpoint ([tenant]-my.sharepoint.com for OneDrive, [tenant].sharepoint.com for all other site collections) exhibited the issue.

 

I had been going back & forth with my network team and Microsoft Premier, with each party blaming each other for the behavior. Microsoft blaming our network/firewall, our network team blaming Microsoft's end or some peering issue.

 

I started the "rubber duck" method as a sanity check on my testing processes. I realized that throughout the course of trying to narrow down any one break point and find common denominators, I had been testing my "no Secure Endpoint installed" scenarios in VMs where the host *did* have Secure Endpoint installed. It never occurred to me that the client could affect connectivity into a VM in such a manner. Well, it did.

 

Once I started removing Secure Endpoint from machines, the issue completely vanished and I could never reproduce it again. When reinstalling Secure Endpoint, the issue comes right back. Physical, virtual, Windows, macOS, Linux, desktop, laptop, doesn't matter. It was fixed for all scenarios, finally, upon removing Secure Endpoint.

 

Notable things about this issue:

 

  1. This is reproducible across Windows, macOS, and various Linux distros when Secure Endpoint is present. All browsers such as Chrome, Firefox, Edge are affected. Only browser downloads are affected... Uploads never broke, and OneDrive sync clients never broke.
  2. This has been happening consistently since we started using SharePoint Online in 2020, so this issue has encompassed, *several* AMP/Secure Endpoint client versions since we pretty much push out new versions within 2 weeks of release.
  3. Moving a sample device to a policy with all engines/scanning features to "Audit" does not typically resolve the issue. The severity/frequency lessens, until it decides to start doing it again. Haven't been able to really narrow down & nail any one engine at play. (Even if I did, my ability to control its behavior in regards to this issue, probably zero)
  4. Adding all SharePoint Online CIDR ranges to a network/IP allow list defined in a policy does *not* resolve the issue.
  5. Secure Endpoint console's device trajectory shows zero blocked activity or any negative details on a computer object that is exhibiting this problem.
  6. Lastly, as stated originally, the problem is completely *not* reproducible if Secure Endpoint is simply removed from a device.

 

Am I crazy? Am I missing something? Has anyone else dealt with a similar behavior in their environment?  It sounds like a bug in one of the engines, but I'm at a loss as to how to convince TAC that this is truly a Cisco problem if I were to open case. 

 

Do Cisco customers just not use OneDrive/SharePoint/Teams for large files? 

 

EDIT: Re-tested since original post, whitelisting SharePoint Online IPs in a policy no longer resolves the issue. This may have been a fluke during past testing.

 

EDIT2: Same exact behaviors experienced with more clean machines, Windows 11 and macOS 12, and latest connectors as of today. (Windows 7.4.5.20701, macOS 1.16.2.853)

3 REPLIES 3
Wojciech Cecot
Cisco Employee

Hi ac513,

Thank you for sharing all details. Assuming you can easily reproduce the issue I think you could open TAC case and report that. Based on notes I see you have already found workaround with whitelisting SharePoint IPs which is quite interesting while I would rather expect some exclusions would be a way to go. If you are planning to open TAC case please attach some examples (preferably some URL to file that TAC could use to repro that in labs).

Regards,
Wojciech

ac513
Beginner

Bump for anyone possibly seeing similar behaviors or having any ideas of what might be happening, before I resort to TAC.

 

Same exact behaviors still experienced with more clean machines, Windows 11 and macOS 12, and latest connectors as of today. (Windows 7.4.5.20701, macOS 1.16.2.853)

We have personally added these exclusions for SharePoint which has helped us without the need to remove security - this might be something Cisco could add to their default exclusions going forward?

SharePoint 2007/2010/2013/2016 HIVE directory

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\*\Data\Applications

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\*\Logs

 

SharePoint 2007/2010/2013/2016 Cache Directories

C:\ProgramData\Microsoft\SharePoint\**\*.ini

C:\ProgramData\Microsoft\SharePoint\**\*.persistedfile

C:\ProgramData\Microsoft\SharePoint\**\*.xml

 

Folder exclusion: e:\spsearchindexes\*

Create
Recognize Your Peers
Content for Community-Ad