Flood of OnPlus alerts (multiple sites) from 9:15PM ET - 9:20PM ET

[davebainum]

Hi all,

Hope all is well with you.  We got a flood of OnPlus alerts from multiple sites/multiple locations from about 9:15PM ET to 9:20PM ET last night, Thu. 7/14.  DNS Status, Connection Status, Configuration backup, you name it.

Just wanted to check in and see if there may have been some sort of "hiccup" on the portal or monitoring server side perhaps around that time frame...

Thanks,

-- Dave Bainum, PMP* (dbainum@ritetech.net)

RiteTech LLC / www.ritetech.net / Tel. +1 (703) 561-0607

[*PMP=PMI Certified Project Management Professional]

[Michael Holloway]

Hi Dave, we're still investigating what caused a few folks to receive bunches of notifications last night around this time. Our service ops team was doing work on the beta and release portals around this time, this could be related to DNS changes or certificate changes that were made around then. I'll let you know once we figure it out.

[Michael Holloway]

Dave, we've determined the root cause for the spate of emails last night. A bad ACL on the onplusbeta.com platform was preventing email delivery for at least a couple of days. Our operations team installed a new pair of routers last night and pushed out the correct ACLs to them, and then mail delivery started working again and then the previously undelivered emails got flushed out.