Dashboard Issues?

[Twitch0331]

Anyone else seeing weird issues with the Dashboard today? I was able to access it just fine earlier today, but then access to the it just went belly-up. Not long after that, I began receiving all of these notifications that devices are going down at all sites, but when I look at them in the Dashboard (which became reachable again) they appear to be just fine. The fact that my email and phone are not blowing-up with people in a panic seems to confirm this.

Are there any known-issues at the moment??

Thanks!

Twitch

[ZeuS41]

looks like Meraki does have a lot of explaining to do, we bought into this "cloud" idea thinking the clients should not be impacted incase if the Meraki dashboard is unreachable, it turns out all VPN tunnels and clients access are linked directly to the cloud. the dashboard should only be used for management not access. if they dont fix this, we are switching our entire Meraki platform very very soon.

[DHAnderson]

My clients and tunnels are still up, otherwise I would have heard. They absolutely rely on Meraki, and not one of them have reported any issues. I even called on client and they reported everything was working. All of my clients absolutely rely on Meraki, and not one of them have reported any issues.

My own equipment reported that it was down, but my home office kept humming along, with emails pouring in!

[El-Guero-HTX]

Did your site to site VPN go down?

We have about 21 sites across the US and all our tunnels are up. We use a separate monitoring tool to monitor.

It is alarming to receive all of the up/down alerts thought.

[anygaard]

Reviewing our separate network monitoring, no Site to Site tunnel issues, all stayed up, we have 50+ locations. Haven't heard any reports of outages of issues impacting end-users, thankfully.

[RodrigoCW]

Hey Everyone,

Justa wanted to jump in and let everyone know that we are aware of the ongoing issues with Dashboard. As of now, we have confirmed that the issue is isolated to a small subset of servers, and as of 20ish minutes ago, service impact should be mitigated. We will share more information once we have conducted a full investigation into the issue.

If you are still experiencing issues, please feel free to let me know.

All the best,

Rodrigo

Meraki Support

[Twitch0331]

Hey @RodrigoCW - Thanks very much for the update. We appreciate everything you guys are doing to figure-out what's going on.

[RodrigoCW]

Hey @Twitch0331,

Thank you for your kind words. We are deeply sorry for the inconvenience and we'll keep you all appraised as things develop. In the meantime, if anyone has any questions or concerns, feel free to DM or tag me!

[Twitch0331]

Our site-to-site VPNs have remained operational. They still show up/up and green. I haven't heard anything to the contrary in terms of remote site connectivity.

[wayne.ranchuk]

I have had all of my Security appliances reporting they are going down but they are not. no other equipment has generated any emails. I have been on hold with support for 30 min so i am assuming it is a big issue for them

[Twitch0331]

Thanks, everyone. Good to know it's not just our network.

Other than the alerts, All is Quiet on the Western Front in terms of user complaints.

[Blueshift]

Can confirm here.
Not only was cloud unreachable, logs show all of our wireless devices re-authing with the APs and all of our manually disabled ports on the switches are now re-enabled.

No users seem to have been affected, however.

[ZeeBoussaid]

same goes for us, we have 200 remote office with VPN tunnels dropped. what happened today should scare the crap out of any engineer. this is going to be a nightmare if all access to clients is tied directly to the Meraki Dashboard. i hope Meraki fix this issue or at least chime in. because im seriously considering dropping all Meraki products in our environment.

[rconiv]

Getting it here also, had a recorded outage from 10:52 to 11:01 PDT at both our local and colo, and then been getting some emails since for either both locations or just the colo loosing/gaining connection to the Meraki Cloud. As others have said, nothing seems to have really died because there was no screaming. Have been getting occasional emails since mostly for the colo. Am on hold with Meraki, who I am pretty sure we have overwhelmed them.

[Remo2]

This happened to my NOC.

Multiple sites seemed to have dropped for about 7 minutes, but there was one site that we manage that was not affected.

Meraki confirmed that this was only a cloud management outage as hosted services migrated to another site since the affected site was under an outage. No actual service impacted issues, just management/visibility.

I imagine you'll get the full details if you call them or open a ticket with them against one of the networks that were affected in your organization.

[PDSKturley]

For what it's worth, None of our Meraki VPN's or site to site VPN's actually dropped. Not a singe call from a user about disconnects or WiFi access lost.

While this is a serious issue, we are relieved it didn't actually affect usability. Just the management portal. Meraki will certainly have to answer the questions about why. And reassure us this platform is resilient. I would expect an explanation of the root cause, and details about the resolution for future mitigation should it happen again.