Internal Error while launching environment

[Martin Votruba]

Hi, I am unable to launch any environment. Whenever I try to launch an environment, it say "could not complete your request because of an internal error". This happens on all environments, also on the always on environments. I do not have any environment active at the moment.

Things I have tried to resolve the issue
- log out myself and log again
- try different sandbox
- try shorter time period

Here is the payload for the environments endpoint which the devnet is making (request and response)

{
"sandbox_name": "Meraki Always On-20240312T13054214",
"blueprint_name": "meraki-always-on",
"owner_email": "martin_votruba@rapid7.com",
"owner_name": "Martin Votruba",
"collaborators": {
"collaborators_emails": [],
"all_space_members": false
},
"source": {
"is_editable": false,
"repository_name": "meraki-always-on"
},
"duration": "PT2H",
"automation": false,
"inputs": {},
"artifacts": {},
"tags": {},
"notes": "",
"workflows": []
} 

{
"errors": [
{
"message": "could not complete your request because of an internal error",
"name": "Internal Server Error",
"code": "InternalServerError"
}
]
}
 

[rich21]

Last week Thursday I got booted out of a UCCX 12.5 lab hours before it was supposed to be finished with an unusual error.  I tried restarting the lab, but got the error below when I tried.  Oh well, I figured I'd just try next week (this week).

Yet I'm still getting this same error when I try to launch the lab.  It happens within about 30 seconds of clicking the button to launch the lab.

Is this a known problem, is someone working on it, do we have an ETA, and is it related to the other problems apparently reported (and reported "resolved") over the past 5 or so day?

SIDE NOTE - for those working on fixing labs, etc - can you please be a little more specific when you update us with what actually you are working on?  There was a string of very vague posts/updates from about 3/7 or 3/8 to yesterday about "The labs are having problems, we're working on it" with no mention of what labs, what problems, etc.  Some comments *seem* to indicate it's problems with the always-on labs, not the on-demand ones, but ... who knows?  Then at resolution yesterday the post was also just a vague "those problems are resolved" without ever even saying what the problems where or what labs it affected. 

[bigevilbeard]

Hey @rich21 that information does not have to provided, it comes under the T&C of devnet and also under the community guidelines if my memory serves me correct. If this was a paid services, then yes expect a RCA/CLCA from Cisco for outages/impact etc.. The only time i see a wide message posted is when many labs or the main platform is impacted, posting update per lab/sandbox could be very time consuming, certainly the always on devices which are impacted mostly due to abuse.

As frustrating that it is that sandbox have issues, this is a free service ran by the devnet team for community to play with.

[bigevilbeard]

The always on, you do not need to select 'reserve' you can access these via the information panel on the right side of the lab.  Taken from another post team advised with the always on devices. 

Always On labs in the near future to become reservation based. Users will need to create a reservation to generate their own credentials to the server. We will no longer be providing the same creds for everyone. It results in far too much abuse of the resource, killing access for everyone else.

 So the error message you get with trying to reserve an always on is a false flag, and as long as you can access the devices with provided shared information, you should be good!

Hope this helps.

[rich21]

Thanks for that update.

Do you have any information on how this does or does not affect the 'not always on' environments we've both mentioned that won't start?

And when does the "Users will have to create a reservation" go into effect, if you know?

[bigevilbeard]

@rich21 only from my exp, when the sandbox was move to the new platform this year, this feature appeared, but is not required as the information to access the always on is publicly visible in the side panel for the lab, when i used these labs, i just grab the username/password and connect, i bypass the reservation step. The always on labs have suffered over the year (as has the community) we constant misuse/abuse which would be the driving factor to make these labs have dynamic password as the admin noted in another thread.

I do not know a date, i saw this in a thread the admins updated today here https://community.cisco.com/t5/devnet-sandbox/announcement-experiencing-high-failure-rates/td-p/5036709