New Updates to CML and NSO sandbox labs

[jokearns1]

Hi Community, 

Yesterday, we released updates to our popular CML and NSO sandboxes. 

Cisco Modeling Labs (CML)
We want to scale the CML sandbox. Right now, if slots are full, users have to wait. This is annoying and we want to address it while keeping the core CML functionality. 

• CML updated to 2.7.2
• Supported Routers --> CSR1000, IOL, IOSv
• Switches --> CSTkv, IOL-L2, IOSvL2, 
• Linux, --> Ubuntu, Alpine, TREX
• Other --> WLC, External Connector

We have removed the huge XRv node. It just ate too many resources to scale the sandbox adequately. If anyone wants to experiment with XRv programmatically, we have a dedicated XRd sandbox for that. 

The sample topology has also been updated. See below:

These changes will allow us to scale up concurrent sessions and help address demand. We cant promise we will ever reach demand but this will definitely help.

Network Services Orchestrator (NSO)

The NSO sandbox also received some updates yesterday. 

• New Docker containers added for NSO
• Updated CML topology
• Updated access information and instructions

This is the first of a two part update, the second being pushed next week.

We would like your feedback on any updates we push. Please reach out to this thread with any questions or comments. 

Thanks, 

Sandbox Support 

[mariem56]

Will you change the reservation time? I think the max is 2weeks? and maybe because of that CML is always full...

[jokearns1]

Hi @mariem56 
Reservation time is normal 5 days. However, we may reduce this to get more labs available. 
I have increased the max number of labs again so more are available now. 
Thanks, 

Joe 

[wahezu]

Hi,

NXOSv was also removed. Is there any other lab where we can test NXOSv?

Thanks,

[wahezu]

Regarding my last comment, I can see other labs where we can test NXOSv but with a single node, I'd like to test a complete spine&leaf topology, which I was able to test  on the CML sandbox until very recently when NXOSv node support was removed.

Thanks,

[Douglas Philyaw]

We're having issues with the cat8000v images using more memory than prior to the update. We've tried increasing the available memory for each the nodes but this doesn't seem to help. We've also tried lowering it and the nodes won't boot up at all. Is there a limit on cat8000v nodes?

[mgottesfeld]

Dear Sandbox Support:

First, I appreciate your efforts to scale the C.M.L. sandbox.  Without the sandbox I never would have passed ENCOR last week after a significant hiatus from hands-on deployments!

That said, in case you are not already aware of this (which does seem unlikely), the A.S.A. is gone from the C.M.L. sandbox, and, as a result, several of the demo labs now refuse to load.  I see nothing in the announcement, above, regarding removal of the A.S.A.  Nor do I see any change to the stated purposes of the C.M.L. sandbox.  I thus assume Cisco wishes to continue to make it available to folks like myself to learn.

Due to the ASA's ubiquity—a tribute to its success—it is quite impossible to model a realistic enterprise network without an A.S.A. pair.

I, too, miss the NXOS 9000.  It was, in my recollection, CML's only layer-three switch with meaningful port density and vPC.  That said—and I speak only for myself here—I can make do with any layer-three switch that supports dynamic routing protocols, FHRPs and vPC, stacking or V.S.S.

Last, it must be mentioned that, thus far, the update effectively ruins the C.M.L. sandbox as a pre-sales tool for smaller Cisco partners who lack the resources to run C.M.L. internally.  Please see the attached YAML file, on which I was working when the update began and which now refuses to load.  This type of lab (yes, I know, it is not all working, as I was in the middle of a reconfiguration), shows the former power of the C.M.L. sandbox as both a learning and sales tool.  (No more than 20 of the nodes must be powered on to demonstrate any particular feature, e.g., D.M.V.P.N., M.P.L.S., eBGP E.C.M.P., etc.)  My understanding is that Juniper continues to offer this kind of free sandbox.  See Juniper vLabs https://jlabs.juniper.net/vlabs/.

I had planned to publish the attached lab for free as a learning and sales tool upon its completion, while I work toward C.C.I.E.

I thank you for your consideration of this matter,
by /s/ mgottesfeld, Cisco Certified Specialist—Enterprise Core; Cisco Certified Network Associate (2003, expired).

[bigevilbeard]

@mgottesfeld some thoughts here. Whilst I understand that many people use the sandbox for different reasons, it’s real reason is to supplement learning for the Devnet learning labs, however as this was built and replaced virl that it should be more open for more general use. But this from what you are describing your use case not suitable and I am sure you have hit limits in your path here etc. Cisco does provide an environment for pre sales and demo, via Cisco dCloud. I would suggest for your use case this is more suitable for your needs and has more CML labs. Though not ideal if you are making content and pointing users to this as they might not all have the access required.

[mgottesfeld]

First, thank you for pointing me to dCloud.  When I am back in the field, which I am not now while I recertify, I will be sure to evaluate it.  It is unclear to me, however, whether dCloud allows for customization of the demos, like adding or removing nodes and offices, changing WAN designs and parameters, saving and loading changes, etc.

Customers want to see their networks, their logos and that a Cisco partner has paid close attention to their specific needs.  In the sales context, generalized demos are unpersuasive.

Second, I respectfully disagree with your statement that the C.M.L. sandbox's "real reason is to supplement learning for the Devnet learning labs."  Cisco's statement of purpose that one reads when launching the C.M.L. sandbox makes no express mention of "Devnet":

Cisco Modeling Labs is a tool for building virtual network simulations (or labs) for you to test out new topologies, protocols and config changes; automate network tests via [continuous integration/continuous delivery] pipeline integration; and learn new things about the cool world of networking.

This sandbox provides access to a Cisco Modeling Labs system that can be used to explore the capabilities of the newest release of Cisco Modeling Labs Personal or Enterprise.

See screen capture, attached (emphasis added).  Cisco may, of course, change this statement if it wishes.  Unless and until it does, I and others may only assume that Cisco says what it means and means what it says.

The key elements of the C.M.L. file on which I was working were—

1. For those pursuing the enterprise-infrastructure (formerly R&S) track, it would have allowed them to study and test a realistic enterprise network in stages as they progressed through the syllabi; and
2. It would have been about as easy as possible for sales engineers to customize to suit particular leads.

The relatively level playing field Cisco has always offered through its certification program, e.g., no formal prerequisites for the C.C.I.E. and exams relatively light on marketing fluff, remains a beacon in a world that seems ever less concerned with a candidate's merit.  The availability of the C.M.L. sandbox went a great way toward opening the door to the widest possible talent pool, to folks who otherwise would be priced out of a C.C.N.A.

Perhaps the primary limit that dissuaded me from pursuing a C.C.I.E. twenty years ago was the cost of effective lab time.  If the changes announced in this thread had been made to the C.M.L. sandbox weeks earlier, I am unsure I would have passed ENCOR to become a C.C.I.E. candidate.

I continue to hope that Cisco is aware of and values the utility of the C.M.L. sandbox to "learn new things about the cool world of networking," especially given there are never enough certified engineers to meet every customer's needs,

by /s/ mgottesfeld, Cisco Certified Specialist—Enterprise Core; Cisco Certified Network Associate (2003, expired).

[bigevilbeard]

@mgottesfeld for sure checkout dcloud it’s a great resource, if you have access - you would need to see if this would meet your needs as far as building something custom. We can go back and forth on wording here of the 'why' CML is there, but using this as a POC, DEV infra, tool testing for company, i would never suggest this and lets face it, if someone is using a free resource which is meant for the community to build and generate money, thats not really fair right? For example in your case, you would not have been able to book CML and pass your exam. There is/was plenty of thread on the reservation of CML and now the team has had to make bigger changes to allow folks to use this now. 

So, DevNet is small team inside Cisco which focuses on education and network automation, they are not part of learning, or Cisco corp so its not really a good fair to say 'Cisco' when talking about DevNet. Hence my comments on POC etc.. as there is a dedicated part of Cisco for small biz, POC etc.. so there is better methods for sale too here. 

Yeah totally with you on the CCIE study, when i was doing this i had stuff from evil bay, was ripped off many times with broken, non working stuff, GNS3 was only just out and not really an option at the time, it was hardware. You will have been part of many talks why Cisco does not provide student images, test images.. this is as old as time rant, or better labs for learning for certs, free tier even which will get you so far, but scaling this is cost, hence why DevNet can only run a limited capacity and footprint. 

So really my point here, DevNet and this small amazing team which provided all these free resources cannot be shouldered with holding up the big machine of Cisco and all its customer use cases.

[mgottesfeld]

Once again, I respectfully disagree.

After your post, Cisco partially validated both our points.  The "primary"—but, noticeably, neither the "real" nor sole---"aim of DevNet is to give developers real world access to Cisco APIs." (emphasis added).  See jokearns1 (1025 hrs. G.M.T. Oct. 16, 2024), below.  Importantly, "The [C.M.L.] sandbox has to be as generic as possible to catch as many use cases as we can while allowing scale."  Id.

Thus it seems that the truth of your statement, that the C.M.L. sandbox is "a free resource which is meant for the community to build and generate money," is wider in scope than you may have first considered.

As just one, recent, example, a 2020 "survey revealed that 86% of enterprises consider[ed] talent shortage to be a key barrier to achieving their desired outcomes."  See Everest Group, 75% of Enterprises Bracing for Widening IT & Analytics Talent Shortage, Despite Recession (Aug. 19, 2020), available at https://www.everestgrp.com/2020-08-75-of-enterprises-bracing-for-widening-it-analytics-talent-shortage-despite-recession-press-release-.html (last accessed Oct. 16, 2024).

To focus on one particular area, relevant to the removal of the Adaptive Security Appliance from the C.M.L. sandbox, the "widening cybersecurity talent gap not only poses a risk to the global economy, it opens the door for increased cyber-attacks—especially as tech like AI makes it easier for bad actors to infiltrate systems."  See Fore, Cisco is continuing to invest in the future of skills certifications.  Here's how the company thinks it'll help fill the lack of cybersecurity experts, Fortune (Oct. 16, 2023), available at https://fortune.com/education/articles/cisco-u-cybersecurity-skills-certification-training-program-cisco-networking-academy/ (last accessed Oct. 16, 2024).  "Thus, says Francine Katsoudas, executive vice president and chief of people, purpose and policy officer[sic] at Cisco, it is more important than ever to address the cybersecurity skills shortage.…  'Cisco's purpose is to Power an Inclusive Future for All, and building cybersecurity skills and tools is a crucial part of creating that future.'"  Id. (emphasis added).

Please do not take for granted that Cisco-certified engineers enable, empower and secure the very solutions you come to DevNet to build.

I see nothing unfair about my or any other certification candidate's use of the platform.  The separation between Cisco and DevNet that you assert is contradicted by some of Cisco's deliberate choices:  We are on the Cisco.com forums, the administrator who started this thread requesting feedback is conspicuously labeled a "Cisco employee," not a "DevNet" or even a "Cisco DevNet" employee.

And, of course, regardless of how Cisco chooses to structure itself, it should aim to exceed, or at least for parity with, its competitors' offerings, e.g., Juniper vLabs.

Ironically, too, those who used GNS3 back in the day to run Cisco platforms, before Cisco offered VIRL images, were almost certainly breaking the law by their non-sanctioned uses of I.O.S. and A.S.A. images.  Cisco could have cracked down hard on the practice.  Its apparent choice to refrain seemed a deliberate, if tacit, acknowledgement of the learning community's needs and the benefits to the wider community of having some such platform available, at least until it got VIRL off the ground.

As for personal C.M.L. purchases, U.S.-based candidates are, for the most part, willing to pay $200 a year for C.M.L.  That cost alone is minor compared to the costs of the actual exams and preparation materials.  And CML's dollar-to-value ratio as a practice resource would be unsurpassable.  Though sufficient for some candidates, especially those with some degree of corporate support, this approach has big showstoppers more broadly.

First, in many if not most parts of the world, $200 is quite a substantial sum.

Second, even in the U.S., most unaffiliated candidates lack the personal hardware to virtualize 20 concurrent nodes.

I thus reiterate that, if Cisco's goal is to tap the widest possible portion of the global talent pool, a learning resource like the C.M.L. sandbox is vital.

All this said, I hope for an outcome with the C.M.L. sandbox that acknowledges and empowers both our use cases.  I will respond directly to jokearns1, below, with ideas, and I welcome your and others' commentary on my proposals.

[bigevilbeard]

@mgottesfeld  I appreciate your detailed and thoughtful response. Let me share some additional context that might help frame this discussion better.

Speaking from my experience (12 years at Cisco, with 5 years on the DevNet team where I helped build many of these resources, inc DevNet cert and the DevNet Expert track), I can tell you that the sandbox challenges you've identified have been ongoing since its 2016/17 inception. Your points about accessibility and learning resources are correct, i 100% agree. The real issue we're grappling with isn't certification candidates or learners using the platform - that's exactly what it's meant for. As Joe noted, for using getting hands on exp with API's this was DevNet was formed.

The challenge comes from companies using it as a replacement for their own QA infrastructure, or people monetizing it through paid courses and services. This kind of usage consumes resources meant for learners and developers, undermining the original DevNet mission. While I strongly support certification candidates using the sandbox (and would never suggest otherwise), we need to acknowledge its technical and resource limitations. These constraints affect everything from scale to platform version control to security posture.

You make an excellent point about cost barriers - I faced similar challenges in my own journey. The solution isn't restricting legitimate learners, but rather finding ways to prevent commercial exploitation while expanding access. During my time at Cisco (and continuing now as a Cisco Champion and VIP), I've advocated for better solutions to support students and learners.

You've provided valuable insights here that highlight the continuing need to balance accessibility with sustainability. I share your vision for more comprehensive learning resources - we just need to ensure they're protected from misuse that could limit their availability to the people who need them most.

BR.

FYI The forums are managed by each Cisco org/team, although yes in the Cisco domain all Cisco staff show as 'employee' regardless of their role within the company.

[mgottesfeld]

Thank you for the clarification as to candidates' legitimate uses of the sandbox.  I totally misunderstood your first paragraph until now!

Indeed, we stand agreed on the major points.  I was unaware of the natures of some of the commercial exploitations of the platform (though I do think folks who author courses should be able to point folks to the sandbox).  And I know the platform could never possibly scale, for free, to meet what will ultimately be its huge global demand.

I say without overstatement that the DevNet team has built a lifechanging resource for learners.  I thank you for your contributions to it,
by /s/ mgottesfeld.

[Neeraj-Kochar]

I have to admit, although I am very grateful for CML, I am incredibly disappointed with the latest release. the Nexus 9k was the only CML switch with any real advanced feature sets and the only Data Center centric switch in the lineup. As someone who works as a Cisco Data Center engineer this decision has significantly if not completely reduced the value of CML for me. I developed a complete mock up of our Data Center in CML and used it as a modeling and simulation lab prior to making any changes in production, which was incredibly helpful, and is now rendered useless. I truly hope this is not permanent.

[bigevilbeard]

@Neeraj-Kochar you have the option to purchase CML?