There’s been no shortage of talk lately about the “AI-ready” data center. But beneath the hype, the technical requirements for supporting enterprise-grade AI workloads are far more demanding than many teams realize. Whether you're working with open-source models, fine-tuning LLMs, or building agent-based systems, the real blocker usually isn’t the model. It’s the infrastructure.
Our Cisco + NVIDIA BrightTALK on the Secure AI Factory offers pragmatic view of what it actually takes to get from POC to production. Here we discuss a few technical requirements that the AI-Ready Data Center must deliver to be able to support the workloads of today, and to position organizations to be able to expand into workloads of tomorrow.
1. AI Workloads Demand High-Throughput, Low-Latency Networking
Training and inference pipelines, especially those that span multiple nodes, rely on dense east-west traffic. You need:
-
400G+ switching fabric for backend interconnect
-
High radix topologies (leaf-spine or folded Clos) to minimize oversubscription
-
RDMA over Converged Ethernet (RoCEv2) support for model parallelism
-
Deterministic latency under load to ensure consistent performance across epochs
Traditional enterprise networks, even those built for virtualization or big data, tend to fall short here. AI networks require not just bandwidth, but predictable behavior under massive distributed I/O. Many focus on the compute aspects of AI - i.e. GPUs and other ASIC accelerators as the crux of AI delivery, but in modern deployments, especially those focused on Inference where test-time compute is critical to operations, AI-ready networking is pivotal.
2. Data Locality Is Non-Negotiable
Most enterprise AI workloads can’t move all data to the cloud. Whether due to data sovereignty, latency, or sheer volume, AI needs to run where the data is.
This introduces the need for:
-
AI-capable compute and storage within enterprise-controlled facilities
-
High-performance object or NFS storage tuned for parallel I/O patterns
-
Secure multi-tenant access models to support mixed development and production environments
A modern AI stack must treat storage, networking, and compute as a tightly coupled system—one that can deliver 10s of GB/s to GPU memory without bottlenecks.
3. Security Must Cover Application, Workload, and Infrastructure Layers
LLMs and multi-agent systems introduce new attack surfaces that go beyond traditional CVEs. We now have to think about:
-
Application layer: Preventing jailbreaks, prompt injection, and off-policy actions
-
Workload layer: Enforcing process-level controls, isolating GPU tenants, stopping lateral movement
-
Infrastructure layer: Encrypting east-west traffic, segmenting networks, and applying zero trust policy consistently across physical and virtual assets
This isn't just theory, there are real-world examples of companies getting fined for actions taken by ungoverned digital agents and AI workloads, especially consumer-facing Chatbots. Enterprises need runtime AI behavior monitoring and low-level network enforcement working in tandem.
4. Observability Is a Core Requirement, Not a Bonus
You can't troubleshoot or optimize what you can’t see, and visibility is critical for performance in enterprise workloads. For AI infrastructure, observability must span:
-
Telemetry from GPUs (utilization, memory bottlenecks, thermal throttling)
-
Network metrics (packet drops, retransmits, ECN marks)
-
Application-level insights (inference latency, queue depth, batch sizes)
-
Security events (model misuse, unauthorized data access)
A modern AI platform should tie all of this together across vendors, preferably with native integrations instead of post-hoc log scraping.
5. Scalable Deployment Requires Modular and Integrated Options
AI projects typically start small and attempt to scale fast, often faster than the infrastructure team is ready for. To avoid hitting a wall, orgs need:
-
Modular designs: GPU + storage + switching in repeatable units, ideal for R&D and dev clusters
-
Vertically integrated clusters: Pre-validated stacks with automated provisioning and lifecycle management for scale-out environments
-
Ethernet-based switching solutions to realize the benefits of scalable, modular infrastructure
The ability to evolve infrastructure incrementally, without forklift upgrades, is critical to sustaining ROI as use cases expand and enterprises move to realize their AI goals.
Bringing It All Together
The real takeaway is that the hard parts of scaling AI (network determinism, security enforcement, multi-domain observability, and modular growth) aren’t going away and they don't get easier with scale. Enterprises need a blueprint to move out on that is scalable and delivers end-to-end security from inception. The good news is that Cisco has delivered that framework with the Secure AI Factory.
Whether you’re using Cisco+NVIDIA or another vendor stack, any serious enterprise AI effort needs to meet these same technical benchmarks. Pre-valided infrastructure from a single-unified vendor, meeting all the above requirements, is a significantly more seamless and manageable way to realize value from AI initiatives.
--
Levi D. McClenny, Ph.D.
Sr. Manager, WW AI Solutions Engineering
