Why AI factories suddenly need a whole rack of CPUs

The rack with no GPUs in it

On 22 October, Supermicro announced a 48U rack that contains something you would not have expected in an AI data centre in 2026: no GPUs at all. The SRS-VCPU-256-M0 is a fully liquid-cooled rack built around 256 NVIDIA Vera CPUs — 22,528 cores and 45,056 threads — rated, in Supermicro's own figures, for up to 22,500 concurrent agent sandboxes [1].[1]

Read that number twice, because it is doing a lot of work: twenty-two thousand five hundred isolated execution environments inside a single rack, with no graphics processor in sight. Three years into the GPU-everything era, one of the world's largest AI server vendors just built its flagship agentic-AI product out of CPUs [1].[1]

The reason is not nostalgia. The simple chatbots of a few years ago were linear — a prompt went in, tokens came out, and the GPU did nearly all the thinking. AI agents are different: they plan, write code, run that code, call tools, check results and loop back. Every one of those steps is a small, branching, latency-sensitive piece of work — precisely the kind of work a CPU is built for [1].[1]

Ten days that made a category

Supermicro's announcement did not arrive alone. The day before, at NVIDIA's GTC Berlin event on 21 October, Dell unveiled its own version: the 48U PowerEdge M9882, also packing 256 NVIDIA Vera CPUs for agentic workload execution — orchestration, tool use, code execution and evaluation — with availability expected in the first quarter of 2027 [2].[2][4]

Ten days earlier, AMD had already staked its claim at the OCP Global Summit. Helion, AMD's open rack blueprint for agentic AI, includes a CPU-only configuration with up to 32,768 Zen 6 “Venice” EPYC cores — a larger CPU count than either Vera rack — alongside an enterprise configuration pairing EPYC nodes with up to 16 Instinct MI350P GPUs [4]. Three vendors, three announcements, one pattern: the rack-scale CPU system, purpose-built for agents, has become a product category almost overnight [2][4].[2][4]

What is new here is not CPUs in the data centre — every GPU server ships with host processors. What is new is the CPU rack as the main event: hundreds of server-class CPUs, their own memory fabric, their own networking and security architecture, sold as the agentic-AI machine rather than as the GPU's sidekick [2][4].[2][4]

What an agent sandbox actually is

To see why agents need racks of CPUs, consider what an agent does all day — and where. It writes a script and runs it. It queries a database. It opens a browser, clicks through pages, fills in forms. It calls an external API, waits, parses the response and decides what to do next. Every one of those actions has to happen somewhere safe [6].[1][6]

That somewhere is a sandbox: an isolated execution environment — typically a lightweight container or virtual machine — where the agent can run code and touch the outside world without touching anything else. If the agent's code goes wrong, or a prompt-injection attack tricks it into something hostile, the damage stays inside the sandbox [1].[1][6]

Now multiply by an enterprise running thousands of agents, each spawning several sandboxes, each needing its own slice of compute, memory and network. Supermicro's stated rationale makes this explicit: agentic AI turns linear inference chains into branched, looped execution with many more intermediate steps — and those steps depend on CPU availability. GPUs still generate the tokens, code and queries; CPU-based sandboxes execute the actions, evaluate results, run tools and orchestrate workflows [1][6].[1][6]

Why GPUs can't do this job

This trips people up, because for years the story has been that GPUs are simply better at AI. True — for what GPUs are good at: massive, regular, parallel arithmetic, the matrix multiplications at the heart of neural-network inference. Give a GPU ten thousand identical calculations and it blazes through them [6].[1][5][6]

But a sandbox is not ten thousand identical calculations. It is a messy sequence of different things: spawn a process, open a socket, parse some JSON, enforce a policy check, kill a runaway thread. That is branching, interrupt-driven, operating-system-shaped work — full of syscalls and unpredictable control flow. GPUs have no operating system, no fast context switching and no talent for one fiddly task at a time; CPUs were designed for exactly this [1][6].[1][5][6]

The industry's own rule of thumb tells the story. In standard language-model deployments, one CPU comfortably supports four to eight GPUs — the CPU just feeds the beast. In agentic configurations that ratio is collapsing toward 1:1, because the orchestration layer — routing tool calls, querying databases, triggering APIs, enforcing security policy — has become a first-class workload rather than an afterthought [5][6].[1][5][6]

The sandbox math

The arithmetic on Supermicro's headline figure reveals what the rack is really for. 256 Vera CPUs with 88 “Olympus” cores each — 22,528 cores, 45,056 threads — rated for up to 22,500 concurrent agent sandboxes [1][3]. Divide threads by sandboxes and you get almost exactly two threads per sandbox: a thin slice, enough for a lightweight container running scripts and API calls, but not for heavy lifting — which is the point, because heavy lifting happens on GPUs elsewhere [1].[1][3]

The memory story is more generous: up to 400 TB of SOCAMM2 LPDDR5X memory, 1.2 TB/s of bandwidth per CPU — 13.6 GB/s per core — for 300 TB/s across the rack [1]. Agent sandboxes are memory-hungry in a particular way: each holds conversation context, tool schemas, retrieved documents and intermediate results, and thousands of sandboxes means thousands of copies of that working set. Bandwidth matters more than capacity, and 300 TB/s is a serious number [1].[1][3]

One honest caveat: Supermicro discloses no memory allocation, workload definition or software configuration behind the 22,500-sandbox rating. Until a reference configuration appears, treat it as a sizing ceiling for lightweight sandboxes — a useful upper bound, not a guaranteed workload [1].[1][3]

OpenShell and Sentry: treating agents as untrusted code

The most interesting part of the Vera rack may not be the silicon — it is the security model. Tens of thousands of agent sandboxes means tens of thousands of pieces of untrusted code: agents that browse the web, execute downloaded scripts and call third-party APIs are exactly what an attacker would love to subvert [1].[1]

NVIDIA's answer, detailed at GTC Berlin the same week, is a two-layer architecture: OpenShell plus Sentry. OpenShell is a reference architecture running on the Vera CPUs, applying least-privilege, policy-based runtime controls — each agent gets only the permissions its task requires, and every action is checked against policy as it happens [1].[1]

Sentry is the cleverer half. It runs on the rack's 64 BlueField-4 data processing units as an out-of-band monitor. Sitting on the DPU rather than the host CPU, it enforces policy, collects attested telemetry and quarantines an agent that steps outside its boundaries in milliseconds — without consuming any host CPU resources [1]. The security layer cannot be starved by the workload it watches, because it does not share that workload's processor. That only works because the DPU is a separate computer with its own view of the system [1].[1]

The plumbing: memory, network and power

A rack of 256 CPUs is, before anything else, a plumbing problem. NVIDIA's reference specification calls for 64 BlueField-4 DPUs across the rack — Supermicro places four per compute tray, at 200 Gb/s per CPU — on an NVIDIA Spectrum-X Ethernet converged fabric with up to 400 Gb/s of north-south bandwidth [1]. The layout: 32 one-unit trays, each with four dual-CPU nodes — eight Vera CPUs per tray — inside a third-generation NVIDIA MGX chassis [3].[1][3]

Liquid cooling is not optional at this density; air cannot move heat out of a rack this full. The rack is 100% liquid-cooled throughout, though air-cooled 1U and 2U Vera systems exist in the broader portfolio for sites that cannot take liquid — the same CPUs, minus the plumbing commitment [1].[1][3]

Power gets its own innovation: the third-generation MGX design offers an 800 VDC sidecar option letting battery backup units sit in the rack instead of a room-scale UPS. Supermicro says this frees floor space, can improve end-to-end power-conversion efficiency, and lowers capital and operating expenditure [1]. Higher-voltage DC distribution has been discussed for years; its arrival in a shipping product line suggests the economics have finally caught up with the theory [1].[1][3]

Deploy it like any other rack

One detail that matters for adoption: the Vera CPU rack is designed to deploy standalone or alongside Supermicro's Vera Rubin NVL72 — the GPU rack it complements — sharing the same MGX infrastructure and DLC-2 liquid-cooling stack [1]. A data-centre operator does not want a science project; they want a rack that fits the power, cooling and management they already bought. Supermicro says the NVL72 is now shipping, so the CPU rack lands in an existing ecosystem rather than demanding a new one [1].[1][7]

And the rack is, in Supermicro's words, available to quote now — though no ship date has been given, and “available to quote” is a sales milestone, not a delivery one [1]. The pitch from NVIDIA's enterprise leadership is unusually specific for launch marketing. “As enterprises scale agentic AI and reinforcement learning, supporting thousands of concurrent agent workflows requires a new class of high-density CPU infrastructure,” said Chris Marriott, NVIDIA's VP of enterprise platforms — a vendor's framing, but one that names the bet precisely [7].[1][7]

Dell's version — and the money behind it

Dell read the same playbook differently. Where Supermicro led with full specifications, Dell announced the PowerEdge M9882 at GTC Berlin with the headline facts — 256 Vera CPUs, 48U, agentic workload execution — a Q1 2027 ship window, and no published memory or networking configuration beyond NVIDIA's reference spec [2].[2][8]

Dell's news was broader than one rack: the PowerEdge XE9810, a rack-scale inference system built on Groq 3 LPUs across 16 liquid-cooled trays, and the air-cooled two-socket Vera server PowerEdge R8822. The through-line is a new tier in Dell's AI Factory sitting structurally between raw GPU clusters and general servers — and Dell says the platform has passed 6,500 customers, up from 5,000 the prior quarter [2].[2][8]

The demand signal is moving balance sheets. On 20 October, Supermicro raised its fiscal Q1 2027 guidance to $18.5–19.5 billion in revenue, up from $14.5–15.5 billion — a roughly $4 billion lift attributed “primarily” to AI platform demand — with gross margin guidance up to 12.0–12.5% [8]. These are preliminary, unaudited figures ahead of the 3 November earnings call, but the direction is unmistakable [8]. Dell's materials, meanwhile, cite analyst projections — Goldman Sachs and Gartner — that agentic AI will drive 24x token consumption while inference costs fall more than 90% by 2030: vendor-cited figures, not independently audited, but they explain why everyone is building for vastly more, vastly cheaper inference [2].[2][8]

AMD's open alternative

The third entrant takes a different road. AMD's Helion, announced at the OCP Global Summit, is not a product but an open rack blueprint — a specification any partner can build to, which AMD plans to contribute to the Open Compute Project [4].[4][6]

The headline configuration is the CPU-only one: up to 32,768 Zen 6 “Venice” EPYC cores per rack for agent workloads. But Helion is a family of profiles — dense 400W for personal agents, balanced 600W for enterprise, 96-core parts up to 5.0 GHz for technical agents — plus an enterprise configuration pairing EPYC nodes with up to 16 Instinct MI350P GPUs at a claimed 74 petaflops of FP4 [4].[4][6]

Two Helion ideas stand out. The first is an integrated token router directing agents' model calls across frontier and open-source models — the rack decides which model answers which subtask, so operators can scale CPUs without buying more GPUs [6]. The second is openness itself: named partners include Cisco, Dell, HPE, Lenovo, Supermicro, Hon Hai, QCT and Wiwynn, with partner configurations expected in November 2026 [4]. NVIDIA's ecosystem is a walled garden with excellent plumbing; AMD bets some buyers would rather own the blueprint. Both bets can be right.[4][6]

One caveat: AMD's claim of up to 102 Tbps of networking comes with no published methodology — the company has not explained how it is calculated — so treat it as a headline until the math appears [4].[4][6]

What this means for the CPU market

Step back and the pattern is bigger than three launches. For a decade, the data-centre CPU was in quiet retreat: every AI announcement was a GPU announcement, and the CPU's job was to stay out of the way. The agent era is reversing that — not because CPUs got faster at AI math, but because the workload grew a dimension GPUs cannot serve [6].[2][6]

The ratio tells the story: one CPU per four to eight GPUs was the standard recipe, and agentic configurations are pushing toward 1:1. That implies that in an agentic data centre, roughly half the silicon budget could be CPUs. For Intel and AMD's server businesses, which spent years watching NVIDIA capture the AI spend, this is the most interesting market development in a long time — a new, growing, CPU-shaped hole in the AI factory [6].[2][6]

It also reframes the custom-silicon wave. Recent attention has gone to accelerators designed to complement GPUs — the ASIC boom covered in our look at ByteDance's MediaTek talks (https://technogalaxy.tech/blogposts/why-bytedance-is-asking-mediatek-to-build-its-ai-chips/). The CPU rack is the other half of that story: while custom silicon attacks the inference cost problem, the CPU rack attacks the orchestration problem. An AI factory in 2027 may be a sandwich — custom accelerators for tokens, server CPUs for everything the agents actually do [6].[2][6]

NVIDIA is notably playing both sides: Vera puts it in the CPU business in a serious way for the first time, competing with the Intel and AMD sockets it used to sit next to — a move our RTX Spark coverage (https://technogalaxy.tech/blogposts/nvidias-rtx-spark-the-ai-superchip-invading-intel-and-amds-turf/) traced from the desktop to the rack [2].[2][6]

What to watch

Three dates matter. First, 3 November, when Supermicro reports full quarterly results — the preliminary guidance raise will be confirmed or qualified, and analysts will ask how much of the $4 billion lift is Vera-shaped [8]. Second, November 2026, when AMD's Helion partner configurations are expected — the moment the open blueprint meets real hardware [4]. Third, Q1 2027, Dell's ship window for the PowerEdge M9882 [2].[1][2][4][6][8]

Beyond the calendar, watch for reference configurations. The 22,500-sandbox rating is a ceiling until Supermicro publishes the workload and software stack behind it; the first independent sizing guides will show what these racks actually do per dollar [1]. Watch whether OpenShell attracts an ecosystem — a security architecture is only as good as the policies written for it, and the DPU-based Sentry model needs third-party tooling to reach its potential [1].[1][2][4][6][8]

And watch the ratio. If 1:1 CPU-to-GPU holds up in real deployments, the data-centre CPU market is about to have its biggest growth story in a decade — built not on beating the GPU, but on doing the work the GPU never could [6].[1][2][4][6][8]