Nvidia Posts First Vera Rubin Benchmark Data: 30x Efficiency Gain on Agentic Workloads

Nvidia released the first on-silicon benchmark data for its Vera Rubin NVL72 rack-scale system at Hot Chips 2026 on Monday, showing the platform delivers up to 30 times higher AI-factory throughput per megawatt than the current-generation GB300 NVL72 on agentic coding workloads — a figure that comes with important caveats but signals a genuine shift in how inference hardware is designed and evaluated. On the same day, Groq 3 LPX, its dedicated low-latency inference accelerator, entered full production, with cloud provider Nebius signed as the first customer. SpaceXAI separately committed to deploying Nvidia's purpose-built Vera CPU as the orchestration layer powering its Grok agentic AI infrastructure, and confirmed plans to extend that same hardware stack into orbit aboard a first-generation Starmind satellite.

Taken together, the three announcements describe an infrastructure thesis that Nvidia has been building since its $20 billion acquisition of Groq's assets in December 2025: that agentic AI workloads are structurally different from conventional language-model inference, and that serving them well requires three distinct processor types working in concert — GPU, LPU, and agent CPU — rather than a single chip doing everything.

Agentic AI Has Broken the Standard Inference Benchmark Model

The benchmark underlying Nvidia's efficiency claims is AgentX, an open-source benchmark from SemiAnalysis that is part of its InferenceX suite. AgentX replays pre-recorded Claude Code coding agent sessions — real production-style traces that include tool calls, code execution, sub-agent spawning, and context windows that grow from roughly 60,000 tokens to 400,000 tokens or more over the course of a session.

The design matters because an agent does not sit still. Each turn in an agentic session can be substantially longer or shorter than the last, KV cache from earlier turns can be reused, and model calls are interrupted by tool-execution gaps where no inference is happening at all. A benchmark using a fixed sequence length — say, 8,000 input tokens and 1,000 output tokens — says nothing about how the system performs when request shape varies continuously across hundreds of sequential calls. Nvidia's own technical blog notes that traditional fixed-sequence benchmarks have been demoted to "maintenance mode" on the InferenceX platform precisely because of this mismatch.

The demand-side driver is real: according to Nvidia, citing OpenRouter data analyzing 100 trillion real-world tokens, average prompt tokens per request have grown roughly fourfold, with a single agentic request consuming 15 times the tokens of an ordinary chat interaction.

The 30x Claim: Performance Gain at the Frontier, Not a Uniform Multiplier

Nvidia's Vera Rubin NVL72 results on the AgentX workload were measured by Nvidia using DeepSeek V4-Pro, a 1.6 trillion-parameter mixture-of-experts model, and are described as pending SemiAnalysis review — meaning the benchmark has not yet been independently confirmed at time of publication.

The efficiency advantage is not a flat multiplier. At 110 tokens per second per user — a relatively standard interactivity target — Vera Rubin NVL72 delivers roughly twice the throughput per megawatt of GB300 NVL72. At 130 tokens per second per user, the advantage grows to approximately 10 times. At 160 tokens per second per user, the gap widens to 30 times. Vera Rubin NVL72 can sustain responsive performance at interactivity levels where GB300 NVL72 has effectively run out of headroom, and it does so more efficiently per unit of power — which is the operationally relevant metric for data centers constrained by power infrastructure.

The GB300 NVL72's own position on the efficiency curve is also substantial: on AgentX with DeepSeek V4-Pro, it delivers up to 15 times the throughput per megawatt of the prior-generation H200 NVL8. Against Kimi K3, a 2.8 trillion-parameter MoE model, GB300 NVL72 reaches up to 80 times the throughput per megawatt of H200 NVL8. Nvidia also says its DSX MaxLPS power management technology allows up to 40 percent more GPUs to be configured within the same megawatt budget, layering a power-envelope optimization on top of the architectural efficiency gains.

AFD: The Architecture That Separates Two Different Compute Problems

The performance gains are not primarily the product of a faster GPU chip. They result from Attention-FFN Disaggregation, or AFD — a technique that physically separates the two computationally distinct operations in the transformer decode loop and routes them to different hardware.

During each decode step, attention over the accumulated KV cache is heavily memory-bandwidth-bound and benefits from the large HBM capacity of GPU memory. Feed-forward network execution — and in MoE models, the routing of each token to a subset of specialized expert networks — involves smaller working sets that are better served by the LPU's high-bandwidth on-chip SRAM. AFD sends these to two different processors per token, exchanging intermediate activation tensors over the platform's interconnect fabric, with Nvidia's Dynamo serving framework orchestrating the routing, session tracking, and KV-cache-aware load balancing.

At rack scale, the Vera Rubin NVL72 connects 72 Rubin GPUs and 36 Vera CPUs through NVLink 6, which Nvidia describes as providing 260 terabytes per second of all-to-all fabric bandwidth with no internal copper cables. This cableless midplane design cuts rack assembly time from 100 minutes to five minutes compared to the prior Grace Blackwell generation — a convenience feature on the ground, and a reliability requirement in the orbital context SpaceXAI is pursuing.

Groq 3 LPX: SRAM-Based Decode at 3,400 Tokens Per Second

The Groq 3 LPX is the inference accelerator rack that implements the LPU side of AFD. It packs 256 Groq 3 LPU chips into a single rack across 32 liquid-cooled compute trays. At the rack level, specifications include 315 petaflops of FP8 compute, 128 gigabytes of total SRAM, 40 petabytes per second of on-chip SRAM bandwidth, and 640 terabytes per second of scale-up bandwidth. Each individual LPU chip contains 500 megabytes of on-chip SRAM and provides 150 terabytes per second of on-chip bandwidth, with 96 chip-to-chip links running at 112 Gbps each.

The LPU's architecture deliberately differs from a GPU in one key way: rather than relying on dynamic hardware schedulers at runtime, the Groq LPU uses a compiler-orchestrated spatial execution model where computation, data movement, and synchronization are all explicitly scheduled at compile time. The hardware enforces plesiosynchronous chip-to-chip coordination — aligning 256 accelerators to act as a coherent system with predictable communication timing, eliminating the execution jitter that can compound across hundreds of sequential agent decode steps.

An independent benchmark by Artificial Analysis measured the Groq 3 LPX running Gemma 4 31B at 3,431 output tokens per second with a 100,000-token context window — the highest recorded throughput for that model and approximately four times faster than the next-best platform measured. To illustrate the practical difference: generating 5,000 tokens takes roughly 1.5 seconds at that speed, compared to 50 seconds at a typical throughput of 100 tokens per second. The Groq 3 LPX chips are manufactured by Samsung; Nvidia's Rubin GPUs are produced by TSMC.

Read more: Nvidia's $27 billion pattern of licensing key AI technology

Nebius, whose Token Factory inference platform already serves Cursor, World Labs, Revolut, and Shopify, will deploy Groq 3 LPX before the end of 2026. "Generation is the phase of inference that determines how responsive an AI system actually is, and that's exactly what Groq 3 LPX is built to accelerate," said Danila Shtan, chief technology officer of Nebius. "As the first AI cloud bringing it to production via Nebius Token Factory, we're making sure every step of an agent's loop feels instant — through the same API developers are already using, with no migration to a new stack." Groq Inc., the startup whose technology Nvidia acquired, is planned as a subsequent early adopter of the platform.

Vera CPU Targets the Orchestration Gap Between Inference Calls

The Vera CPU addresses a bottleneck that GPU and LPU benchmarks do not capture: what happens between inference calls. An agent running on a GPU finishes a reasoning step, and then the framework parses output, executes tool calls — code sandboxes, database queries, file operations, API calls — collects results, assembles the next context, and schedules the next inference call. All of that orchestration runs on the host CPU. A GPU that sits idle waiting on CPU-bound orchestration is wasted capital at every step.

Nvidia's analysis of more than 163,000 agentic sessions found that over 97 percent exhibit unique execution profiles — scatter-gather memory access patterns that standard server CPU architectures, tuned for throughput on predictable workloads, do not serve efficiently. The Vera CPU pairs 88 custom Olympus cores with LPDDR5X memory delivering up to 1.2 terabytes per second of bandwidth, tuned for the access patterns agentic orchestration demands.

Internal Nvidia testing from July 2026 found that Vera CPU delivers up to 1.5 times the per-core agentic workload performance of AMD's Venice processor — a figure based on estimated SPEC CPU 2026 results and not independently validated by third parties. AMD has disputed Nvidia's broader Vera CPU performance claims, asserting its 256-core Zen 6 Venice architecture outperforms Vera 3.3 times at the rack level; neither Nvidia's nor AMD's figures have been validated by independent testing at time of publication.

SpaceXAI announced it will deploy Vera CPUs for Grok's agentic AI infrastructure, becoming the second major standalone Vera deployer after Meta. "Vera gives us the CPU performance and memory bandwidth to run enormous amounts of orchestration, code, and data processing while keeping GPUs doing what they do best," said Mike Nicolls, president of SpaceXAI. SpaceXAI also plans to extend the same hardware stack into orbit with its first-generation Starmind AI1 satellite — an optimized Vera Rubin NVL72 system designed for a 120-kilowatt average compute payload — with Elon Musk confirming a Q4 2027 target launch date on X. Both Nvidia and SpaceX classify the orbital deployment as a forward-looking statement carrying no legal commitment to that timeline. Substantial engineering challenges remain unresolved: radiation qualification of commercial silicon for low-Earth orbit, thermal management at power levels exceeding the International Space Station's full thermal capacity, and chip supply constraints that SpaceX has acknowledged.

Integration Is Now the Competitive Moat

No competing vendor currently ships an equivalent integrated GPU-LPU-CPU stack with a unified serving framework at commercial scale. AMD, Intel, Google, and Amazon each address portions of the inference problem but none has announced a production-ready combination analogous to Vera Rubin NVL72 plus Groq 3 LPX plus Dynamo orchestration.

The advantage Nvidia is accumulating is therefore a systems-integration moat: the performance gains from AFD require Dynamo's orchestration; Dynamo depends on NVLink 6's fabric connecting Rubin GPUs and Groq 3 LPUs; the economics of the complete stack depend on all three processor types running coordinated workloads. An AI cloud provider wanting to deliver production-grade agentic inference at competitive latency and power efficiency would need to replicate not just individual chips but the entire layered serving system — a substantially higher barrier to competitive parity than matching any single GPU's throughput figure.

The Vera Rubin NVL72 AgentX benchmark results remain subject to SemiAnalysis's independent review. The independent confirmation of the 3,431 tokens-per-second figure from Artificial Analysis provides a meaningful data point, but the headline 30x efficiency claim for the GPU rack awaits third-party corroboration. That distinction matters less for understanding the direction of the architectural shift than for calibrating the magnitude — and on the direction, the data from both Nvidia and third-party sources points the same way: the benchmark era defined by fixed-sequence GPU throughput tests is over, and the hardware vendors that can demonstrate efficient performance under variable-context, multi-step, tool-interrupted agent workloads will define the next competitive frontier.