NVIDIA Vera Rubin NVL72 Makes MLPerf Debut With 3.7x Throughput Lead Over Blackwell

MLCommons published MLPerf Inference v6.1 results today, and NVIDIA's Vera Rubin NVL72 entered the record for the first time with preview submissions on two of the benchmark suite's most demanding workloads — delivering up to 3.7 times the throughput of the current-generation GB300 NVL72 on Qwen3-VL and up to 2.5 times on DeepSeek-R1, according to NVIDIA's blog post published September 16, 2026. The results, produced within the MLCommons consortium's peer-reviewed methodology rather than through NVIDIA's own testing, give hyperscalers and cloud operators their first externally validated data point for Vera Rubin's inference performance ahead of a general commercial rollout.

The numbers cover more than a single-rack comparison. In a separate submission, four GB300 NVL72 racks running as a 288-GPU cluster achieved 99% scaling efficiency on DeepSeek-R1 in the offline scenario — meaning throughput grew nearly in direct proportion to the added hardware. On the software side, continued optimization of the GB300 NVL72 stack lifted Qwen3-VL performance by up to 1.6 times compared with the same platform's April 2026 MLPerf Inference v6.0 results, demonstrating that the benchmark gap between generations keeps widening even before the next generation ships. NVIDIA Jetson AGX Thor also submitted results on a newly introduced Edge-Agentic benchmark, extending the measurement framework to on-device agentic inference for the first time.

What the Benchmark Results Actually Say — and What They Don't

MLCommons, the open engineering consortium behind MLPerf, governs its inference benchmarks through a peer-reviewed submission process: hardware vendors must run defined workloads under specified conditions, and results undergo accuracy validation before publication. That methodology matters here because the Vera Rubin NVL72 numbers are "preview" submissions — the hardware is production-ready, but NVIDIA's software stack for the platform is still under active development. NVIDIA explicitly notes that performance will continue to improve beyond what v6.1 captures.

NVIDIA cited MLPerf Closed Division entries 6.1-0106 and 6.1-0074 as the basis for the comparison, with the Qwen3-VL workload using vLLM with NVIDIA Dynamo and the DeepSeek-R1 workload using TensorRT-LLM. Separately, Nebius — the cloud provider that has been an early Vera Rubin adopter — submitted its own Vera Rubin NVL72 preview results and showed comparable performance, providing a second data point from an operator independent of NVIDIA's internal benchmarking team.

The throughput ratios should also be understood in the context of what the benchmarks measure. MLPerf Inference v6.1's data center suite covers offline, server, and interactive scenarios for large language models and vision-language models. The offline scenario maximizes throughput with unlimited batch size; the server scenario adds latency constraints; the interactive scenario enforces tight time-to-first-token requirements. The Vera Rubin 3.7x advantage on Qwen3-VL is reported across all three scenarios — meaning it holds under latency constraints, not just maximum-throughput conditions, which is the more operationally relevant result for production deployments.

How Vera Rubin Achieves Higher Throughput: Disaggregated Serving, NVFP4, and MoE Parallelism

The throughput gains are not primarily the product of faster raw silicon, though Vera Rubin does represent a substantial architectural step up from GB300. The performance improvement results from a combination of hardware features and serving techniques that were designed together.

The central serving technique is disaggregated serving, which NVIDIA runs through its Dynamo open-source inference framework. In conventional aggregated serving, one GPU handles both phases of LLM inference: prefill and decode. During prefill, the system processes the input prompt and builds a key-value cache that represents the conversation state. During decode, the model generates one token at a time, repeatedly reading the accumulated key-value cache. The two phases have opposite computational characteristics: prefill is compute-intensive and benefits from high GPU parallelism; decode is memory-bandwidth bound and benefits from large, fast memory. Running both on the same hardware means neither phase uses the GPU optimally.

Disaggregated serving separates the two phases into dedicated worker pools. Prefill workers build the key-value cache; NIXL — NVIDIA's inter-GPU exchange library — transfers the key-value state over NVLink or InfiniBand to decode workers, which continue token generation without competing with incoming prefill traffic. The result is better hardware utilization across the pool and higher sustainable throughput, particularly for long-context requests that would otherwise stall decode queues while a heavy prefill completes. For Vera Rubin's Qwen3-VL test, vLLM served as the model server backend with Dynamo managing the prefill-decode routing above it.

The other key mechanism is NVFP4 precision. NVFP4 is NVIDIA's 4-bit floating-point format, using a two-level scaling scheme: E2M1 values (one sign bit, two exponent bits, one mantissa bit) are grouped into blocks of 16 elements, each sharing an FP8 (E4M3) block scale. Unlike uniform integer 4-bit quantization, which treats all values within a block identically, the floating-point semantics give NVFP4 wider dynamic range and better accuracy recovery than INT4 at equivalent bit width. The practical effect is that model weights, activations, and the key-value cache can all be stored and processed at 4-bit precision — halving the memory footprint versus FP8 while keeping output quality within benchmark accuracy thresholds. NVIDIA notes that NVFP4 precision reduces memory footprint across model weights, attention, and key-value cache — increasing throughput with minimal loss of output quality.

Both benchmark models — DeepSeek-R1 and Qwen3-VL — are mixture-of-experts architectures. In a dense transformer, every parameter is used for every token. In a mixture-of-experts model, a routing layer selects a small subset of specialized expert networks for each token and sends the token's activations to those experts only. DeepSeek-R1's 671 billion total parameters reduce to approximately 37 billion active parameters per token; Qwen3-VL's 235 billion total reduce to 22 billion active. Mixture-of-experts models can pack more capacity into a given compute budget, but expert routing creates all-to-all communication patterns across GPUs — each token may need to reach a different expert on a different GPU. NVIDIA's large-scale expert parallelism technique distributes expert networks across the 72-GPU NVLink domain, using the NVLink 6 fabric's all-to-all bandwidth per GPU to move expert dispatches without Ethernet overhead.

The 99% Scaling Efficiency Number Is the Understated Result

Most of the attention in the Vera Rubin v6.1 announcement will land on the 3.7x throughput headline. The 99% multi-rack scaling efficiency deserves equal weight for anyone planning AI infrastructure at production scale.

Adding hardware to a distributed inference system does not automatically produce proportional throughput gains. Cross-rack communication introduces latency; request orchestration adds overhead; imbalanced GPU utilization wastes capacity. If doubling the GPU count delivers only a 30% improvement in throughput, the economics of expansion break down: the hardware cost grows linearly while the revenue-generating capacity grows sublinearly. At 99% scaling efficiency, GB300 NVL72 effectively converts each marginal GPU addition into a marginal throughput addition — which is the necessary condition for AI factories to scale linearly in both capacity and cost.

NVIDIA's demonstration ran DeepSeek-R1 from a single GB300 NVL72 rack at 72 GPUs to four racks at 288 GPUs, achieving near-proportional throughput growth in the offline scenario. The architecture enabling this is NVLink's rack-to-rack fabric combined with high-bandwidth InfiniBand for cross-rack communication, with NVIDIA's software managing request distribution and key-value cache routing across the expanded cluster. This four-rack, 288-GPU configuration — which NVIDIA first demonstrated at MLPerf Inference v6.0 — now shows 99% scaling efficiency in v6.1.

The multi-rack scaling result is also a signal that the Vera Rubin era's interconnect investments were not theoretical. The GB300 to Vera Rubin transition doubles NVLink bandwidth from 130 to 260 terabytes per second per rack; showing that even the current GB300 system achieves 99% efficiency at 288 GPUs establishes the baseline that Vera Rubin will need to match — or exceed — as cluster sizes grow.

Software Optimization Is Still Compounding the Generational Lead

A less-discussed aspect of the v6.1 results is that the existing GB300 NVL72 generation has continued improving through software after its MLPerf debut. The platform's Qwen3-VL performance increased by up to 1.6 times between the April 2026 v6.0 submission and today's v6.1 submission, with NVIDIA attributing the gains to lower key-value cache precision, additional kernel fusion, better inference kernels, and the rollout of disaggregated serving with vLLM and Dynamo.

Software optimization compounding on fixed hardware has been a consistent feature of NVIDIA's MLPerf history. During the Hopper generation, the H100's MLPerf performance improved by more than 6x from its debut submission to the final round before Blackwell replaced it. This matters for infrastructure planning: the Vera Rubin NVL72 preview results shown today represent an early point on what will likely be an improving curve. NVIDIA itself notes that post-submission results on GPT-OSS-120B and DLRMv3 show further gains, though those figures have not yet been verified by MLCommons.

Read more: Nvidia Posts First Vera Rubin Benchmark Data: 30x Efficiency Gain on Agentic Workloads

MLPerf Benchmarks in Context: What They Measure, and Their Limits

MLPerf Inference benchmarks measure throughput, latency, and minimum quality in defined scenarios under controlled conditions. They are not cost-per-token benchmarks, energy efficiency benchmarks, or real-world serving benchmarks — though the results can be used to calculate relative economics when combined with published power figures.

The Qwen3-VL workload in v6.1 is based on the 235-billion-parameter vision-language model and tests multimodal inference combining image and text inputs. The DeepSeek-R1 workload tests reasoning-capable LLM inference at 671 billion total parameters. Both represent frontier-scale production workloads that real operators deploy. The benchmark's closed division requires that output quality meet specified accuracy thresholds — results cannot be achieved by degrading output quality below the minimum standard.

The key limitation of the v6.1 Vera Rubin results is the preview classification. NVIDIA chose to submit before the platform's software reached its projected optimum, which reflects confidence in the hardware but means the comparison to GB300's mature software stack is not entirely symmetric. The expected trajectory is that Vera Rubin's numbers improve in v6.2 and subsequent rounds as TensorRT-LLM, vLLM, and Dynamo optimization for Rubin matures. Benchmark results from a platform at a similar stage of its lifecycle — GB200 NVL72's debut numbers, which later improved substantially — suggest the current figures are a floor rather than a ceiling.

Vera Rubin vs. the Competitive Landscape

The MLPerf v6.1 round does not include AMD's Helios rack, which pairs 72 MI455X accelerators with EPYC Venice CPUs and 31 TB of HBM4 memory. AMD rates Helios at 2.9 exaFLOPS of FP4 inference compute, below Vera Rubin NVL72's 3.6 exaFLOPS NVFP4. AMD has not submitted Helios to MLPerf benchmarking as of this writing, meaning there is currently no apples-to-apples MLPerf comparison between the two rack-scale systems.

At the GPU level, AMD's Instinct MI355X submitted to MLPerf Inference v6.0 in April 2026 and reached within single-digit percentage points of NVIDIA's B200 on several server inference workloads — the closest the gap has been in the benchmark's history. Whether that competitive position extends to the Helios rack-scale system under the same disaggregated serving techniques remains unresolved by public benchmark data.

The competitive significance of the v6.1 results is therefore primarily in the GB300 to Vera Rubin generational step, which now has benchmark-backed quantification, rather than NVIDIA versus AMD at equal hardware generations. Hyperscalers deciding between maintaining GB300 capacity and transitioning to Vera Rubin can now factor audited throughput ratios into that decision rather than relying solely on NVIDIA-generated numbers from its own testing environment.

It is also worth noting what MLPerf's historical record shows about generational improvement trajectories. When NVIDIA GB200 NVL72 submitted its first MLPerf results in early 2025, it posted up to 30x higher throughput than prior H100-based systems on certain workloads. As software matured on the same hardware across subsequent rounds, performance continued climbing. The GB300 NVL72 — Blackwell Ultra — then delivered another 45% per-GPU improvement over GB200 on DeepSeek-R1 when it debuted at MLPerf Inference v5.1 in September 2025. Today's GB300 software showing a 1.6x gain over its own April 2026 baseline illustrates that the curve has not flattened. This trajectory matters for anyone comparing the value of deploying Vera Rubin now versus waiting: the historical evidence suggests early-generation preview numbers underrepresent where a platform ends up, making the present 3.7x comparison a conservative floor rather than a stable ceiling.

Jetson AGX Thor and the Edge-Agentic Benchmark Debut

The v6.1 round also introduced MLPerf's first Edge-Agentic benchmark, designed to measure the kind of on-device inference that runs tools, processes sensor inputs, and executes multi-turn reasoning loops on edge hardware without cloud connectivity. NVIDIA submitted Jetson AGX Thor results on this benchmark using TensorRT Edge-LLM with the Qwen3.6-27B model.

The MLCommons Edge-Agentic benchmark specification focuses on multi-turn agentic coding inference at long contexts, replaying recorded interaction traces that include tool calls, code execution, and growing conversation histories — patterns that simple text generation benchmarks do not capture. For edge deployments in robotics, industrial automation, and on-device AI agents, this benchmark represents a more realistic measure of what the hardware must sustain than the fixed-sequence tests that previously defined edge inference performance.

Jetson AGX Thor's 2,070 FP4 teraflops of compute make it the most powerful edge device in NVIDIA's embedded lineup, designed specifically for physical AI — robots, autonomous systems, and always-on agents that cannot rely on round-trip cloud latency for each reasoning step. The debut in v6.1 aligns with the MLCommons Edge LLM Taskforce's stated priority of bringing standardized measurement to agentic edge workloads. Results from 19 NVIDIA ecosystem partners appeared across the broader v6.1 submission set, with eight partners submitting multi-node Blackwell NVL72 systems.

The Edge-Agentic benchmark's use of Qwen3.6-27B — a 27-billion-parameter model running at Q4_K_M quantization on a single edge accelerator — is a deliberate design choice. A 27-billion-parameter model at 4-bit quantization fits within the memory constraints of high-end edge hardware, while its instruction-following and tool-use capability is sufficient to run real coding-agent tasks. The Berkeley Function Calling Leaderboard v4 provides the accuracy gate: the system must correctly invoke external functions and parse their results above a threshold score before performance numbers count. This two-part structure — deterministic accuracy gate, then performance measurement — is a departure from LLM generation benchmarks that simply measure token generation speed. For edge deployments where an agent calling the wrong API has operational consequences, accuracy-gated measurement reflects actual deployment requirements better than throughput-only numbers.

Inference Economics and What the Data Center Sector Watches Next

The throughput gains documented in MLPerf Inference v6.1 translate directly into token economics. In AI factory deployments, throughput determines revenue per rack: a system generating 3.7 times more tokens per second on the same model, in the same power envelope, produces 3.7 times more revenue per hour at the same token price. Higher throughput also enables lower cost-per-token pricing, which expands the addressable user base for model providers. This is why NVIDIA frames inference benchmarks in terms of more tokens generated, more users served, and more revenue rather than abstract compute counts.

The 99% multi-rack scaling efficiency complements this picture: it means the marginal cost of adding inference capacity is approximately equal to the marginal cost of the additional hardware, without the sublinear returns that would otherwise force operators to over-provision.

Two near-term milestones will extend what today's MLPerf data establishes. First, SemiAnalysis will conduct its independent verification of the Vera Rubin v6.1 results under its InferenceX methodology — the same benchmark framework that previously documented agentic efficiency figures for the platform. Agreement between the MLCommons-audited numbers and the SemiAnalysis InferenceX methodology would significantly strengthen the case for the Vera Rubin transition economics. Second, MLCommons is developing the MLPerf Endpoints benchmark, which will measure AI inference at the API service level — capturing end-to-end latency, throughput under real traffic patterns, and cost per query — rather than at the hardware layer. That benchmark will be the next standardized test of whether the throughput gains documented today translate into better outcomes for the developers and organizations calling the APIs.

The benchmark era defined by fixed-sequence GPU throughput tests is giving way to one where agentic workloads, multi-rack scaling, and energy efficiency per token carry equal weight. Today's MLPerf Inference v6.1 results position Vera Rubin NVL72 at the head of the transition — with the caveat that the platform's software remains in active development, and the numbers published today represent a starting point on an improvement curve that the benchmark's own history suggests will keep climbing.