CoreWeave Scales Vera Rubin to Multi-Rack Clusters as MLPerf v6.1 Confirms 3.7x Gain

CoreWeave announced on September 16 that it brought up a multi-rack NVIDIA Vera Rubin NVL72 configuration on its cloud platform, connecting hundreds of Rubin GPUs as a single scale-out cluster — the first AI cloud provider to do so. The announcement arrived on the same morning that MLPerf Inference v6.1 results were published by MLCommons, independently confirming for the first time that Vera Rubin NVL72 delivers up to 3.7x higher throughput than NVIDIA's previous-generation GB300 NVL72 on multimodal reasoning workloads. Together, the two developments mark the point at which Vera Rubin transitions from an exciting hardware program to a production-grade cluster platform with third-party performance validation.

CoreWeave also announced two new capabilities in its AI Object Storage service: cross-region write acceleration, which lets training checkpoints land at local NVMe latency while asynchronously replicating to a second region, and an Archive storage tier with no retrieval, deletion, or egress fees.

From Single Rack to Cluster: What the Scale-Out Milestone Actually Changes

A single NVIDIA Vera Rubin NVL72 rack is a remarkable piece of hardware: 72 Rubin GPUs and 36 Vera CPUs connected by a sixth-generation NVLink fabric, with NVIDIA-published specifications listing 20.7 terabytes of HBM4 GPU memory, 1,400 TB/s of memory bandwidth, and 3,600 PFLOPS of sparse NVFP4 inference compute per rack. The rack's cableless midplane design means all 72 GPUs communicate at full NVLink bandwidth with no physical cables between them — a significant reliability advantage over prior generations.

But a rack is still just a rack. For large-scale frontier model training, the jobs that matter often span hundreds or thousands of GPUs simultaneously. When multi-step agentic workloads call a model dozens or hundreds of times in a single session — each call building on the context from previous ones — the infrastructure underneath has to keep every GPU synchronized, fed with data, and connected to every other GPU with enough bandwidth to execute distributed collective operations without waiting.

Before this announcement, Vera Rubin existed on CoreWeave as isolated 72-GPU islands. A model training run that needed more compute than one rack could provide would have faced the same inter-rack networking constraints that slow down any distributed job. The multi-rack configuration announced on September 16 crosses that boundary, presenting hundreds of Rubin GPUs to a running workload as a unified pool.

"With multi-rack Vera Rubin, we are connecting hundreds of Rubin GPUs as a single scale-out cluster," said Chen Goldberg, CoreWeave's executive vice president of product and engineering, in the company's press release. "For customers building agentic AI, that means greater scale, faster iteration, and higher productivity as models and agents continuously learn and improve."

Inside the Multi-Rack Architecture: NVLink 6, Spectrum-X, and the Straggler Problem

The engineering challenge in building a multi-rack cluster is not simply plugging racks together. Within each NVL72 rack, NVLink 6 provides the scale-up fabric — an all-to-all interconnect that allows any GPU to send to any other GPU in the same rack at full bandwidth with sub-microsecond latency. Across racks, the communication path is entirely different: NVIDIA Spectrum-X Ethernet carries the inter-rack traffic using RDMA over Converged Ethernet, a protocol that delivers low-latency, high-throughput networking without the overhead of traditional TCP/IP data transfer.

CoreWeave's implementation equips each Rubin GPU with two NVIDIA ConnectX-9 SuperNICs, yielding 1.6 terabits per second of scale-out connectivity per GPU. The network is designed as a two-tier, non-blocking, multi-rail, multi-plane fabric — meaning traffic between any two GPUs has multiple independent paths through the fabric, and no single link becomes a bottleneck that backs up the entire cluster. The company says its current modular design supports approximately 128,000 GPUs per rail, with additional racks able to join without requiring a fabric redesign. That figure represents a design capacity, not a demonstrated deployment at that scale.

The technical distinction between scale-up and scale-out matters for how workloads perform. NVLink 6 handles the tightly coupled, low-latency communication that happens within a single model's tensor-parallel execution — one GPU sending an activation matrix to an adjacent GPU. Spectrum-X handles the coarser communication that happens in pipeline-parallel and data-parallel training, where gradient updates need to cross rack boundaries. Keeping both paths performing well simultaneously, under sustained load, is what makes multi-rack AI infrastructure practically difficult.

A particularly important failure mode in distributed training is the GPU straggler: one underperforming accelerator that forces every other GPU in a collective operation to wait at a synchronization barrier. A degraded PCIe connection, a marginal thermal case, or a firmware inconsistency can produce a GPU that passes diagnostic tests but runs measurably slower than its peers under load — and in a 1,000-GPU training job, one slow GPU can stall 999 others at every AllReduce step.

CoreWeave's bring-up process addresses this with a multi-stage validation pipeline. Each node undergoes individual GPU diagnostics, PCIe bandwidth testing, and training-workload benchmarks before being admitted to a rack. Once individual nodes pass, rack-level collective benchmarks run all 72 GPUs simultaneously and compare results against a known-good performance envelope. Any rack that tests below range goes to troubleshooting rather than production. After rack-level validation, distributed workloads run across multiple racks and force traffic over the backend network fabric to verify that the connected system performs reliably — not just as a collection of passing racks.

The cooling infrastructure is managed through Valvey, a software-defined liquid cooling control system that CoreWeave describes as giving programmable visibility into the coolant loop. At the power densities of a Vera Rubin deployment — NVIDIA specifies 45°C liquid-cooling inlet requirements — cooling is not a facilities function that can be monitored passively. Valvey, alongside Racky (the rack control layer) and the Rack LifeCycle Controller (a Kubernetes-native orchestrator that treats an entire NVL72 rack as a single programmable entity), form the automation stack that CoreWeave uses to bring racks up in a known state and keep them there.

Read more: Nvidia posts first Vera Rubin benchmark data: 30x efficiency gain on agentic workloads

MLPerf v6.1: The Day Vera Rubin Got Its First Third-Party Score

The significance of this timing becomes clearer when the multi-rack announcement is read alongside MLPerf Inference v6.1, published September 16 by MLCommons with Vera Rubin NVL72 making its first appearance in the benchmark.

MLPerf is an industry-standard AI inference benchmark run by MLCommons, a consortium of researchers and companies. The Datacenter Closed division, where Vera Rubin results appear, uses standardized models and evaluation procedures to compare systems on a common basis. Results are verified by MLCommons before publication, which gives them a different evidentiary status than company-reported benchmark numbers.

In its preview submission, NVIDIA Vera Rubin NVL72 delivered up to 3.7x higher throughput than GB300 NVL72 on Qwen3-VL-235B-A22B — a 235-billion-parameter multimodal mixture-of-experts model — across offline, server, and interactive inference scenarios, according to NVIDIA's MLPerf v6.1 blog. On DeepSeek-R1-671B, the widely deployed 671-billion-parameter reasoning model, Vera Rubin reached 2.5x higher throughput than GB300 NVL72. Both comparisons use systems running on the same generation of NVIDIA's Ethernet networking.

The benchmarks used disaggregated serving — separating the prefill stage (processing the input prompt) from the decode stage (generating output tokens) across different GPU pools. This separation allows the compute-intensive prefill and the memory-bandwidth-intensive decode to be optimized independently, improving overall system throughput without either phase waiting on the other. Expert parallelism distributed token routing across Vera Rubin's MoE expert layers across multiple GPUs, and NVFP4 precision reduced the memory footprint of model weights, attention tensors, and the KV cache — allowing more effective batch sizes without hitting memory limits.

MLPerf benchmarks have important limitations. They measure throughput under controlled conditions, not cost-per-token or latency variability under production workload patterns. They use specific model versions and software stacks, and results can shift significantly as software is updated. NVIDIA's own blog notes that post-submission software optimization delivered further performance gains on GPT-OSS-120B and DLRMv3 that have not yet been verified by MLCommons. CoreWeave submitted Blackwell and Blackwell Ultra results in MLPerf v6.1 and led cloud providers on several model categories, but has not yet published a separate Vera Rubin submission through its own account — the Vera Rubin results in this round were submitted by NVIDIA.

CoreWeave also previously measured and published its own comparison of Vera Rubin against GB200 NVL72 on DeepSeek R1 in July 2026, finding 10x more tokens per second per megawatt at matched interactivity targets. That measurement is company-reported, not MLCommons-verified, and was conducted with all major inference optimizations enabled including multi-token prediction and disaggregated prefill. The MLPerf v6.1 figures, being independently verified, are more reliable for comparison purposes — though they use different baseline hardware (GB300 rather than GB200).

AI Object Storage: The Hidden Bottleneck in Agentic Training Loops

The same announcement included two new storage capabilities that target a problem GPU performance benchmarks do not capture: what happens when a training job needs to read or write data across regions.

Agentic AI training is unusual in that it involves tightly coupled cycles: a model runs a reasoning step, produces an output, that output gets evaluated or used by a tool, and the result becomes part of the next input context. Each step in this loop is sensitive to latency — not just GPU latency but also the time it takes to retrieve context from storage, commit a training checkpoint, or replicate state to a backup region. In a conventional training run, checkpoint writes are relatively infrequent. In reinforcement learning loops and multi-step agentic post-training, checkpoint frequency can increase substantially.

CoreWeave's cross-region write acceleration addresses what happens when training compute is in one region but the canonical copy of training data lives in another. Previously, a job writing a checkpoint would either wait for the write to complete in both regions (adding latency proportional to the inter-region round-trip) or accept the risk of a checkpoint being available in only one region. The new capability writes locally at NVMe latency — immediately visible to the training job — while CoreWeave replicates the data to the second region asynchronously in the background. Because the application sees a single bucket, no code changes are required and access-control policies apply consistently regardless of which region the write originated from.

The Archive tier targets a different problem: the data that gets deleted not because it has no value but because retention costs money. In practice, AI teams routinely delete checkpoints from training runs that came close to working, dataset versions that may be needed to reproduce a result, and model versions that might be needed for comparison or debugging months later. Archive is priced to make keeping this data economically viable, with no retrieval fee, no early deletion penalty, and no egress fee for reads from within the tier.

Cécile Robert-Michon, director of internal infrastructure at Cohere, said in the announcement that cross-region dataset access latency had previously constrained training schedules: "Our datasets span multiple regions, and we can't afford to have our training schedule dictated by cross-region retrieval delays." CoreWeave says LOTA — the local object transport accelerator that caches data at NVMe speeds on each Kubernetes Service node — reduced cross-region read latency by 8x for Cohere's workloads. That figure is customer-reported via the vendor announcement and not independently audited, but it is directionally consistent with the expected benefit of converting remote-storage reads into local-cache reads.

Competitive Context: Who Else Is Running Multi-Rack Vera Rubin

CoreWeave has been the first cloud provider to deploy each of NVIDIA's recent rack-scale platforms: GB200 NVL72 general availability came first on CoreWeave, then GB300 NVL72, and now multi-rack Vera Rubin NVL72. The pattern reflects a combination of NVIDIA's allocation strategy — which has historically prioritized partners who can absorb and operate new platforms quickly — and CoreWeave's operational infrastructure, which is purpose-built for rack-scale GPU deployment rather than adapted from a general-purpose cloud architecture.

Nebius, the European AI cloud that spun out of Yandex, has announced plans to offer Vera Rubin NVL72 in the US and Europe from the second half of 2026 and submitted Vera Rubin NVL72 preview MLPerf v6.1 results alongside NVIDIA. No announcement of multi-rack Vera Rubin specifically has been identified from Nebius as of this writing.

The major hyperscalers — AWS, Google Cloud, and Microsoft Azure — do not yet have confirmed multi-rack Vera Rubin NVL72 deployments. Azure was named among MLPerf v6.1 ecosystem partners for Blackwell-era systems. Neither AWS nor Google Cloud has announced Vera Rubin availability at any rack configuration. This creates a near-term window in which frontier AI labs and large-scale inference operators who require the compute density and energy efficiency of Vera Rubin have limited cloud options.

The competitive differentiation CoreWeave offers is not solely in access timing. SemiAnalysis rated CoreWeave Platinum in both editions of its ClusterMAX rating system, which evaluates AI cloud performance, reliability, and efficiency — the only cloud to hold that rating in both versions. That rating is an independent commercial analyst's assessment rather than a benchmark result. Artificial Analysis, an independent AI infrastructure benchmarking firm, ranked CoreWeave first for inference speed and price-performance for Moonshot AI's Kimi K2.6 and Kimi K2.7 Code models.

Limitations and Claims That Still Require Independent Verification

Several important claims in this announcement are company-reported or customer-reported rather than independently verified, and should be read with that context.

The "10x tokens per megawatt" figure comparing Vera Rubin to GB200 NVL72 was measured by CoreWeave on its own hardware in July 2026 and published on its engineering blog. The comparison was made with all major inference optimizations enabled and on a specific workload (DeepSeek R1). It is not a figure verified by MLCommons or a neutral third party. The MLPerf v6.1 comparison — 2.5x to 3.7x versus GB300 NVL72 rather than GB200 — uses different baselines and different measurement conditions and is independently verified, but covers different model families.

The claim that the Spectrum-X fabric supports approximately 128,000 GPUs per rail represents a design capacity, not a demonstrated deployment at that scale. No cloud provider has publicly reported operating anywhere near 128,000 GPUs in a single non-blocking fabric. The number describes the theoretical ceiling of the architecture, not a capability validated at production scale.

The 8x latency improvement for cross-region dataset access reported by Cohere comes via CoreWeave's press release citing a Cohere executive. It has not been confirmed by independent measurement or Cohere's own public documentation. It is directionally plausible given the technology (local NVMe caching versus remote storage cluster reads), but the specific 8x figure reflects one customer's production workload under conditions that may differ substantially from other deployments.

CoreWeave has not yet disclosed the specific number of racks in its current multi-rack Vera Rubin configuration. "Hundreds of Rubin GPUs" implies at least two NVL72 racks (144 GPUs minimum), but the actual cluster size is not stated in the announcement.

What Changes Now — and What to Watch Next

The multi-rack milestone matters most to AI teams whose workloads have outgrown what a single 72-GPU rack can deliver. For reinforcement learning post-training — where a policy model generates trajectories, a reward model scores them, and gradients update the policy across many simultaneous parallel environments — the bottleneck has been the number of GPUs that can participate in a synchronized training step. A multi-rack Vera Rubin cluster removes the rack-size ceiling for that compute pool.

For large-scale inference deployments serving reasoning models like DeepSeek R1 at high concurrency, the same applies: the fraction of a model's parameters that can be held in HBM memory at full activation expands as more Vera Rubin racks join the pool, which directly affects the per-token cost and per-user latency of production deployments.

The pending benchmark to watch is a CoreWeave-submitted MLPerf Vera Rubin result under its own account, which would allow direct comparison of CoreWeave's infrastructure stack on Vera Rubin against Nebius and NVIDIA's first-party submissions under identical benchmark conditions. CoreWeave's MLPerf v6.1 Blackwell submissions demonstrated leading per-GPU throughput on GPT-OSS-120B and Llama 2 70B against other cloud providers; whether that optimization advantage extends cleanly to Vera Rubin at multi-rack scale will be visible when that submission is published.

The larger question that neither today's announcement nor MLPerf v6.1 settles is what Vera Rubin multi-rack clusters cost to access. CoreWeave has not published per-GPU-hour pricing for Vera Rubin at multi-rack configuration. The economics of 10x more tokens per megawatt are compelling in principle, but the relevant figure for most AI teams is cost per million tokens at their required quality and latency — a number that depends on pricing structures that remain unpublished.

Read more: Nvidia posts first Vera Rubin benchmark data: 30x efficiency gain on agentic workloads