Resources 
NVIDIA Vera Rubin NVL72 combines 72 Rubin GPUs and 36 Vera CPUs in a single NVLink domain, and it is the flagship compute rack of the Vera Rubin platform, succeeding Grace Blackwell. NVIDIA announced it at CES in January 2026 and has said that production shipments begin this fall.
NVIDIA does not offer a Rubin GPU as a discrete part to add to existing servers, and the training and inference figures it publishes are rack figures, so it sells the rack rather than the chip. At Hot Chips 2026, the company went further, presenting Rubin specifications against a 100 MW AI factory footprint rather than against a single accelerator, which sets out what NVIDIA now treats as the smallest meaningful unit of the design.
Seven co-designed chips make up the platform:
- Rubin GPU, the compute accelerator
- Vera CPU, the custom Arm-compatible host processor
- NVLink 6 switch, holding the 72 GPUs in one scale-up domain
- ConnectX-9 SuperNIC, providing scale-out bandwidth per GPU
- BlueField-4 DPU, handling storage, networking, and tenant isolation
- Spectrum-6 Ethernet switch, connecting racks across the facility
- Groq 3 LPU, an SRAM-based inference accelerator that ships in its own rack
Those chips are distributed across five rack-scale systems rather than assembled into one cabinet, and the systems combine into what NVIDIA calls a Vera Rubin POD. A full POD runs to 40 racks, holding 1,152 Rubin GPUs, close to 20,000 dies and 1.2 quadrillion transistors, and NVIDIA rates it at 60 exaflops with 10 PB/s of scale-up bandwidth.
Vera Rubin NVL72 rack design and naming
The rack is built on the third generation of the NVIDIA MGX architecture, and holds its 72 GPUs and 36 CPUs across 18 compute trays and 9 NVLink switch trays alongside ConnectX-9 SuperNICs and BlueField-4 DPUs. NVIDIA puts the assembled rack at roughly 1.3 million components, close to 1,300 chips, and about 4,000 lbs. The NVLink spine at the rear holds four modular copper cable cartridges carrying 5,000 cables.
At GTC in 2025, the rack was announced as NVL144, on a convention that counted GPU compute dies rather than packages, since each Rubin package carries two reticle-limited compute dies joined by an inter-die link. The name reverted to NVL72 in late December 2025, shortly before CES confirmed it. NVL72 counts the 72 packages, and NVIDIA’s current materials use it throughout without noting that the name changed. A specification quoted against the wrong convention is off by a factor of two on GPU count.
The Vera Rubin NVL144 CPX rack announced in September 2025 pairs standard Rubin GPUs with Rubin CPX, a GDDR7-based accelerator aimed at the compute-bound prefill phase of long-context inference.
Vera Rubin architecture
The Rubin GPU
NV-HBI links the two compute dies on each Rubin package. The GPU carries 336 billion transistors, 224 streaming multiprocessors, and 896 Tensor Cores, with a third-generation Transformer Engine rated at up to 50 petaflops of NVFP4 inference.
The larger change is in memory, where Rubin carries up to 288 GB of HBM4 in 12-high stacks at up to 22 TB/s. That bandwidth is 2.8 times that of Blackwell and Blackwell Ultra, both at 8 TB/s, and NVIDIA attributes the increase to HBM4 doubling the interface width relative to HBM3e, along with a new memory controller and tighter compute-memory integration.
Reasoning and agentic workloads spend more of their end-to-end runtime decoding, where inference is bound by the memory subsystem rather than peak compute. That makes the bandwidth increase the part of the generational step most likely to show up in serving throughput.
Several of the smaller architectural changes target specific stalls rather than peak numbers.
- Rubin Tensor Memory Accelerator descriptors:
A mixture-of-experts model holds hundreds of expert matrices of identical shape at different addresses, and the Tensor Memory Accelerator that moves them between HBM and on-chip memory needs a descriptor for each one giving its shape, layout, and address. Blackwell requires a separate descriptor per expert; Rubin supports inline descriptor updates, keeping one and overriding the pointer and stride at runtime.
- Rubin K-dimension and softmax throughput:
Tensor Core throughput per clock doubles along the K dimension, the shared inner dimension of a matrix multiply, so a multiplication that required four passes on Blackwell completes in two. The saving is largest at high tensor parallelism, where each GPU holds a small slice of the output but K stays large. Faster matrix operations move the bottleneck into attention, so exponential throughput rises to twice Blackwell for FP32 and four times for BF16 and FP16 to keep softmax from falling behind.
- Rubin attention sparsity:
Adaptive 2:4 sparsity compresses intermediate attention scores. NVIDIA presented it at Hot Chips 2026 as a runtime flag rather than a default, and most models require no fine-tuning to use it.
- Rubin NVLink counted writes:
Communication between GPUs carries overhead beyond the payload itself. NVLink adds counted writes for device-initiated transfers, letting the receiving GPU track completion without the barrier and atomic-flag sequence Blackwell requires.
The Vera CPU
Vera is NVIDIA’s first fully custom CPU core rather than a licensed Arm design. NVIDIA’s Vera CPU documentation describes 88 Olympus cores on a monolithic 3 nm compute die, Arm-compatible, with NVLink-C2C running at 1.8 TB/s to the paired GPUs. One Vera CPU and two Rubin GPUs form the Vera Rubin Superchip, and each compute tray holds two superchips.
The cores are built for single-threaded speed rather than core count, which is the design target for agentic work. Spatial Multithreading runs 176 threads across the 88 cores, each carrying 2 MB of L2 backed by 164 MB of unified L3, and a second-generation Scalable Coherency Fabric moves 3.4 TB/s of bisection bandwidth between them.
Host memory is LPDDR5X rather than HBM, up to 1.5 TB per CPU at 1.2 TB/s across eight SOCAMM2 modules, with PCIe Gen 6 and CXL 3.1 for connectivity beyond the superchip. Capacity matters for agentic serving because it gives the platform room to hold context that does not fit in the 288 GB of HBM4 on each GPU.
Single-threaded performance on the host processor sets how quickly work reaches the accelerators, and agentic workloads add CPU-side sandboxing for tool execution and validation on top of the usual data movement. A GPU waiting on its host is idle the same way a GPU waiting on storage or a collective is, which is why we expect the host processor to matter more over the next cycle.
The rack and the fabric
NVLink 6 delivers 3.6 TB/s per GPU and 260 TB/s of scale-up bandwidth across the rack, holding all 72 GPUs in a single NVLink domain. The boundary of that domain is what matters in operation, since traffic between GPUs inside it moves at NVLink speed while traffic that crosses it drops onto a scale-out network carrying 28.8 TB/s per rack against the domain’s 260. NVLink-C2C adds 65 TB/s across the rack between the Vera CPUs and the GPUs they pair with, the path anything in host memory takes to reach a GPU.
Scale-out runs at 0.4 TB/s per GPU over ConnectX-9 SuperNICs, with either Spectrum-X Ethernet or Quantum-X800 InfiniBand above the rack. Any job spanning more than 72 GPUs crosses this network rather than NVLink, so it sets the ceiling for multi-rack training and for serving deployments large enough to span racks. The two options are alternatives, not tiers, and the choice determines the monitoring and tuning stack the operator works with afterward.
Operators can place NVLink switch trays into maintenance mode and replace them while the rack continues operating, and the GPUs run continuous, non-disruptive health checks. Whether those features translate into higher goodput depends on how the operator uses them; the hardware provides the mechanism, and the operator configures the detection and eviction policy afterward.
The other four Vera Rubin POD racks
The Vera Rubin POD surrounds NVL72 with four specialized racks. NVL72 is built on the MGX NVL rack architecture around its NVLink spine, while the other four use MGX ETL, configurable up to 256 chips per rack, with a Spectrum-X Ethernet spine in the Vera CPU and BlueField-4 STX racks, and a direct chip-to-chip spine in Groq 3 LPX.
NVIDIA lists all five rack types as in production, but publishes a full specification table only for NVL72. The figures for the other four racks come from platform announcements rather than product documentation and therefore carry less detail.
Groq 3 LPX holds 256 LP30 accelerators across 32 liquid-cooled trays, eight to a tray, and serves the low-latency portion of agentic workloads alongside NVL72. The accelerators carry no external memory, running instead on 128 GB of aggregate SRAM at 40 PB/s, with NVIDIA rating the rack at 315 PFLOPS of FP8 and 640 TB/s of scale-up bandwidth. NVIDIA reached the LPU design through a licensing and integration agreement with Groq rather than an acquisition.
A Vera CPU rack holds up to 256 CPUs and, on NVIDIA’s figures for the POD, sustains more than 22,500 concurrent reinforcement learning or agent sandbox environments. Post-training and agentic evaluation need somewhere to execute tool calls and verify results, and that work is CPU-bound rather than GPU-bound, so it needs its own rack.
A BlueField-4 STX rack hosts the CMX context memory platform, which offloads KV cache into a dedicated storage tier rather than holding it in HBM. BlueField-4 combines a Vera CPU with ConnectX-9 on one device, and NVIDIA projects that the DOCA Memos software layer raises inference throughput by up to five times, a figure that carries the same projection caveat as the rest of the platform comparisons.
Spectrum-6 SPX networking racks connect the POD, using a 102.4 Tb/s switch with 512 lanes and 200 Gb/s co-packaged optics.
Vera Rubin NVL72 specifications
All figures in the table below, except rack power, come from NVIDIA’s product page, which footnotes the table as preliminary, with all values subject to change. Training figures are dense specifications, and the two emulated rows show single- and double-precision matrix multiplication run on Tensor Cores rather than native FP32 and FP64 units.
* These figures have been superseded since the 2025 announcement. Per-GPU HBM4 bandwidth was originally given as 13 TB/s and is now 22 TB/s, and NVLink 6 was quoted at 250 TB/s before settling at 260. The rack headline numbers moved from 3.5 exaflops of NVFP4 and 1.4 PB/s of memory bandwidth to 3,600 PFLOPS and 1,580 TB/s. The 75 TB of fast memory quoted for the rack was an aggregate of HBM and LPDDR, and reconciles with the current 20.7 TB of HBM4 plus up to 54 TB of LPDDR5X. CPU memory is the maximum installable rather than the amount shipped.
The compute and memory rows are per-GPU figures multiplied by 72, and the two CPU rows are per-CPU figures multiplied by 36. NVIDIA rounds the per-unit values, so several rows land close rather than exact; for instance, the native FP64 gives 2,376 for the rack at 33 TFLOPS per GPU, against the 2,400 on the table. Thirty chips per superchip across 36 superchips comes to 1,080 rather than 1,296, because the rack total takes in the NVLink switch trays and network devices that sit outside the superchips.
Vera Rubin NVL72 rack power
NVIDIA has not published a rack power figure for Vera Rubin NVL72 on its product page, in its datasheet, or in its technical blogs. One published estimate puts the rack between roughly 190 kW and 230 kW, against 120 kW to 130 kW for GB200 NVL72, and Bernstein has used about 220 kW in its own modeling. Supermicro has published data center building block designs sized for 227 kW per rack, which is a design target from an OEM rather than a rating from NVIDIA.
The first third-party verified benchmark ran on a production SKU rated at 2,300 W TDP per GPU, which puts the 72 GPUs alone at about 166 kW before the CPUs, switch trays, and networking are counted. The same analysis models all-in utility power, including facility overhead, at 3.3 kW per Rubin GPU against 2.12 kW for GB300.
Vera Rubin cooling and power management
All MGX racks are designed for a 45°C warm-water inlet, which allows facilities in many climates to run dry coolers and ambient air rather than compressors. That is a ceiling, not a requirement, and a site can supply colder water if its loop calls for it, though the savings are at the top of the range. 45°C cooling alone enables about 10 percent more NVL72 racks within the same power budget; dynamic Max-Q provisioning combined with 45°C cooling enables up to 30 percent more GPUs; and DSX MaxLPS at the AI factory layer enables up to 40 percent more GPUs at energy-efficient operating points.
As AI training runs thousands of GPUs in lockstep, power draw swings sharply between idle and full load, and the cluster mirrors and amplifies a single node’s pattern; grid generation cannot ramp on those timescales, which is why synchronized AI workloads destabilize the supply.
NVIDIA introduces Intelligent Power Smoothing with Vera Rubin NVL72, which uses a bank of capacitors in the rack to even out sudden swings in the power it draws. When a workload calls for a lot of power at once, the capacitors release what they have stored, so the amount the rack pulls from the grid stays steady or climbs slowly. When the workload stops, they recharge. The capacitors only smooth the rack’s own demand, and none of the stored power goes back to the utility.
The rack carries 400 J of energy storage per GPU, six times the 65 J per GPU on GB300, with a new closed-loop system that lets the GPUs monitor the state of charge of the capacitors. Average power falls by about 10 percent versus previous-generation power-smoothing techniques, and 50 ms peak power by about 20 percent. The peak figure matters when a site is capped by the capacity it contracted for with the utility, and the average figure when the limit is energy cost.
Vera Rubin performance
NVIDIA puts Vera Rubin NVL72 at one-tenth the cost per million tokens and up to ten times the tokens per megawatt of GB200 NVL72, running Kimi-K2-Thinking at a 32K input and 8K output length. Training a 10T mixture-of-experts model on 100T tokens within a fixed one-month window takes a quarter as many GPUs on Vera Rubin NVL72 as on GB200 NVL72.
NVL72 paired with LPX racks is rated at thirty-five times the throughput per megawatt of Blackwell for trillion-parameter models, a figure calculated across five tiers of a modeled subscription service, each tier running a different model at a different cache depth, from Qwen-3 235B at 32K cached tokens on the free tier to GPT-MoE 2T at 400K on the top one. At the GPU level, Rubin is rated at up to ten times the agentic throughput per unit of energy of Blackwell on an internal 2T mixture-of-experts workload. All of these are labeled projected and subject to change, and the LPX pairing has not been measured in a customer deployment.
NVIDIA’s own figures, first shown at Hot Chips 2026 and updated in September with data from the agentic benchmark below, put Vera Rubin NVL72 at up to 30 times the throughput per megawatt of GB300 NVL72, rising from about ten times as the interactivity target climbs, and at 45 times lower token cost. The Hot Chips presentation also gave a 13 percent reduction in peak power for LLM training on NVL72, and 40 percent more GPUs per provisioned watt as a system-level figure.
The first third-party verified results, published in September 2026 on early pre-release TensorRT-LLM software, ran DeepSeek V4 Pro at FP4 on a production Rubin rack. Against the strongest GB300 NVL72 configuration at each interactivity target, Vera Rubin NVL72 delivered about 1.45 times the throughput per megawatt at 75 tokens per second per user, 2.1 times at 100, and 7.2 times at 150, narrowing to 2.7 times at 200. It also reached about 61 percent higher peak interactivity than GB300 on TensorRT-LLM, though GB300 on SGLang came close to matching it. The results are published as a preview, subject to change as validation continues.
At 170 tokens per second per user, Rubin measured 62.9 times the throughput per megawatt of GB300 running TensorRT-LLM, whose curve is near the end of its measured range at that speed, and 5.56 times GB300 running SGLang, so the serving engine on the baseline moved the ratio more than tenfold at a single target. Every multiplier here compares one workload shape and one software configuration to another, so serving 32K-in, 8K-out requests will not produce the ratio that holding 400K tokens in cache does.
What Vera Rubin is used for
The platform targets agentic inference, where a single prompt produces a long sequence of reasoning steps, tool calls, and retrievals rather than one response. That shape stresses decode throughput, KV cache capacity, and low per-step latency at once, which is why NVIDIA pairs high-bandwidth HBM4 GPUs with SRAM-only LPX accelerators and a dedicated context memory tier.
Large mixture-of-experts models are another target, since expert routing generates heavy all-to-all traffic and a 72-GPU NVLink domain absorbs traffic that a smaller domain pushes onto slower Ethernet. The descriptor-sharing and K-dimension changes in the Tensor Cores are aimed at the same class of model.
Scientific computing is a third use, with Vera Rubin systems anchoring several announced national programs, including Doudna at NERSC and Blue Lion at LRZ, alongside classified and open systems at Los Alamos. Buyers sizing for FP64 should read the specification carefully, since the larger of the two FP64 rows on the table comes from Tensor Core emulation, and the native FP64 line is different and much smaller.
If you are planning Vera Rubin capacity, evaluating if your existing sites can take it, or trying to work out what the published numbers mean for your workload rather than for a benchmark configuration, talk to our team.
