NVIDIA GB200 NVL72: 72 Blackwell GPUs wired to think as one

One rack, one GPU

The engineering idea is audacious and simple to state: stop treating GPUs as separate accelerators that talk over a network, and wire 72 of them into a single NVLink domain with 1.8 TB/s of GPU-to-GPU bandwidth and a 130 TB/s NVLink Switch fabric, so the rack behaves — in NVIDIA’s phrase — as a single giant GPU. The chassis carries 36 Grace CPUs alongside the 72 Blackwell GPUs, liquid-cooled because air gave up long ago at this density. For the models that now matter — trillion-parameter systems that must hold their weights and their conversation in memory at once — that unity is not elegance; it is the difference between possible and not.

The manufacturer’s figures sketch the scale: up to 1,440 PFLOPS of NVFP4 sparse compute, 13.4 TB of HBM3e at 576 TB/s of memory bandwidth, 17 TB of LPDDR5X, and 2,592 Arm cores across the Grace CPUs — with NVIDIA claiming 30× faster real-time inference on trillion-parameter models than the H100 generation. Those are vendor numbers, to be read as vendor numbers; but the architecture underneath them — memory capacity, bandwidth and interconnect treated as one design problem — is the real announcement. This rack is why “AI factory” stopped being a metaphor: inputs of power and data, outputs of tokens, one integrated machine per line.

Inside the domain: why the wiring is the product

Here is the problem the NVL72 exists to solve, stated plainly. A frontier model is far too large for one accelerator, so its work is divided across many GPUs at once — and divided work must constantly recombine. During training, the machines exchange the corrections that keep their shared model in agreement; during inference on a very large model, they exchange the intermediate results from which each next token is produced. That conversation never stops, and its speed sets the system’s real pace. If the interconnect is slow relative to the arithmetic, the most expensive silicon on Earth spends its time waiting — processors idle, power burning, tokens late. The brutal economics of AI computing are therefore not really about how fast a GPU can calculate. They are about how rarely it has to wait. That is the entire case for building an interconnect domain instead of a network: inside the domain, the conversation runs at fabric speed, and the waiting mostly disappears.

The memory tells the same story from the other side. A model’s parameters, and the working state of the conversations it is holding, have to live somewhere fast. High-bandwidth memory — stacked memory placed immediately beside the processor, trading capacity for extraordinary speed — is that somewhere. Pool 72 GPUs into one domain and their combined memory starts to behave as a shared reservoir the model can be spread across, rather than 72 separate ponds with slow channels between them. The CPU side of the rack plays the supporting role the design intends: the Grace processors feed data, coordinate work and handle everything that is not dense matrix arithmetic, from their own large pool of lower-cost memory, so the GPUs are never starved of the next thing to compute. Every layer of the specification is the same idea repeated — remove a bottleneck, then remove the one behind it.

Read the headline compute figure with an engineer’s care, because NVIDIA labels it honestly and readers should honor the label. The rating is expressed in NVFP4 — a compact floating-point format that stores each number in very few bits. Fewer bits per number means more numbers moved per second and more operations completed per cycle: precision is deliberately traded for throughput, in workloads where experience shows the trade is safe. And the figure is a sparse rating: sparsity exploits the fact that large fractions of a model’s values can be treated as zero and skipped, so hardware that skips them does proportionally more useful work. None of this makes the number false — it makes it conditional, like a top speed recorded under stated conditions. The professional habit is the one this magazine keeps recommending: vendor figures, read as vendor figures, architecture read as architecture.

Finally, the liquid. Air cooling fails at this density for a simple physical reason: past a certain concentration of heat in a rack, you cannot move enough air through the machines to carry the heat away, however loud the fans get. Liquid carries heat far more effectively, so coolant does the job at the component level and the facility’s job changes from air-conditioning a room to plumbing a plant. It is less a cooling choice than a change of industrial discipline — and one more reason the rack, the hall and the power contract now have to be designed as a single system.

The decision this sets

For infrastructure buyers, the NVL72 reframes procurement from servers to systems: the unit of purchase, cooling, power and networking is now the rack-scale domain, and designs that assume PCIe-era composability will strand performance. For capacity planners, the liquid-cooling requirement is a facilities decision with a date — halls that cannot feed and cool these racks cannot host this generation. And for competitors, the bar is explicit: match the domain, not the chip.