What GPU to GPU compression is worth on a $1 billion GPU budget
The biggest training runs are limited by the network between racks, not by the GPUs. Here is what sending 24.5% to 49% fewer bytes, bit for bit, is worth on a 100,000-GPU run like GPT-6 Astra and on a $1 billion GPU budget.
OpenAI's GPT-6 Astra was reportedly trained on more than 100,000 NVIDIA Grace Blackwell GPUs at the Stargate site in Texas. NVIDIA's CEO called out the number publicly. Outside estimates put the compute for the main run somewhere between half a billion and a billion dollars.
At that scale, the GPUs are not the only thing that sets the speed of training. The network between them does. This post walks through why, and what it is worth to send fewer bytes across that network without changing a single bit of what arrives.
A 100,000-GPU run is a networking problem
No single fabric links 100,000 GPUs directly. A run like Astra is built from NVL72 racks: 72 Blackwell GPUs joined by NVLink into one large unit. 100,000 GPUs works out to about 1,389 of those racks.
Inside a rack, GPUs talk over NVLink at about 900 GB/s each way, per GPU. Between racks, every byte goes over the data center network instead, InfiniBand or Ethernet such as NVIDIA's Quantum or Spectrum-X platforms. That link is much narrower.
| Link | Gigabits per second | Gigabytes per second |
|---|---|---|
| 400G generation port (Quantum-2, Spectrum-4) | 400 Gb/s | 50 GB/s |
| 800G generation port (Quantum-X800, Spectrum-6) | 800 Gb/s | 100 GB/s |
| ConnectX-9 SuperNIC, top host adapter speed | 1,600 Gb/s | 200 GB/s |
| NVLink inside an NVL72 rack, per GPU, each way | about 900 GB/s |
Divide gigabits by 8 to get gigabytes. A GPU that can reach its rack neighbours at 900 GB/s reaches the other 1,388 racks at 50 to 100 GB/s: 9 to 18 times slower.
Switches multiply this out, but the per-port limit stays. A Quantum-X800 switch with 144 ports at 800 Gb/s moves 115.2 Tb/s, or 14.4 TB/s. A 512-port Spectrum-X SN6800 reaches 409.6 Tb/s, or 51.2 TB/s. Those are enormous numbers, and they are still the narrowest part of the machine.
Why training waits on the network
Every training step ends with an exchange. Each GPU has computed gradients on its share of the data, and those gradients must be combined across the whole cluster before the next step can start. Weights and optimizer state move too, depending on how the model is split. Inside a rack that is fast. Across racks it runs at network speed.
Engineers work hard to hide this behind computation, and at small scale they mostly succeed. As the cluster grows, the exchange crosses more racks and more of it shows up as GPUs waiting. Every second of waiting is paid for at the full price of 100,000 GPUs.
What changes with GPU to GPU compression
weights.zip GPU to GPU compresses tensors on the GPU right before they go onto the network, and restores them right after they arrive. Nothing is approximated: every tensor comes back exactly as it was sent, bit for bit, so training and its results are unchanged.
Measured on NVIDIA B200, using 10 of its 148 streaming multiprocessors, under 7% of the chip:
- bf16 gradients are 24.5% smaller, compressed at 223 GB/s and restored at 263 GB/s.
- fp32 gradients are 29% to 49% smaller with the high-ratio setting.
- bf16 weights are 24.7% smaller, restored at 281 GB/s.
- 1,200 of 1,200 tensors came back bit-exact.
The bf16 rates are faster than a 100 GB/s port can carry, so compression keeps up with the link instead of adding a new bottleneck. The high-ratio fp32 setting compresses at 62 to 76 GB/s on those 10 SMs; throughput scales close to linearly with more SMs, so matching a 100 GB/s port takes a slightly larger slice of the GPU. The practical effect: the same 100 GB/s port carries about 132 GB/s worth of bf16 gradients, and up to 196 GB/s worth of fp32 gradients. That is the equivalent of a third more network, up to nearly double, with no new switches, cables or cards.
The dollars on a run like Astra
Nobody outside OpenAI knows how much of each Astra training step was spent waiting on the network, so we show three scenarios. To put a price on the run, we assume 100,000 GPUs for 100 days at $3.00 per GPU-hour. That is 240 million GPU-hours and about $720M, inside the range outside analysts have estimated. It is our assumption, not a disclosed figure.
If network time scales with the bytes sent, 24.5% fewer bytes removes 24.5% of the network wait:
| Step time on the network | Run time saved | Days sooner | Saved, bf16 | Saved, fp32 |
|---|---|---|---|---|
| 10% | 2.45% | about 2.5 days | $17.6M | up to $35.3M |
| 20% | 4.9% | about 4.9 days | $35.3M | up to $70.6M |
| 30% | 7.35% | about 7.4 days | $52.9M | up to $105.8M |
In the middle scenario, that is 11.8 million GPU-hours handed back, or a frontier model finished nearly five days sooner. On a race measured in weeks, those days matter as much as the money.
The same math on a $1 billion GPU budget
Most companies do not train frontier models, but many now spend on this scale across training and serving. Here is the same calculation for $1 billion of GPU time spent on training:
| Step time on the network | Saved, bf16 gradients | Saved, fp32 gradients |
|---|---|---|
| 10% | $24.5M | up to $49M |
| 20% | $49M | up to $98M |
| 30% | $73.5M | up to $147M |
The alternative way to get the same effect is to buy more network: a third more port bandwidth to match the bf16 result, close to double to match fp32. At these cluster sizes, that is a capital project. Compression is software on hardware you already own.
Inference pays too
Serving moves weights more than gradients, and weights shrink just as well: bf16 weights are 24.7% smaller, restored at 281 GB/s.
- Cold starts and autoscaling. When traffic spikes and new nodes come up, the model must cross the network before the node can serve. A quarter fewer bytes means the weights arrive about 25% sooner.
- Model swaps. Fleets that serve many models or fine-tunes move weights between storage, hosts and GPUs all day. Each move carries a quarter fewer bytes.
- Memory. Weights kept compressed in GPU memory free about a quarter of the space they used, room for larger batches, longer contexts or more cached requests. Our Resident product holds 1.5x as many models per card this way.
As an illustration: if a $1 billion inference fleet spends 10% of its GPU time waiting for weights to arrive, cutting those bytes by a quarter returns up to 2.5% of the fleet, about $25M a year.
How to read these numbers
The compression results above were measured on NVIDIA B200 with production open-source models, every tensor checked bit for bit. The dollar figures are illustrative upper bounds, not measurements. They assume network time scales with bytes sent and that compression overlaps with the transfer. Real savings depend on the model, how it is split across GPUs, the network and the workload. The Astra figures use the publicly reported GPU count and our own cost assumption. OpenAI has not published the run's details, and Astra was not trained with weights.zip.
The way to get a real number is to measure it. We run GPU to GPU on your models, your parallelism and your network, then pilot it on one workload. See the full results on the GPU to GPU page, or request a pilot.
OpenAI and GPT-6 are trademarks of OpenAI. NVIDIA, Grace Blackwell, B200, NVLink, Quantum, Spectrum-X and ConnectX are trademarks of NVIDIA Corporation. weights.zip is not affiliated with or endorsed by either company.