NVIDIA notes that, driven by factors such as the continued expansion of AI models, GPU memory pressure keeps mounting, HBM supply is tightening, and prices are on the rise. To address this, the company proposes improving memory utilization efficiency through collaborative design across GPU, CPU, network, storage, and software—rather than simply increasing GPU memory capacity. Specific measures include: compressing model weights and KV cache using low-precision formats, offloading inactive context to lower-cost storage tiers, and deploying across multiple GPUs in a distributed manner—thereby freeing up HBM space. Test data shows that, under specific workloads, GB300 NVL72 achieves up to 10x the per-unit-cost inference throughput of the H200 platform, while Vera Rubin NVL72 delivers up to 30x higher per-unit-cost inference throughput compared with GB300 NVL72—even though single-GPU HBM capacity increases by only about 3%.