On March 19th, NVIDIA unveiled Dynamo, an innovative "operating system for AI factories." Dynamo serves as a distributed inference serving library tailored to tackle the open-source challenge of token production inadequacy in meeting user demands. During the launch event, NVIDIA CEO Jensen Huang presented an inference chart, where the X-axis represented the user's tokens per second and the Y-axis represented the factory's tokens per second. He highlighted that the synergistic action of Dynamo and the Grace Blackwell NVLink72 superchip has enhanced Blackwell's performance by 25 times compared to its predecessor, Hopper. Notably, in inference models, Blackwell's performance surpasses Hopper by an impressive 40 times. Huang quipped that with these cutting-edge technologies, manufacturers establishing AI factories will experience the benefits of "economies of scale," where increased purchases translate into greater savings.
