Apple and NVIDIA Propel Large-Parameter On-Device Deployment: AI PCs Step into the 100-Billion-Parameter Era
3 day ago / Read about 0 minute
Author:小编   

In the past, deploying large-scale models directly on devices posed significant challenges, primarily due to hardware constraints. Today, however, NVIDIA's RTX Spark and Apple's Mac Studio have broken through these barriers, achieving on-device deployment of large models boasting 120 billion parameters and nearly one trillion parameters, respectively. This milestone signifies the official entrance of personal AI PCs into the 100-billion-parameter era. Both companies leverage a technical strategy that combines the Mixture of Experts (MoE) architecture with high-bandwidth unified memory, facilitating the on-device deployment of large models across diverse product lines. This advancement empowers users to cut down on cloud inference expenses, safeguard against sensitive data breaches, and cater to the productivity demands of professional developers, intensive content creators, and other user groups. While such products currently come with a premium price tag, costs are anticipated to gradually decline as unified memory technology continues to evolve.