Recently, EXO Lab unveiled its latest breakthrough: by integrating two NVIDIA DGX Sparks with a Mac Studio powered by an M3 Ultra chip, they achieved a remarkable 2.8-fold performance boost in AI large language model inference tests, compared to running the Mac Studio solo.

This innovative solution leverages EXO Lab’s self-developed open-source framework, EXO, which intelligently distributes workloads to form an “AI Mesh” architecture. Similar to how a WiFi Mesh network extends coverage by linking multiple nodes, this setup harmonizes computing resources. During experiments, the DGX Sparks took charge of computationally heavy prefilling tasks, while the M3 Ultra focused on bandwidth-sensitive decoding tasks. Real-time transmission of KV caches enabled seamless parallel processing.

Benchmark tests using the Meta Llama-3.1 8B model confirmed that this hybrid architecture significantly enhanced system throughput efficiency. However, it’s worth noting that the EXO 1.0 version remains in its early experimental phase and has not yet been made available to the public.