Li Feifei, along with two other co-founders of World Labs, appeared on a16z to provide a detailed introduction to Atlas, a multimodal world model. The core of Atlas lies in new perspective prediction. Unlike large language models (LLMs) that predict the next token or video models that predict the next frame, Atlas can generate images at any spatiotemporal location based on multiple perspectives or descriptions of an input scene. Atlas is the first model to integrate generation and reconstruction functions into a single model, capable of natively processing multimodal information such as text, images, videos, 3D data, and camera poses. Traditional 3D reconstruction typically requires hundreds or even thousands of photos, whereas Atlas significantly reduces the data collection volume to just a few or dozens of images. With just three iPhones, it can capture bullet-time videos similar to those in The Matrix. Atlas is an iterative advancement from the previous-generation model, Marble. Marble used Gaussian splatting as its output representation, which had certain limitations. Atlas, on the other hand, uses new perspective prediction as its fundamental primitive, breaking through these constraints. It achieves a unified approach to generation and reconstruction, addressing blind spot issues common in traditional reconstruction methods without relying on generative completion. It also features a scalable spatial context window. In terms of application scenarios, Atlas can provide 3D-consistent generated content for creative professionals and can also be applied in fields such as architectural design and construction. In robotics, Atlas can significantly enhance the efficiency of real-world-to-simulation transfer, addressing the issue of insufficient training data for robots. In the future, it could incorporate dynamic data to enable connection with motion planning. Currently, Atlas already possesses preliminary dynamic processing capabilities and will further develop in areas such as dynamics, editing, and interaction, with the goal of building a system that allows users to intuitively interact with the 3D world. The research team believes that new perspective prediction holds equal importance to next-token prediction and may be one of the fundamental primitives leading to artificial general intelligence (AGI).
