AutoNavi Teams Up with Nanjing University, Tsinghua University, and Peking University to Launch WorldRoamBench, Establishing a Fresh Benchmark for Long-Duration Roaming Assessment of Interactive World
2 day ago / Read about 0 minute
Author:小编   

The unveiling of Genie 3 heralds a new era in the realm of interactive world models, signifying a shift from mere video generation to the simulation of interactive worlds. Nevertheless, there exists a notable gap in the systematic evaluation of long-duration interactions, as current assessment methods predominantly concentrate on brief video clips, thereby posing challenges in pinpointing issues that arise during extended interactions. To bridge this gap, the AutoNavi Vision Technology Center, in partnership with several esteemed universities, has introduced the WorldRoamBench evaluation framework. This innovative framework encompasses over 1,000 long-duration roaming scenarios and assesses 12 leading interactive world models across four key dimensions: action, vision, physics, and memory. By doing so, it effectively tackles numerous limitations inherent in existing evaluation methods, such as the tendency of action trajectories to obscure local errors and the oversight of mid-session visual drifts when comparing only the start and end frames. The actual test outcomes reveal that no single model excels uniformly across all dimensions. For instance, while Genie 3 demonstrates exceptional performance in certain areas, it also exhibits notable shortcomings. Presently, WorldRoamBench has opened its evaluation portal to the public, with plans to gradually release relevant code and scripts, thereby ushering in a new phase in the competitive landscape of interactive world models, with a heightened focus on achieving comprehensive stability in long-duration interactions.