The World Model Evaluation Platform, known as WorldOracle, which has been developed through a collaborative effort among researchers from Zhiyuan, NVIDIA, and a consortium of universities, has been officially rolled out and is currently open for evaluation submissions. Constructed upon a foundation of 335 real-world operational trajectories, with roughly 40% of them encompassing unsuccessful or less-than-optimal interactions, the platform's primary objective is to rigorously assess the predictive prowess of models by leveraging actual execution outcomes. Upon conducting tests on seven distinct world models, it was observed that although certain models exhibited a high degree of accuracy in replicating robotic arm movements, there persisted substantial discrepancies in predicting object states. Furthermore, their proficiency in making long-sequence predictions also witnessed a marked decline.
