Alibaba has introduced HappyOyster 1.0, an innovative open-world model product that harnesses the power of deep learning to comprehend the dynamics of state transitions in the real world. It possesses the ability to proactively infer causal relationships and uphold long-term coherence. This product boasts two primary functionalities: world exploration and real-time guidance, complemented by enhanced interactive features such as attacking, jumping, and storyline revisiting. With just a single sentence or an image, users can effortlessly create an interactive, exploratory, and customizable AI-driven digital environment. In contrast to text-to-video models, HappyOyster 1.0 excels in learning the transition patterns between states and actions, allowing it to acquire knowledge from natural videos and apply it to novel situations.
When compared to its predecessor, its interactive capabilities have undergone significant refinement. The world exploration mode facilitates in-depth exploration, intricate physical interactions, and seamless real-time movement and camera manipulation. The real-time guidance mode enables users to pause, revisit previous moments, explore branching narratives, and generate real-time footage extending beyond three minutes, with the added convenience of instant content sharing.
This versatile product finds applications across various industries, including interactive gaming, virtual companionship, interactive short dramas, cultural tourism experiences, and live streaming. Presently, the Alibaba team is working in tandem with Nanjing University to establish the inaugural industry benchmark for world models, with plans to soon unveil a comprehensive API interface.