Video generation can now be code-driven—introducing the Code Video Model
6 day ago / Read about 0 minute
Author:小编   

Current mainstream video diffusion models suffer from insufficient constraints on physical laws and difficulties in precise control. To address this, the Microsoft team has introduced the Code Video Model, which uses executable code as the control representation for videos. Through a Coding Agent, it transforms prompts into executable scene programs in Three.js to generate preview videos. Before generation, users can inspect, edit, and validate scene layouts, motion trajectories, and physical parameters. Through systematic experiments, the research reveals the mechanisms of different conditional pathways and proposes a training-free, phased conditional scheduling paradigm. This paradigm introduces spatiotemporal structural constraints from previews early on while freezing the parameters of pre-trained video models. Later, it removes the preview conditions, allowing text and reference images to guide appearance refinement. This approach retains the structural and motion accuracy defined by the program while avoiding dominance of the preview's low-polygon style in the generated results. It supports controllable video generation for various scenarios, including animation, bullet-time shots, scene editing, and physical process reconstruction, shifting the traditional iterative process of post-generation trial-and-error lens adjustments to a pre-generation preview validation stage.

  • C114 Communication Network
  • Communication Home