When it comes to tasks that demand whole-body coordination in humanoid robots, such as handling boxes, the current solutions of tracking reference motions frame by frame are inadequate. They simply can't cope with scenarios where only the target position of the box is provided. To tackle this issue, researchers have put forward the Unified Multimodal Control Framework (ULTRA) and have put it to the test on the Unitree G1 humanoid robot. The relevant research has been accepted by IROS 2026 and has made it onto the shortlist as a candidate for the conference's Mobile Manipulation Paper Award.

ULTRA makes use of physics-driven neural motion retargeting technology. This technology can convert human motion capture data into executable trajectories for the robot. By employing teacher policy distillation and reinforcement learning fine-tuning, it trains a controller that is backed by the same set of policy parameters. This controller is quite versatile. On one hand, it can carry out precise tracking when reference motions are at hand. On the other hand, when only sparse objectives are given, it can autonomously generate whole-body motions to accomplish tasks. It does this by utilizing perceptual information from external motion capture systems or its own first-person-view depth camera. This accomplishment opens up a practical route for humanoid robots to move from relying on human motion experience to achieving flexible and autonomous whole-body control.