A research team, headed by Ding Ning from Tsinghua University and working in tandem with scholars from various universities and relevant organizations, has developed a versatile simulation platform and evaluation benchmark for cross-embodiment dexterous manipulation, named HandsOn. This initiative effectively bridges the gap caused by the absence of a systematic and trustworthy evaluation system in the realm of dexterous hands. The benchmark encompasses 66 tasks verified by humans and 18,000 instances of human teleoperation demonstrations within a simulated environment. It divides these tasks into two distinct systems: Core-50 and Transfer-16. The former is tailored to evaluate learning execution capabilities, while the latter is designed to assess generalization transfer capabilities.

HandsOn facilitates the random integration of diverse robotic arms, dexterous hands, and various object assets. Additionally, it meticulously records the progression of multi-step tasks and pinpoints failure points. The team conducted evaluations on models derived from different technical methodologies. Their findings revealed that the highest success rate of current multi-task policies on Core-50 stands at a mere 29.1%, with the average success rates on Transfer-16 all falling below 1%. Even the zero-shot GPT-6 model achieved success rates of only 5.2% and 1.9% on these two task sets. These outcomes underscore the formidable challenges in bimanual coordination, precise contact, and maintaining long-duration states for dexterous hands.

At present, all pertinent papers, code, and data resources have been made accessible to the public. This move provides unified and comparable experimental conditions for future research endeavors in dexterous manipulation and aids in progressively delineating the capability boundaries of dexterous hand operations within the industry.