SenseTime Team Assesses GPT-6-Astra: Excelling in Traditional Embodied Evaluations, Yet Struggling in Practical Robot Operations
8 hour ago / Read about 0 minute
Author:小编   

The SenseTime team, working in conjunction with several universities and enterprise partners, carried out 12 simulation evaluations and conducted systematic analyses across eight real-world robot platforms to evaluate the embodied intelligence capabilities of GPT-6-Astra. The study revealed that, when utilizing contextual learning and with more lenient action step constraints, GPT-6-Astra attained near-flawless scores in numerous traditional embodied intelligence assessments. However, in real-world robot operation tasks, it frequently made errors in scenarios requiring precise contact and during prolonged tasks. These issues included misjudging the success states of physical operations, making irreversible mistakes, and incurring high execution costs. The research concluded that although GPT-6-Astra has surpassed the basic threshold for embodied intelligence capabilities, there is still a significant gap to bridge before it can be reliably deployed in practical settings. The team also pointed out the limitations of current embodied intelligence assessments, stressing the importance of evaluations that more closely align with real-world household requirements. To tackle this challenge, they introduced the recursively self-evolving SenseNova-RoboRSI embodied system framework. This framework continuously refines robot performance through execution feedback, without the need to update the foundational model weights. Related solutions have been proven effective in multiple simulation evaluations.