US Startup Establishes AI-Powered Scientific Research Facility, Tests Reveal AI Models Lack Practical Lab Operation Capabilities
4 hour ago / Read about 0 minute
Author:小编   

The San Francisco-based startup C5R has constructed Facility-0, an AI-driven cross-disciplinary research facility spanning biology, chemistry, and materials science, over a 12-week period. The company also introduced the SciUniverse benchmark to evaluate AI models' operational capabilities in real laboratory environments. Test results revealed that while cutting-edge AI models possess solid theoretical scientific knowledge, they exhibit significant deficiencies in physical operational tasks. Examples include overlooking sample freezing status, causing sample contamination by reusing pipette tips, and ignoring solvent evaporation during open-well vortex mixing. In Level 1 task evaluations—which skilled scientists can complete within hours—the best-performing model, Claude Fable 5.1 Pass@1, achieved only a 45.3% pass rate among six leading large models. Additionally, tests found no ideal positive correlation between model performance and inference costs. The SciUniverse benchmark comprises 92 Level 1 tasks covering core research processes such as sample preparation, instrument control, and protocol adaptation. Unlike the industry trend toward fully robotic autonomous laboratories, C5R's system integrates human operators into the workflow, simultaneously controlling instruments and issuing instructions to humans. Its core focus lies in evaluating whether models can translate research objectives into executable experimental procedures for both humans and machines while guiding subsequent decisions based on experimental results. C5R argues that the current bottleneck in AI-driven autonomous research has shifted from computation to physical execution. To enable AI to conduct genuinely impactful scientific research, it must be provided with physical workspaces connecting physical instruments and experiments, rather than systems that merely process digital data.