ScienceVision Benchmark: Frontier Large Models Fall Short in 3D Scientific Understanding
21 hour ago / Read about 0 minute
Author:小编   

Earlier, a video of GPT-6 Astra generating CAD models sparked heated discussion, with its BenchCAD score reaching 95.9%, fueling debate over whether CAD engineers might be replaced. But professional engineers pointed out that such generated results may look realistic yet do not necessarily meet actual engineering requirements. The recently launched ScienceVision—an interdisciplinary visual 3D understanding and reasoning benchmark—covers five scientific fields: biology, chemistry, medical imaging, architecture, and mechanical CAD. Spanning 22 subfields in total, it includes 250 questions built on real 3D data, created by 66 domain experts who collectively invested nearly 500 hours. The benchmark uses deterministic rule-based verification of answers, requires no language model as a judge, and can also serve directly as a reinforcement learning environment. It simulates the real 3D observation and operation scenarios that experts face in various fields, requiring models to autonomously operate 3D objects to complete observation, measurement, and analysis tasks—putting AI's 3D scientific observation and understanding capabilities to the test. Currently, 12 frontier large models have participated in the evaluation. The highest-scoring model, GPT-6-Astra paired with Codex, scored only 0.53, and no model led simultaneously across all five disciplines; even when equipped with interactive inspection tools, most models showed very limited improvement—exposing the fact that current frontier large models still have obvious shortcomings in 3D observation and reasoning for real scientific tasks. This benchmark, for the first time under a unified framework, measures the capability boundary of AI's 3D scientific observation, filling the gap in systematic scientific 3D reasoning evaluation.