17 Leading AI Agents Take Part in Quantum Engineering Tests, Majority Underperform with Vast Performance Disparities
5 day ago / Read about 0 minute
Author:小编   

At present, AI agents have been granted access to real-world quantum hardware operation settings. Nevertheless, depending solely on a solitary correct outcome is not enough to guarantee their dependable functioning, since hallucinations might result in substantial physical damages. To tackle this problem, relevant teams have introduced the idea of Verified Autonomy and established a virtual quantum laboratory, Quantum-Harbor, which is furnished with real-world physical dynamic feedback. Additionally, they have created the QIQCBench evaluation set, encompassing multi-dimensional quantum engineering tasks. This evaluation set utilizes an innovative evidence-binding scoring system to confirm the evidence backing for AI conclusions, the rationality of resource utilization, and the physical robustness of the results. The team carried out stress tests on 17 top mainstream AI agents, uncovering notable differences among various models. Most models proved to be unreliable in aspects such as resource limitations and reasoning in unfamiliar physical scenarios. Only the leading models showcased the viability of autonomous quantum engineering. Presently, the virtual experimental environment and evaluation suite are accessible to the public, and a relevant challenge competition has been initiated to further promote AI's autonomous abilities in precision science domains.