Google's Gemini 4 Argon model has attained an impressive 97.5% score on the ARC-AGI-3 test, a benchmark widely regarded as the ultimate challenge for Artificial General Intelligence (AGI). With this achievement, it has become only the second AI lab, following OpenAI, to exceed the 95% threshold on this rigorous assessment. The model also demonstrated exceptional performance in the ARC-AGI-1 and ARC-AGI-2 tests, achieving a perfect completion rate across all three iterations of the exam suite.

Since the launch of the ARC-AGI-3 test seven months ago, scores for state-of-the-art models have skyrocketed from less than 1% to near-perfect marks. Unlike conventional benchmarks, the ARC-AGI-3 test is structured as an interactive game environment devoid of explicit instructions or predefined commands. This unique design aims to evaluate a model's capacity for on-the-fly learning and adaptation when confronted with unfamiliar tasks—a hallmark of true AGI.

Gemini 4 Argon not only outperformed human baselines in terms of efficiency but also matched or even surpassed the performance limits of speedrun players on certain levels. However, a notable finding emerged regarding the model's performance variability: under different settings, the same model exhibited significantly different scores. Specifically, it achieved 97.5% under a vendor-customized Provider Adapter console setup but only 49.8% under the official unified Standard harness setup. The top-ranked GPT-6 Astra model displayed a similar pattern.

This observation underscores that competition in the current AI landscape extends beyond the intelligence of the models themselves. It encompasses the entire system, including the model and its console environment. Therefore, when reviewing AI rankings, it is crucial to consider not only the scores but also the specific harness or console setup used during testing. This holistic perspective ensures a more accurate and fair assessment of AI capabilities.