A new AI benchmark has produced the first direct, quantified evidence that frontier AI systems have surpassed human experts at the core of scientific research work — designing experiments and interpreting their results — reaching expert-level performance at less than half the compute cost and at roughly one-thirtieth the per-attempt expense of a human researcher. The finding, from P-Zero Research's TasteVal benchmark published October 5 measures what the field calls "research taste" — the combination of skills that distinguishes productive scientific research from capable task execution.

The benchmark's best result came from Anthropic's Claude Opus 5.5, which achieved a compute multiplier of 2.30 against a human expert baseline of 24 researchers recruited from OpenAI, Google DeepMind, Nvidia, and Microsoft Research. The 95 percent confidence interval for that figure spans from 1.15 to 4.37 — wide enough to span the difference between marginal superiority and decisive advantage, but with the entire range above the parity threshold of 1.0. Across multiple frontier model generations, the capability TasteVal measures has roughly doubled every three months since December 2025, compared to a doubling time of approximately 14 months before that period — a phase transition the benchmark's authors treat as among their most significant findings.

The benchmark deliberately and explicitly excludes one dimension of research judgment: choosing which problem to work on in the first place. That gap, which TasteVal's own authors identify as a remaining and unmeasured frontier, is the central limit on what the results can say about how close AI systems are to automating scientific research entirely.

What Recursive Self-Improvement Means

Recursive self-improvement (RSI) is the threshold at which an AI system can train a more powerful successor entirely without human involvement — not automation of individual research tasks, but a closed loop in which a smarter model helps build a still-smarter model, potentially compressing decades of incremental progress. The concept has a lineage reaching back to I.J. Good's 1965 paper on ultraintelligent machines and Alan Turing's earlier "child machine" proposal; Eliezer Yudkowsky popularized the term "seed AI" to describe the self-replicating recursive improvement form of the phenomenon.

The question of when RSI becomes achievable has gained urgency as measurable proxies have appeared. Anthropic publicly reported that over 80 percent of merged code at the company was generated by AI models by May 2026. OpenAI announced in September 2026 that its research agents were logging approximately 3.1 agent-workdays of effort for each 8-hour human workday, though over half of successful 4-to-8-hour tasks still required human intervention. A September 28 paper co-authored by researchers including Geoffrey Hinton, Yoshua Bengio, Jakub Pachocki, and Jack Clark found that AI's share of lightly supervised R&D work had risen from 1 percent to 26 percent between March and August 2026.

TasteVal enters this landscape as the first benchmark specifically designed to measure the judgment component of research work — not task completion speed or code generation, but the skill of designing experiments and interpreting what they show.

How TasteVal Measures Research Judgment

The architecture behind TasteVal is built around a deliberate conceptual separation. Scientific research judgment, the benchmark's authors argue, has three interlocking components: identifying which problems are worth pursuing; designing experiments to answer those problems; and correctly interpreting what the experimental results mean. TasteVal measures the second and third of these. The first — problem selection — is explicitly outside the benchmark's scope in this iteration.

To isolate judgment from coding ability, TasteVal uses a Researcher/Coder split architecture. The AI model under evaluation acts as the Researcher: it proposes experiments and draws conclusions from results. A fixed external Coder agent handles all implementation on a single Nvidia H100 GPU. Each run is capped at 40 GPU-hours of compute or 120 wall-clock hours, whichever comes first. This structure means a model cannot compensate for poor experimental judgment by writing faster or more efficient code — the coding work is held constant across all models.

The benchmark spans eight tasks drawn from frontier AI development work, covering training data selection, pretraining, fine-tuning language models, preference modeling, and defense against adversarial prompts. The human baseline was constructed from 24 experts who had recently held research positions at OpenAI, Google DeepMind, Nvidia, and Microsoft Research, with at least two experts assigned to each task. TasteVal uses the best human result per task as the bar to beat — not the average — which sets a high baseline while also meaning the benchmark measures against expert floor performance rather than median researcher output.

Performance is expressed as a compute multiplier: the ratio of compute a human expert used to complete a task, divided by the compute the model required to reach the same score. A multiplier above 1.0 means the model reached expert-level performance using less computation than the human. Claude Opus 5.5's multiplier of 2.30 means it achieved expert-level results while spending roughly 43 percent of the compute a human expert used, at approximately one-thirtieth the per-attempt cost.

The Phase Transition Since December 2025

Among TasteVal's most significant findings is not the absolute level of performance but the shape of the improvement curve. Before approximately December 2025, the compute multiplier for research taste in frontier models doubled roughly every 14 months — consistent with gradual capability improvement. Since that period, the doubling time has compressed to approximately 3.0 months, with a 95 percent confidence interval of 1.7 to 5.0 months.

The TasteVal paper does not name a cause for this phase transition. One possible explanation, not stated in the paper but consistent with the timing and the broader institutional data, is that this inflection corresponds to the period in which AI systems began using other AI systems as research instruments in significant volume. Anthropic's own report on AI-generated code share — which rose from low single digits in early 2025 to over 80 percent of merged code by May 2026 — and the Cambridge paper's finding that AI's R&D automation share rose from 1 to 26 percent in the five months from March to August 2026 together suggest a feedback dynamic in which AI assists in AI research, accelerating the development of AI research capability itself. If that mechanism is driving the TasteVal doubling rate, it would not necessarily slow as human researcher productivity saturates — which is what makes the phase transition the most consequential element of the benchmark's findings.

The TasteVal paper is explicit that these results may overstate the trend. Because TasteVal tasks are scored quickly and cheaply, they are precisely the kind of task that AI labs find easiest to optimize for — raising the possibility that the steep improvement curve reflects benchmark-specific optimization as much as genuine underlying capability growth.

Where the Benchmark Deliberately Stops

The capability TasteVal measures is not the whole of what research taste means. Choosing which scientific problem to pursue — from a vast, open-ended space of possible research directions — is generally considered the deepest expression of scientific judgment, and it is explicitly outside TasteVal's scope.

This gap is substantiated by separate research. A 2026 study on AI post-training behavior found that AI agents in research settings locked into their initial strategy early in a task, with only 2.1 percent of training runs revising their committed approach; experience improved execution but not strategy revision. The paper identified the missing capability as the decision to reopen a committed strategy and consider an alternative — precisely the kind of problem-level judgment that TasteVal's task structure removes from the model's plate by pre-specifying the research goal.

The Anthropic alignment team's own separate TASTE benchmark, published August 28, provides a complementary data point. Anthropic's TASTE (not to be confused with TasteVal) measures AI models' ability to select the more promising of two AI safety research proposals — directly testing problem selection, the dimension TasteVal excludes. On that benchmark, the best-performing AI model tested (Claude Fable 5) scored approximately 60 percent against an estimated 77 percent for human researchers, a gap that remains substantial. Taken together, TasteVal and Anthropic's TASTE mark the boundary: AI has likely crossed or approached expert level at experiment design and result interpretation; it has not yet done so at research agenda selection.

There is also a structural limitation in TasteVal's human baseline. Using the best human result per task, rather than representative results, means the benchmark measures against the floor of expert performance — not the ceiling of what the best researchers in the world would produce with unlimited time and context.

How Clark Applied These Findings to His Forecast

Jack Clark, co-founder of Anthropic and author of the widely-read Import AI newsletter, updated his forecast for RSI in the newsletter's 476th edition, published October 12. Clark wrote that he now believes RSI — which he defines as a closed-loop system capable of training a more powerful successor with "no-human-involved" training — is now more likely in 2027 than in 2028, a compression of the timeline he had set as recently as May.

In May, in Import AI #455, Clark had put the probability at roughly 60 percent by end of 2028 and 30 percent by end of 2027. His October update revised the directional judgment without updating the probability figures — he stated only that his "more likely" year had shifted. The distinction matters: Clark is forecasting possibility, not inevitability. Whether any lab pursues RSI if and when it becomes technically possible depends on compute availability, safety frameworks, competitive dynamics, and regulatory context.

Clark cited two categories of evidence for the update: TasteVal's findings on AI research capability, and a wave of reports of AI systems producing what he described as "creative-seeming math breakthroughs."

A disclosure is worth stating clearly. Clark co-founded Anthropic, the organization whose model leads the TasteVal leaderboard. The benchmark comes from P-Zero Research, an independent group, but the most prominent person drawing forward-looking conclusions from it is affiliated with the top-performing lab. Clark's newsletter is his personal publication and does not represent Anthropic's official position. Clark also co-authored the September 28 Cambridge intelligence explosion paper in a personal capacity — not as an Anthropic representative — alongside Hinton, Bengio, and others who have called for governments to establish mandatory reporting on AI R&D automation levels, independent auditors inside AI companies, and emergency capability-pause mechanisms.

Clark's NAIAC membership, his 2023 briefing of the UN Security Council on AI threats, and his decade of work on AI policy mean his updated forecast carries weight in governance discussions — and that weight should be assessed against both the benchmark data he cites and the institutional position from which he makes it.

What the TasteVal Paper's Own Projections Show

The TasteVal paper goes further than Clark's newsletter in spelling out potential implications, using what the authors call the "AI Futures Model" developed by the AI Futures Project. The projections are labeled throughout as "naive extrapolation rather than a forecast" — the authors are explicit that these numbers should not be treated as predictions.

If the observed TasteVal improvement rate continues uninterrupted, the model calculates that the probability of what the paper calls a "taste-only singularity" — a scenario in which research taste improvements alone trigger recursive acceleration — would rise from 51 percent to 88 percent. The median projected arrival of artificial superintelligence (ASI) in this model shifts from 2030.5 to 2028.9; the probability of ASI before 2030 shifts from 46 percent to 69 percent.

The caveats the authors attach to these projections are substantive: the doubling rate is uncertain (confidence interval: 1.7 to 5.0 months), the tasks may be gameable, and the extrapolation applies only if the observed trend holds and only to the taste component of the broader RSI picture. Problem selection, multi-week planning, and real-world experimental execution are not reflected in these numbers.

The Counterarguments

The benchmarks' findings are contested, and the primary challenges point to exactly the limits already identified in the benchmark's design.

AI safety researcher Herbie Bradley's argument, which became a touchstone in the RSI debate following Clark's May essay, holds that AI systems had not demonstrated "research taste, genuine creativity, and coherent long-term research planning" at a level relevant to RSI. TasteVal directly challenges the experiment-design component of that claim. It says nothing about the third element Bradley named: coherent long-term research planning and problem selection.

The benchmark's Researcher/Coder split architecture, while solving the problem of isolating judgment from coding ability, also creates a task structure that differs from how actual research is conducted. Real research involves choosing what to build and revising that choice in light of unexpected results — precisely the capability the post-training study found AI agents were missing. A model optimized to perform well within a pre-specified research goal may not generalize to open-ended research agenda setting.

Ajeya Cotra at METR distinguishes three thresholds in AI research capability: adequacy (AI can do some research tasks without humans), parity (AI-only matches human-AI collaboration), and supremacy (AI-only outperforms human-AI collaboration). TasteVal's results speak to experiment-level adequacy and possibly parity on specific task types, not to overall research parity. Helen Toner at CSET has argued that using AI tools in AI research does not qualify as RSI under the classic definition, which requires that no humans are involved — a threshold not approached by any current system.

Broader Institutional Context

Clark's revised timeline arrives in a period of rapid institutional repositioning. Anthropic publicly acknowledged in June 2026 that RSI has not yet been achieved but warned that it "could come sooner than most institutions are prepared for." OpenAI, releasing its research automation data in September, was explicit that reaching measurable automation in research tasks does not mean RSI is a goal worth pursuing without human oversight in place.

The Cambridge intelligence explosion paper — the one co-authored by Clark, Hinton, Bengio, Pachocki, Eric Horvitz, and Dawn Song, among more than 20 researchers in total — made concrete policy asks tied directly to the data it presents. The paper's call for government-mandated reporting on AI R&D automation, independent auditors embedded in AI labs, and emergency pause mechanisms was grounded in the empirical finding that AI's share of lightly supervised R&D work had risen from 1 percent to 26 percent in five months. TasteVal's results, published three weeks after that paper, add a specific capability measurement to the empirical backdrop the Cambridge authors were working from.

What has changed is the evidentiary basis of the debate. The argument about RSI timelines has historically been driven by extrapolation from general-purpose benchmarks — reasoning scores, coding ability, task-completion duration. TasteVal represents an attempt to build a purpose-built instrument for the specific capability that matters most: whether AI systems can function as productive research scientists, not just productive coders or document processors. Its findings are not conclusive. Its methodology is contested. Its extrapolations are explicitly labeled speculative. But as a signal that the RSI debate has moved from abstract forecasting to measurement, it is a shift worth tracking carefully.

Frequently Asked Questions

What exactly does TasteVal measure, and how is it different from other AI benchmarks?

TasteVal measures two of the three components of scientific research judgment: experiment design (proposing which experiments to run in order to answer a specified research question) and result interpretation (drawing correct conclusions from experimental outcomes). It uses a Researcher/Coder split, in which the AI model proposes experiments while a separate fixed agent handles coding, specifically to isolate judgment from implementation skill. It does not measure problem selection — choosing which research question to pursue — which most benchmarks on AI research capability have also not specifically targeted. Its novel metric, the compute multiplier, expresses model performance as a ratio: how much compute a human expert needed divided by how much compute the model needed to reach the same score. A multiplier above 1.0 means the model is more compute-efficient than the human expert.

Has AI actually surpassed human experts at AI research tasks?

On the specific tasks TasteVal covers — designing experiments and interpreting results within pre-specified research goals — the benchmark finds that the best current AI model (Claude Opus 5.5, at 2.30x compute multiplier) has surpassed the human expert baseline used in the study. That baseline is the best result achieved by any human expert among 24 recruited from major AI labs, not the average. The finding is specific to these two dimensions of research judgment. A separate benchmark, Anthropic's TASTE safety evaluation, found AI models scoring approximately 60 percent versus 77 percent for humans on research proposal selection — the dimension TasteVal excludes — indicating that problem selection remains below human expert level.

What is recursive self-improvement in AI, and what would it actually take to achieve it?

Recursive self-improvement refers to a state in which an AI system can train a more capable successor model without any human involvement — not just faster or cheaper AI-assisted research, but a genuinely closed loop. Under the definition Jack Clark and others use, achieving RSI would require that AI systems can identify which research problems to pursue, design and run experiments to solve them, interpret results correctly, and apply those results to build a better AI system — all without a human in the loop at any stage. TasteVal addresses two of those steps. Problem selection, multi-week research planning, and the ability to revise committed research strategies are among the capabilities not yet demonstrated at the levels RSI would require.

What should I understand about the doubling rate TasteVal reports before treating it as a prediction?

The 3.0-month doubling time for research taste capability applies only since approximately December 2025, according to the TasteVal paper. Before that, the doubling time was approximately 14 months. The paper treats the recent rate as a phase transition, not an established baseline — the 95 percent confidence interval spans 1.7 to 5.0 months, meaning the true doubling time could be significantly slower than the central estimate suggests. The paper's authors explicitly label their extrapolations as "naive extrapolation rather than a forecast" and note that the benchmark's task structure makes it particularly susceptible to optimization by AI labs, potentially inflating the observed improvement rate.