As U.S. law schools scramble to set AI policy — the University of Chicago Law School unveiled its strategy just days ago, including plans to ban laptops from all first-year core courses — new research from Stanford Law School offers a data point that complicates the picture: in a rigorous blind study, law professors themselves preferred AI-generated answers to student questions over answers written by their professional peers, roughly three times out of four. And when both types of answers were flagged as harmful to student learning, professors chose the human-written answers far more often.

The study, published June 1, 2026, by Stanford Law Professor Julian Nyarko and co-authors at Yale, New York University, the University of Chicago, and other institutions, is among the first to test AI against professional human expertise in a domain that resists simple right-or-wrong grading. Contracts law requires synthesizing competing arguments, applying doctrine to novel facts, and explaining legal reasoning in ways that build student judgment rather than supply answers. That, the researchers argued, is precisely what makes the finding hard to dismiss.

"We focused on law precisely because it requires judgment, nuanced reasoning, and the ability to navigate ambiguity — not just factual recall," said Nyarko, who directs Stanford Law School's Legal Innovation through Frontier Technology Lab, or liftlab.

Blind Study Design: Professors Graded Their Own Peers Without Knowing It

The study enrolled 16 contracts law professors at 14 top U.S. law schools, all teaching from the same casebook. Each professor wrote office-hours-style answers to 40 representative student questions — the kind of question a first-year student might ask after a confusing lecture on consideration or promissory estoppel. Those same questions were then posed to AI systems, primarily Google's Gemini 2.5 Pro and its casebook-grounded variant, Google's NotebookLM.

The evaluation phase was designed to prevent the professors from knowing whether the answer they were rating came from a colleague or a machine. Nearly 3,000 pairwise comparisons were assessed blind, with each professor choosing which of two anonymized answers they would rather give a student. The team also calibrated AI response length and structure to match human answers, eliminating formatting as a tell.

The result: AI won 75% of the head-to-head matchups, and its performance was comparable to the single best human instructor in the study.

Perhaps more revealing than the win rate was the "pedagogically harmful" flag. Professors rated AI responses as potentially misleading or likely to confuse students in just 3.5% of cases. The same flag was applied to peer-written answers at a rate of 12% — with one instructor's answers judged as harmful by colleagues in nearly 40% of cases. The arithmetic is striking: a law student asking a randomly selected professor for help had roughly a one-in-eight chance of receiving an answer their colleagues would consider actively counterproductive.

Sarath Sanga, a Yale Law School professor who co-authored the study, summed up the structural challenge. In most domains where AI gets benchmarked, he noted, there's a correct answer. Legal reasoning is different — two competing arguments can both be well-constructed, and what the study asked was whether AI could meet the professional standard that lawyers apply when evaluating each other. "In this case, the answer was yes," Sanga said.

Why Adding More Context Made the AI Worse

One of the study's more technically significant findings involves what did not work: feeding the model the casebook itself.

The research team tested both the standard version of Gemini 2.5 Pro and NotebookLM, a casebook-grounded variant that ingests a specific text as its primary knowledge source. The intuition behind NotebookLM was reasonable — if the AI knows exactly which casebook students are using, it should give more relevant, on-point answers. In practice, the general-purpose model outperformed the context-enriched one.

Nyarko and his team hypothesize that injecting the full casebook introduced noise rather than clarity — a pattern researchers call the "lost in the middle" problem. This is a documented architectural limitation of transformer-based language models rooted in how they assign attention weights across long inputs. Transformer architectures using Rotary Position Embedding (RoPE), the positional encoding standard found in most modern large language models, exhibit a long-term decay effect: tokens near the beginning and end of an input context receive more attention than tokens in the middle, regardless of their relevance. When a full casebook is loaded as context, the model's attention is stretched across hundreds of thousands of tokens. The doctrine directly relevant to a specific student question may land precisely in the middle of that long context — and receive less attention than the casebook's preface or its final appendices.

Research published at Stanford and the University of Washington has documented degradation of more than 30% on retrieval tasks when relevant information shifts from the edges to the middle of the context window. The practical implication for legal AI tool design is counterintuitive: adding more information does not automatically make an AI more helpful. A targeted retrieval approach — locating the specific relevant passage and providing only that — may substantially outperform naive whole-document ingestion.

This constraint belongs in any honest evaluation of AI legal tutoring tools, because it defines a failure mode that doesn't disappear as models get larger. It must be engineered around, not assumed away.

Where the Risk Hides: AI Bias in Legal Advice

The optimistic tutoring results sit alongside a separate body of work from liftlab that documents a harder problem: systematic discrimination embedded in AI responses.

In a paper co-authored with Amit Haim and Alejandro Salinas, Nyarko's team applied a methodology drawn from classical employment discrimination research — the social science audit design — to probe whether leading large language models, including GPT-4, treat users differently based on names that signal race or gender. The methodology involved submitting identical scenarios to the same model while varying only the name of one party, then measuring whether the advice changed.

The results documented what researchers classify as disparate treatment: AI systems giving systematically worse outcomes to users with names commonly associated with racial minorities or women. In one tested scenario involving negotiating the purchase price of a used bicycle, the model recommended paying a lower price to a seller with a name coded as white. Names associated with Black women received the least favorable outcomes across the board. The bias persisted across 42 different prompt templates and multiple models — a pattern suggesting systemic behavior rather than isolated anomalies.

The liftlab team has since pushed further into the mechanism. Using a technique called model pruning — selectively deactivating specific neurons identified through mechanistic interpretability methods — researchers Sibo Ma, Salinas, Peter Henderson, and Nyarko showed in a February 2025 paper (arXiv:2502.07771) that bias-driving neurons can be identified and removed without significantly degrading overall model performance. The caveat is important: the biases are highly context-specific. A model pruned to reduce bias in bicycle-price negotiations may still exhibit bias in a hiring scenario. This context-specificity has a legal implication the researchers flag explicitly: it is difficult to hold foundation model developers liable for bias they cannot anticipate across all downstream deployments. A more tractable enforcement target is the company actually deploying the model in a specific use case.

Law Schools Are Divided on What to Do Next

The week Stanford's study circulated, the legal education establishment was already divided over a different question: should law schools restrict AI access, or expand it?

The University of Chicago Law School's July 9 announcement split the difference. Its new strategy will ban all electronic devices — laptops, tablets, phones — from core first-year classes starting this fall, on the grounds that students using AI to gather and regurgitate material in real time are less likely to develop independent reasoning skills. Chicago's approach is, as legal education commentator Derek Muller wrote in this week's roundup, "hardly anti-AI" — elsewhere in the curriculum, students will be required to learn to use AI tools to prepare for practice. But it explicitly rejects the notion that more access to AI assistance during instruction is uniformly beneficial.

This is the genuine tension the Stanford data opens rather than closes. The liftlab study measures the quality of written answers produced on demand; it does not measure what happens to a student's own reasoning capacity if they receive better answers without being required to produce better thinking. A 2025 MIT Media Lab study found that participants who used ChatGPT to write essays showed lower brain engagement, weaker memory recall, and reduced critical thinking than those who worked without AI assistance — a pattern the researchers described as "cognitive debt."

Nyarko is careful to draw the boundary of what his study does and does not establish. "Our study evaluates the quality of answers given by AI tools," he said. "But how to implement these tools to most effectively improve student learning is still an open question. We're not advocating for wholesale adoption of AI tutors. But our data suggests that blanket skepticism may be equally unwarranted."

AI as a Research Tool and a Socratic Sparring Partner

Beyond the classroom, liftlab is using AI to change the pace of legal scholarship itself.

Empirical legal research has traditionally required slow, manual work: reading thousands of contract clauses, coding judicial opinions one by one, tracking how legal language shifts across decades of case law. The liftlab team applies large language models to these tasks at scale, testing theoretical claims that would previously have taken years to investigate. Nyarko has also described using AI as an intellectual sparring partner in his own research process — a Socratic interlocutor that can stress-test hypotheses and surface counterarguments before they reach peer review.

The lab's early prototypes translate this research ambition into tools. A Contractual Drafting Risk Assessment tool draws on thousands of court opinions on contract interpretation to identify language historically prone to causing disputes, giving contract drafters a real-time signal about ambiguity risk. A separate tool is in development for Stanford's Immigrants' Rights Clinic, functioning as an AI intake specialist that can help prospective clients describe their situation and determine what kind of assistance they need.

liftlab's founding advisory partners — law technology company Harvey and law firms Cleary Gottlieb, Davis Wright Tremaine, and Vorys — ensure that these prototypes are tested against actual professional workflows, not just academic assumptions about what legal practice requires. Megan Ma, liftlab's executive director and a Stanford CodeX research fellow, has described the arrangement as providing "a research-backed, neutral space to test, refine, and develop what actually works."

Does AI Close the Access-to-Justice Gap — or Does It Encode the Gap?

The larger ambition animating liftlab's work is one the legal profession has debated for decades without resolution: high-quality legal counsel is inaccessible to the vast majority of people who need it. The Legal Services Corporation has estimated that roughly 80% of the civil legal needs of low-income Americans go unmet. A routine contract dispute, an immigration question, or a consumer protection issue can cost thousands of dollars that most individuals do not have.

If AI tools can match human professional quality in instruction, they may be able to match it in advice — not by replacing lawyers, but by extending the reach of competent guidance to people who cannot afford an attorney at all. The contractual drafting tool is an early proof of concept: it surfaces expertise that previously lived only in the heads of experienced litigators and makes it available to anyone drafting an agreement.

But the bias research is a direct complication of that promise. An AI system that delivers worse guidance to users with Black-coded names is not democratizing legal access — it is automating, at scale, the same inequities the legal system already produces through human bias. The challenge liftlab is trying to solve is not just making AI legal tools smarter. It is making them fair enough to trust with the people who have the fewest alternatives.

Frequently Asked Questions

Can AI actually replace law professors?

Not yet — and the Stanford study is careful to say so. The finding is that professors, in blind evaluations, preferred AI-written answers in about three out of four direct comparisons. But the study measured only the quality of short written answers to student questions, not the full experience of instruction, relationship, feedback over time, or the development of a student's independent reasoning. The University of Chicago Law School's July 2026 AI strategy, which restricts device use in first-year classes precisely because AI-assisted passive learning may reduce reasoning development, reflects a legitimate concern the Stanford data does not address.

Is AI biased when it gives legal advice?

Documented bias has been found in multiple large language models in legal-adjacent scenarios. Stanford liftlab researchers used the same audit methodology employed in landmark employment discrimination studies and found that AI models, including GPT-4, gave systematically worse outcomes to users with names associated with racial minorities and women. The bias was consistent across 42 different prompt versions and multiple models. Newer research from the same lab shows that bias-associated neurons can be surgically removed through model pruning, but the bias is highly context-specific — removing it in one scenario does not guarantee its removal in others.

Why did adding the casebook make the AI worse, not better?

This reflects a documented architectural limitation called the "lost in the middle" problem. Transformer language models using Rotary Position Embedding, the positional encoding standard in most modern large language models, apply more attention to tokens near the beginning and end of a long input than to those in the middle. Loading an entire casebook as context stretches attention across hundreds of thousands of tokens; the doctrine relevant to a specific student question may end up in the middle of that context, receiving less weight than less relevant material at the edges. Research has shown this can cause performance degradation of more than 30%. Well-designed AI tutoring tools should retrieve only the specific relevant passage rather than loading entire source documents.

What practical steps could make AI legal tools more trustworthy for people who can't afford lawyers?

Access-to-justice applications of AI require at minimum: bias auditing across demographic groups before deployment (not just at the foundation model level, but by the deploying organization in the specific use case); scope limitations that prevent the tool from answering questions outside its reliable knowledge; clear escalation pathways to human attorneys for complex or high-stakes situations; and transparency to users about what the tool is and is not. Stanford liftlab's Contractual Drafting Risk Assessment tool and its prototype for the Immigrants' Rights Clinic represent early attempts to apply these principles in practice.