Summary: “Law Professors Prefer AI Over Peer Answers” is the title of a blind-evaluation study that Stanford Law School released on June 1, 2026. Finally, in the interest of a level field, AI output was tuned in advance to resemble human answers in length and structure — removing any tell based on formatting — and the team leaned on multiple evaluation methods that could check one another.
“Law Professors Prefer AI Over Peer Answers” is the title of a blind-evaluation study that Stanford Law School released on June 1, 2026. Sixteen law professors from U.S. law schools wrote answers to 40 contract-law questions, then blind-graded an anonymized pool in which AI-generated responses were mixed in with their own — never knowing which was which. Across close to 3,000 blind comparisons, AI came out ahead in 75% of the pairings.
The result carries weight for reasons that go beyond AI topping one more benchmark. The people handing out scores belong to a profession that dissects weak arguments for a living: law professors. What got judged was tutoring answers, not exam performance. And the share of AI responses judged pedagogically harmful or misleading came to just 3.5%, below the 12% recorded for answers written by professor peers. Julian Nyarko, a Stanford Law professor, led the work through the Legal Innovation through Frontier Technology Lab, known as liftlab; liftlab researcher Alejandro Salinas served as first author, and the co-author list includes Sarath Sanga of Yale Law alongside scholars from NYU, the University of Chicago, and other institutions.
The Design Behind the Study
The setup rests on three deliberate choices. To start, the prompts stayed ordinary: 40 contract-law questions of the sort a student would genuinely raise after class or in office hours, not competition stumpers. Next, nobody knew what they were scoring: each professor drafted answers first, then rated anonymized submissions with nothing revealing whether AI or a colleague had written them. Finally, in the interest of a level field, AI output was tuned in advance to resemble human answers in length and structure — removing any tell based on formatting — and the team leaned on multiple evaluation methods that could check one another.
A panel of sixteen professors and around 3,000 anonymized comparisons will never be mistaken for a large sample. The blind protocol, though, strips out brand halo and identity bias — the two confounds most “AI beats X” research leaves wide open. The calibration to human length and structure does real work as well: left uncalibrated, graders habitually reward wordiness or punish unfamiliar formatting, and the matchup drifts toward whichever side happened to put more words on the page.
The Numbers That Matter
- In 75% of pairings, AI came out ahead of the professor-drafted answer
- Overall, AI sat level with the strongest human instructor in the study
- The share of AI answers marked pedagogically harmful or misleading was 3.5%; for peer-written answers it reached 12%
Number three draws the least commentary and may be the one that counts. Grading blind, professors voiced fewer safety concerns over AI answers than over what their colleagues had written — the mirror image of the widespread hunch that AI is what will mislead students.
The Systems Put to the Test
On the system side, the study looked at commercial tutoring products along with Google’s NotebookLM, and performance was uneven across them. No full model list appears in the official press release; the full paper can be read as a PDF, and it also lives on SSRN (abstract 6849678). The detail worth underlining: everything tested was an off-the-shelf product rather than a model built for the occasion. The same tools sit within any student’s reach today, which makes these findings far less hypothetical than a typical lab evaluation.
Where the Study Stops, by Its Own Authors’ Account
Nyarko made a point of stressing that the study measured answer quality alone — “how to implement these tools to most effectively improve student learning is still an open question.” Put plainly: an AI that answers well is not proof that students taught with AI learn more. The blind protocol has edges, too. Every one of the 40 questions comes from a single subject, contracts, and a panel of 16 professors is too small to stand in for legal education as a whole, never mind other fields.
Once published, the study drew heavy traffic on Hacker News (417 points, 356 comments), and the bulk of the debate circled the same gap: quality of answers versus educational outcome.
Implications for EdTech
Three takeaways. One: entry-level Q&A looks like the earliest candidate for large-scale AI adoption in higher education — the demand recurs constantly, it eats instructor hours, and its quality carries a blind-testable signal. Two: the moment AI answers outscore the instructor’s own under blind conditions, the institutional debate stops being “do we prohibit this” and becomes “how should tutoring and assessment be redesigned” — prohibitions have no power over tools every student already has. Three: with NotebookLM, a system anchored in course materials, now appearing inside a rigorous study, “AI plus your own materials” rests on measurable evidence instead of marketing copy alone. Whether AI belongs in the classroom is not something this study settles, and its authors admit as much. What it changes is where the burden of proof sits: from here on, the case against AI tutoring has to be made with data, not intuition.