Economy
AI Outscores Human Peer Reviewers on Every Measure, a New Study Finds
By Bear Creek Staff, . Bear Creek Capital Network.
A study by Gary King and Steven Worthington, flagged by Marginal Revolution, concludes that AI-generated peer reviews now outperform human expert reviews across every measure the researchers could build, and that the margin is not close.
What the study found
The researchers compared AI-generated reviews against those written by well-compensated human experts, reviews that journal editors and authors had already judged to be of unusually high quality. The AI reviews came out ahead on every metric. The model used was Claude Opus 4.8, which King and Worthington explicitly describe as not among the best currently available, meaning the performance gap they measured may already be wider than their paper captures.
A secondary finding is the more pointed one for anyone who commissions or relies on peer review. The study found that errors in the papers under review ran in the ideological direction of the papers' own authors 97% of the time. Marginal Revolution notes that this figure aligns with an earlier human assessment by researcher Phil Magness, who reached a similar conclusion through an extensive review of the relevant body of work. The AI analysis confirmed what a careful human reader had already argued.
Why it matters beyond academia
Peer review is the quality-control layer underneath a large share of the economic and policy research that informs regulatory decisions and shapes the forecasts that analysts cite. If that layer is systematically biased in the direction of authors' prior views, and if AI can now detect that bias more reliably than human referees, the credibility of the research pipeline becomes a practical question for anyone who uses peer-reviewed work as an input.
Marginal Revolution's Tyler Cowen calls on the National Science Foundation, prize committees, universities, tenure committees and journals to conduct their own AI-enabled audits, and notes that so far the appetite for doing so has been limited.
What is still unknown
The study does not establish how the AI reviews perform across all fields or whether the ideological-error finding generalises beyond the specific body of work examined. The paper's scope, the journals covered, the disciplines included and the period of review are not detailed in the summary available. Whether the 97% figure holds in other areas of research is an open question the study does not answer.
What the study does establish is a proof of concept with a model the authors themselves describe as below the current frontier. For anyone who uses peer-reviewed research as an input, the practical question is whether the human review process they are relying on is catching the errors that matter, or missing them in a predictable direction.
Bear Creek Capital Network publishes content for informational and educational purposes only. Nothing in this article constitutes financial, investment, tax or legal advice, or a recommendation to buy or sell any security. Consider your own circumstances and consult a qualified professional before making financial decisions.