AI reviewers are here - we are not ready

"The latest experiment of openRxiv, the non-profit organization in New York City that runs the bioRxiv and medRxiv repositories, is perhaps its most provocative yet. Last month, openRxiv announced that it was integrating a reviewing tool driven by artificial intelligence into its preprint sites. The tool, from the start-up company q.e.d Science in Tel Aviv, Israel, offers rapid AI-generated feedback (typically within 30 minutes) on biomedical manuscripts - judging originality, identifying logical gaps and suggesting more experiments and tweaks to the text."

"The allure of an AI reviewer is undeniable. For any scientist who has languished for months awaiting a decision, or decoded a snarky remark from a hostile 'Reviewer #2', an algorithmic alternative sounds like the efficiency upgrade that publishing desperately needs. Large language models (LLMs) can provide feedback in seconds, potentially without conflicts of interest. But there is a big difference between a process that is efficient and one that is valid."

"Peer review has two purposes. It must validate the majority of routine scientific work - careful studies that test predictions and fill gaps in understanding - by scrutinizing statistics, methods and logical coherence. It must also recognize the rare discovery that introduces anomalous findings or challenges established frameworks, by assessing not whether the rules were followed, but whether they still apply."

openRxiv integrated an AI-driven reviewing tool from q.e.d Science into bioRxiv and medRxiv to provide rapid feedback on biomedical preprints, judging originality, logical gaps, suggested experiments, and text tweaks. AI reviewers can dramatically reduce turnaround, check statistics, catch plagiarism, and verify citations, thereby transforming routine review tasks. However, AI may struggle to assess truly novel or anomalous findings, can hallucinate, inherit biases, be vulnerable to adversarial manipulation, and incentivize gaming of publication incentives. Safeguards require transparency about methods, rigorous evaluation, human oversight for non-routine work, gradual deployment, monitoring for harms, and community governance to preserve scientific validity.

Read at Nature