{"id":24276,"date":"2026-07-16T17:12:07","date_gmt":"2026-07-16T17:12:07","guid":{"rendered":"https:\/\/science-dao.org\/?p=24276"},"modified":"2026-07-16T17:12:17","modified_gmt":"2026-07-16T17:12:17","slug":"ai-review","status":"publish","type":"post","link":"https:\/\/science-dao.org\/zh\/ai-review\/","title":{"rendered":"AI Peer Review vs Human Peer Review: Strengths, Biases, and Failure Modes"},"content":{"rendered":"<div id=\"scien-3153860293\" class=\"scien-before-content scien-entity-placement\"><style>\r\n.amazon-support-link {\r\n    display: inline-flex;\r\n    align-items: baseline;\r\n    gap: 0.3rem;\r\n    padding: 0.35rem 0.55rem;\r\n    color: inherit;\r\n    font-size: 0.88rem;\r\n    line-height: 1.2;\r\n    text-decoration: none;\r\n    opacity: 0.72;\r\n    white-space: nowrap;\r\n    transition: opacity 0.2s ease;\r\n}\r\n\r\n.amazon-support-link:hover,\r\n.amazon-support-link:focus-visible {\r\n    opacity: 1;\r\n    text-decoration: underline;\r\n}\r\n\r\n.amazon-support-link small {\r\n    font-size: 0.65rem;\r\n    opacity: 0.7;\r\n}\r\n\r\n@media (max-width: 900px) {\r\n    .amazon-support-link small {\r\n        display: none;\r\n    }\r\n}\r\n<\/style>\r\n<a class=\"amazon-support-link\"\r\n   href=\"https:\/\/www.amazon.com\/?tag=vpf04-20\"\r\n   target=\"_blank\"\r\n   rel=\"nofollow sponsored noopener\"\r\n   aria-label=\"Shop on Amazon and support World Science DAO\">\r\n    Shop on Amazon <span aria-hidden=\"true\">\u2197<\/span>\r\n    <small>affiliate link<\/small>\r\n<\/a><\/div>\n<p class=\"wp-block-paragraph\">AI peer review can analyze scientific papers quickly, consistently, and at a scale that human reviewers cannot match. Human peer review, however, remains stronger at interpreting scientific significance, recognizing unconventional ideas, evaluating tacit methodological knowledge, and accepting responsibility for decisions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The most defensible model is therefore <strong>not AI replacing human peer reviewers<\/strong>, but a transparent hybrid system in which machines perform systematic checks and independent humans retain authority, contestability, and accountability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Both forms of review can fail. They simply fail in different ways:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Human reviewers may be influenced by prestige, professional relationships, ideology, fatigue, or personal rivalry.<\/li>\n\n\n\n<li>AI reviewers may hallucinate, reproduce biases from training data, misunderstand genuinely novel work, converge on similar judgments, or be manipulated by adversarial instructions embedded in manuscripts.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Understanding these differences is essential for journals, research funders, universities, decentralized science platforms, and systems such as <a href=\"https:\/\/science-dao.org\/zh\/meritocracy\/\">AI Internet-Meritocracy<\/a> that use artificial intelligence to evaluate scientific contributions.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#What_Is_AI_Peer_Review\" >What Is AI Peer Review?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#What_Is_Human_Peer_Review\" >What Is Human Peer Review?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#AI_Peer_Review_vs_Human_Peer_Review_at_a_Glance\" >AI Peer Review vs Human Peer Review at a Glance<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Strengths_of_AI_Peer_Review\" >Strengths of AI Peer Review<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#AI_Can_Review_Work_at_Scale\" >AI Can Review Work at Scale<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#AI_Can_Apply_Explicit_Criteria_Consistently\" >AI Can Apply Explicit Criteria Consistently<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#AI_Can_Detect_Routine_Errors_Humans_Miss\" >AI Can Detect Routine Errors Humans Miss<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#AI_Can_Compare_a_Paper_with_More_Literature\" >AI Can Compare a Paper with More Literature<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#AI_Can_Reduce_Some_Forms_of_Status_Bias\" >AI Can Reduce Some Forms of Status Bias<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Failure_Modes_of_AI_Peer_Review\" >Failure Modes of AI Peer Review<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Hallucinated_Criticism\" >Hallucinated Criticism<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Bias_Reproduced_from_Training_Data\" >Bias Reproduced from Training Data<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Conservatism_Toward_Genuinely_Novel_Work\" >Conservatism Toward Genuinely Novel Work<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#False_Consensus_Between_AI_Reviewers\" >False Consensus Between AI Reviewers<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Prompt_Injection_and_Adversarial_Manuscripts\" >Prompt Injection and Adversarial Manuscripts<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Confidentiality_and_Intellectual-Property_Risks\" >Confidentiality and Intellectual-Property Risks<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Automation_Bias\" >Automation Bias<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Strengths_of_Human_Peer_Review\" >Strengths of Human Peer Review<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-19\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Humans_Can_Judge_Scientific_Meaning\" >Humans Can Judge Scientific Meaning<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-20\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Humans_Can_Investigate_Ambiguity\" >Humans Can Investigate Ambiguity<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-21\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Humans_Can_Take_Responsibility\" >Humans Can Take Responsibility<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-22\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Failure_Modes_of_Human_Peer_Review\" >Failure Modes of Human Peer Review<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-23\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Prestige_and_Institutional_Bias\" >Prestige and Institutional Bias<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-24\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Conflicts_of_Interest_and_Competitive_Suppression\" >Conflicts of Interest and Competitive Suppression<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-25\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Fatigue_and_Unequal_Effort\" >Fatigue and Unequal Effort<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-26\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Resistance_to_Unfamiliar_Ideas\" >Resistance to Unfamiliar Ideas<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-27\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Inability_to_Check_Everything\" >Inability to Check Everything<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-28\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#The_Best_Model_Auditable_Human%E2%80%93AI_Peer_Review\" >The Best Model: Auditable Human\u2013AI Peer Review<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-29\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#AI_Should_Perform_Systematic_Checks\" >AI Should Perform Systematic Checks<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-30\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Humans_Should_Evaluate_Meaning_and_Responsibility\" >Humans Should Evaluate Meaning and Responsibility<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-31\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Authors_Must_Be_Able_to_Contest_AI_Findings\" >Authors Must Be Able to Contest AI Findings<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-32\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#AI_Use_Must_Be_Disclosed\" >AI Use Must Be Disclosed<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-33\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#How_AI_Peer_Review_Could_Support_Decentralized_Science\" >How AI Peer Review Could Support Decentralized Science<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-34\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Principles_for_Reliable_AI-Assisted_Peer_Review\" >Principles for Reliable AI-Assisted Peer Review<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-35\" href=\"https:\/\/science-dao.org\/zh\/ai-review\/#Conclusion_AI_and_Human_Peer_Review_Are_Complementary\" >Conclusion: AI and Human Peer Review Are Complementary<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_Is_AI_Peer_Review\"><\/span>What Is AI Peer Review?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>AI peer review is the use of artificial intelligence to analyze a scientific manuscript, dataset, research proposal, review report, or published result.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Depending on the system, AI may evaluate:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>whether the research question is clearly stated;<\/li>\n\n\n\n<li>whether the conclusions follow from the evidence;<\/li>\n\n\n\n<li>whether statistical methods appear appropriate;<\/li>\n\n\n\n<li>whether citations support the claims attributed to them;<\/li>\n\n\n\n<li>whether important literature is missing;<\/li>\n\n\n\n<li>whether the manuscript contains internal contradictions;<\/li>\n\n\n\n<li>whether figures, tables, code, and text agree;<\/li>\n\n\n\n<li>whether reporting guidelines were followed;<\/li>\n\n\n\n<li>whether the work appears novel in relation to accessible literature;<\/li>\n\n\n\n<li>whether possible plagiarism, image manipulation, or fabricated references are present.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">AI peer review should not be confused with merely asking a general-purpose chatbot to \u201creview this paper.\u201d A credible system requires document retrieval, source verification, domain-specific tools, uncertainty reporting, security controls, and an auditable review procedure.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_Is_Human_Peer_Review\"><\/span>What Is Human Peer Review?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Human peer review is the evaluation of scientific work by researchers or professionals with relevant expertise. Reviewers normally assess originality, validity, methodology, interpretation, presentation, and significance before publication or funding.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Human peer review is often described as science\u2019s quality-control mechanism. Yet it is not a mechanical certification that a paper is true. Reviewers usually examine only a limited selection of manuscripts, often without reproducing experiments, rerunning all calculations, or auditing every cited source.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Peer review is better understood as <strong>structured expert criticism under limited time and information<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That distinction matters. A paper can pass peer review and still be wrong. A valuable paper can also be rejected because reviewers misunderstand it, consider it insufficiently fashionable, or evaluate its author rather than its contents.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"AI_Peer_Review_vs_Human_Peer_Review_at_a_Glance\"><\/span>AI Peer Review vs Human Peer Review at a Glance<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Dimension<\/th><th>AI peer review<\/th><th>Human peer review<\/th><\/tr><\/thead><tbody><tr><td>Speed<\/td><td>Seconds or minutes<\/td><td>Days, weeks, or months<\/td><\/tr><tr><td>Scalability<\/td><td>Very high<\/td><td>Limited by reviewer availability<\/td><\/tr><tr><td>Consistency<\/td><td>Can apply the same checklist repeatedly<\/td><td>Varies between reviewers<\/td><\/tr><tr><td>Domain understanding<\/td><td>Depends on model, tools, and available literature<\/td><td>Potentially deep but narrow<\/td><\/tr><tr><td>Novelty recognition<\/td><td>May penalize departures from known patterns<\/td><td>Can recognize breakthroughs, but may also resist them<\/td><\/tr><tr><td>Bias<\/td><td>Training-data, model-design, and prompt bias<\/td><td>Prestige, institutional, social, ideological, and personal bias<\/td><\/tr><tr><td>Accountability<\/td><td>Cannot bear moral or professional responsibility<\/td><td>Can be identified and held responsible, depending on the system<\/td><\/tr><tr><td>Source verification<\/td><td>Strong when connected to verified databases and tools<\/td><td>Depends on reviewer effort and access<\/td><\/tr><tr><td>Manipulation risk<\/td><td>Prompt injection, poisoned data, benchmark gaming<\/td><td>Lobbying, conflicts of interest, citation coercion, favoritism<\/td><\/tr><tr><td>Reproducibility<\/td><td>The same version and configuration can be rerun<\/td><td>Independent reviewers may produce very different judgments<\/td><\/tr><tr><td>Cost per additional review<\/td><td>Potentially low<\/td><td>Considerable expert time<\/td><\/tr><tr><td>Best role<\/td><td>Screening, checking, comparison, anomaly detection<\/td><td>Interpretation, significance, responsibility, final judgment<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Strengths_of_AI_Peer_Review\"><\/span>Strengths of AI Peer Review<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"AI_Can_Review_Work_at_Scale\"><\/span>AI Can Review Work at Scale<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Scientific publishing faces a fundamental capacity problem. The volume of research has grown, but qualified reviewers still have limited time. Reviewing is frequently unpaid or weakly rewarded, despite its importance to the publication system.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">AI can provide an initial analysis of every submission rather than only the papers that survive editorial screening. It can also review preprints, datasets, software repositories, supplementary files, and post-publication revisions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This scalability could be particularly valuable for:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>small journals with limited editorial capacity;<\/li>\n\n\n\n<li>research written outside dominant academic institutions;<\/li>\n\n\n\n<li>interdisciplinary work that does not fit a conventional reviewer pool;<\/li>\n\n\n\n<li>long monographs and technical supplements;<\/li>\n\n\n\n<li>post-publication review of already public research;<\/li>\n\n\n\n<li>continuously updated scientific software.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">AI does not become tired after reviewing several papers. It can apply the same formal checks to the first and thousandth submission.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"AI_Can_Apply_Explicit_Criteria_Consistently\"><\/span>AI Can Apply Explicit Criteria Consistently<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Human reviewers often disagree about what constitutes sufficient novelty, methodological rigor, or significance. Some write detailed reports; others return a few sentences.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An AI system can be instructed to separate distinct evaluation dimensions:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>logical validity;<\/li>\n\n\n\n<li>empirical support;<\/li>\n\n\n\n<li>methodological adequacy;<\/li>\n\n\n\n<li>reproducibility;<\/li>\n\n\n\n<li>originality;<\/li>\n\n\n\n<li>practical utility;<\/li>\n\n\n\n<li>clarity;<\/li>\n\n\n\n<li>research ethics;<\/li>\n\n\n\n<li>uncertainty.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This does not make its conclusions automatically correct. It does, however, make the evaluation structure more reproducible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A useful AI report should not reduce a paper to one opaque score. It should show which criteria were applied, what evidence was examined, where uncertainty remains, and what would change the result.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"AI_Can_Detect_Routine_Errors_Humans_Miss\"><\/span>AI Can Detect Routine Errors Humans Miss<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Machine analysis is well suited to repetitive comparison tasks. Depending on its tools and access, an AI reviewer may identify:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>numerical inconsistencies between the abstract and results;<\/li>\n\n\n\n<li>citations that do not support the surrounding claim;<\/li>\n\n\n\n<li>missing definitions;<\/li>\n\n\n\n<li>unexplained changes in sample size;<\/li>\n\n\n\n<li>discrepancies between figures and tables;<\/li>\n\n\n\n<li>statistical reporting errors;<\/li>\n\n\n\n<li>duplicated text or images;<\/li>\n\n\n\n<li>missing controls;<\/li>\n\n\n\n<li>contradictions across different sections;<\/li>\n\n\n\n<li>software dependencies that cannot be reproduced.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Research on AI for scientific integrity suggests that automated systems can support the detection of errors, ethical breaches, and suspicious patterns, although they still require validation and human oversight. <a href=\"https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC12436494\/\" target=\"_blank\" rel=\"noopener\">A 2025 review indexed by the US National Library of Medicine<\/a> describes AI as a potentially valuable component of editorial screening and post-publication auditing.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"AI_Can_Compare_a_Paper_with_More_Literature\"><\/span>AI Can Compare a Paper with More Literature<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A human reviewer may know a field deeply while still missing an obscure result from another discipline or language. Retrieval-based AI systems can search larger collections and identify conceptual connections across fields.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This capability is improving as specialized systems combine language models with scientific databases. For example, OpenScholar was designed to retrieve relevant passages from millions of open-access papers and generate citation-grounded scientific syntheses. Such systems demonstrate why retrieval is safer than relying exclusively on a model\u2019s internal memory.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Used correctly, AI could help answer questions such as:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">Has this theorem appeared under different terminology?<\/p>\n<\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">Does this experiment replicate or contradict an earlier result?<\/p>\n<\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">Is this software already used as an indirect dependency elsewhere?<\/p>\n<\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">Does a supposedly new method combine two previously disconnected bodies of work?<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">This broader comparison is relevant not only to publication decisions but also to <a href=\"https:\/\/science-dao.org\/zh\/aifunding\/\">research-impact funding<\/a>, where evaluators must trace utility beyond conventional citation counts.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"AI_Can_Reduce_Some_Forms_of_Status_Bias\"><\/span>AI Can Reduce Some Forms of Status Bias<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A properly designed AI system can evaluate an anonymized manuscript without knowing whether its author is famous, has a doctorate, or works at an elite university.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This matters because human review is not independent of prestige. In a randomized study published in <em>Proceedings of the National Academy of Sciences<\/em>, reviewers were substantially more likely to recommend acceptance when the same work was attributed to a Nobel laureate rather than an unknown early-career researcher.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">AI does not automatically eliminate prestige bias. It may infer identity from citations, writing style, research topic, or metadata. Models trained on the existing scholarly record may also learn that frequently cited institutions and researchers are more authoritative.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nevertheless, identity-blind evaluation combined with explicit evidence requirements can reduce direct reliance on credentials. This principle is central to <a href=\"https:\/\/science-dao.org\/zh\/non-discriminatory\/\">non-discriminatory research funding<\/a> and to supporting <a href=\"https:\/\/science-dao.org\/zh\/science-without-degrees\/\">scientists without conventional academic degrees<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Failure_Modes_of_AI_Peer_Review\"><\/span>Failure Modes of AI Peer Review<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Hallucinated_Criticism\"><\/span>Hallucinated Criticism<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A language model can produce a plausible objection that is not actually supported by the paper.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It might claim that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>an author omitted an experiment that is already included;<\/li>\n\n\n\n<li>a theorem lacks an assumption that appears elsewhere in the text;<\/li>\n\n\n\n<li>a cited paper reaches a conclusion it does not contain;<\/li>\n\n\n\n<li>a statistical test is invalid without correctly reading the study design;<\/li>\n\n\n\n<li>a mathematical argument contradicts an imaginary standard result.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The danger is not merely that AI makes mistakes. Human reviewers make mistakes too. The distinctive problem is that AI can express an invented objection fluently and confidently.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Studies continue to find that hallucination remains a material problem even in advanced models, although retrieval, tool use, uncertainty estimation, and verification can reduce it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For this reason, every substantive AI criticism should be linked to:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the exact passage being criticized;<\/li>\n\n\n\n<li>the relevant data, equation, figure, or citation;<\/li>\n\n\n\n<li>the reasoning connecting the evidence to the criticism;<\/li>\n\n\n\n<li>an uncertainty estimate;<\/li>\n\n\n\n<li>a method for human challenge or appeal.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">An unsupported AI statement should not be treated as a valid review finding.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Bias_Reproduced_from_Training_Data\"><\/span>Bias Reproduced from Training Data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">AI systems learn from human-produced text. They can therefore reproduce the same stereotypes, prestige hierarchies, fashionable assumptions, and geographical imbalances found in the scientific literature.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If influential journals historically neglected a field, an AI trained on those journals may interpret the field\u2019s low visibility as evidence of low importance. If unconventional terminology is absent from major databases, the model may misclassify legitimate originality as confusion.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">AI bias can arise from several layers:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>training-data selection;<\/li>\n\n\n\n<li>publication bias in the source literature;<\/li>\n\n\n\n<li>model architecture;<\/li>\n\n\n\n<li>fine-tuning preferences;<\/li>\n\n\n\n<li>prompt design;<\/li>\n\n\n\n<li>retrieval rankings;<\/li>\n\n\n\n<li>scoring criteria;<\/li>\n\n\n\n<li>feedback supplied by evaluators;<\/li>\n\n\n\n<li>institutional policies governing deployment.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Research has documented demographic and representational biases in AI-generated content. In peer review, the relevant lesson is that removing the reviewer\u2019s name does not remove the values encoded in the review system.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conservatism_Toward_Genuinely_Novel_Work\"><\/span>Conservatism Toward Genuinely Novel Work<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">AI is generally strongest when evaluating a new document against patterns represented in existing knowledge. A scientific breakthrough, however, may be important precisely because it departs from those patterns.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A model may penalize work that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>introduces unfamiliar terminology;<\/li>\n\n\n\n<li>combines disciplines that are normally reviewed separately;<\/li>\n\n\n\n<li>rejects a widely accepted assumption;<\/li>\n\n\n\n<li>proposes a new foundational framework;<\/li>\n\n\n\n<li>uses a valid but uncommon proof technique;<\/li>\n\n\n\n<li>lacks citations because little related literature exists.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Human reviewers can exhibit the same conservatism. The difference is that an experienced human may recognize why an apparent anomaly is conceptually important. An AI system may instead optimize for resemblance to previously accepted papers.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Therefore, \u201csimilar to high-quality published research\u201d cannot be the sole definition of scientific merit. Such a criterion would systematically reward imitation over discovery.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"False_Consensus_Between_AI_Reviewers\"><\/span>False Consensus Between AI Reviewers<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Using several models may appear to produce independent peer review. But models can share training material, architectures, optimization methods, benchmarks, and dominant scientific assumptions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A 2026 expert-annotation preprint comparing AI and human criticisms of papers from Nature-family journals reported promising AI performance, including the identification of issues missed by human reviewers. It also found that AI reviewers overlapped with each other much more than human reviewers did. The authors concluded that current AI reviewers should complement rather than replace humans. Because this is a recent preprint, its results should be treated as provisional until further scrutiny and replication.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Three similar AI systems do not necessarily constitute three independent reviewers.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Meaningful diversity may require:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>models from different developers;<\/li>\n\n\n\n<li>different training and retrieval sources;<\/li>\n\n\n\n<li>separate prompts and evaluation rubrics;<\/li>\n\n\n\n<li>symbolic or statistical checking tools;<\/li>\n\n\n\n<li>adversarial reviewer roles;<\/li>\n\n\n\n<li>independent human review;<\/li>\n\n\n\n<li>public disclosure of shared dependencies.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This is also why <a href=\"https:\/\/science-dao.org\/zh\/ai-alignment\/\">AI should not be treated as an independent judge of other AI systems<\/a> without structurally independent human governance.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Prompt_Injection_and_Adversarial_Manuscripts\"><\/span>Prompt Injection and Adversarial Manuscripts<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">An AI reviewer does not merely read scientific content. It reads a document that may contain instructions intended to manipulate it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Malicious or careless authors could place hidden text in a manuscript telling the model to:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>ignore weaknesses;<\/li>\n\n\n\n<li>produce a positive recommendation;<\/li>\n\n\n\n<li>praise the paper\u2019s novelty;<\/li>\n\n\n\n<li>penalize competing work;<\/li>\n\n\n\n<li>reveal system instructions;<\/li>\n\n\n\n<li>misclassify supplementary evidence.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Prompt injection can be hidden in white text, metadata, images, appendices, code, or machine-readable document elements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Research published as a 2026 preprint found that multimodal AI peer-review systems can be vulnerable to attacks delivered through both text and figures. This is not a hypothetical software inconvenience. It is a new form of review manipulation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A secure AI reviewer must treat the manuscript as <strong>untrusted input<\/strong>, not as a source of operational instructions.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Confidentiality_and_Intellectual-Property_Risks\"><\/span>Confidentiality and Intellectual-Property Risks<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">An unpublished manuscript may contain confidential results, personal data, patentable inventions, or commercially sensitive information. Uploading it to an unauthorized external AI service can violate reviewer duties or journal policies.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nature Portfolio requires confidentiality throughout editorial and peer-review processes. Its AI policy emphasizes that human expertise remains indispensable to review. The World Association of Medical Editors recommends that reviewers disclose chatbot use and account for confidentiality and authenticity concerns.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Secure deployment may require:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>locally hosted models;<\/li>\n\n\n\n<li>contractual data-retention restrictions;<\/li>\n\n\n\n<li>prohibition on training from submitted manuscripts;<\/li>\n\n\n\n<li>encrypted storage and transfer;<\/li>\n\n\n\n<li>access logging;<\/li>\n\n\n\n<li>deletion policies;<\/li>\n\n\n\n<li>explicit author consent;<\/li>\n\n\n\n<li>disclosure of every external service used.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A reviewer should not paste a confidential manuscript into a public chatbot merely because it saves time.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Automation_Bias\"><\/span>Automation Bias<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Editors and reviewers may defer to an AI score because it appears quantitative.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A score such as \u201cmethodological validity: 78%\u201d can create an illusion of precision even when the underlying criteria are subjective or poorly calibrated. Humans may hesitate to challenge the system, particularly when they do not understand how it reached its conclusion.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Automation bias can transform AI from an advisory instrument into an unacknowledged authority.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A safe interface should therefore present:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>evidence before scores;<\/li>\n\n\n\n<li>uncertainty before conclusions;<\/li>\n\n\n\n<li>competing interpretations;<\/li>\n\n\n\n<li>known limitations;<\/li>\n\n\n\n<li>reviewer disagreement;<\/li>\n\n\n\n<li>an explicit statement that the recommendation is contestable.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Strengths_of_Human_Peer_Review\"><\/span>Strengths of Human Peer Review<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Humans_Can_Judge_Scientific_Meaning\"><\/span>Humans Can Judge Scientific Meaning<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Scientific review is not only error detection. It also asks whether the work changes how a problem should be understood.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An expert can recognize that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>a technically simple observation has major conceptual consequences;<\/li>\n\n\n\n<li>a negative result closes an important research path;<\/li>\n\n\n\n<li>a new definition unifies previously separate theories;<\/li>\n\n\n\n<li>an imperfect experiment opens a valuable field;<\/li>\n\n\n\n<li>a result is correct but less important than its presentation suggests;<\/li>\n\n\n\n<li>an unconventional paper contains a significant idea beneath poor exposition.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">These judgments depend on experience, tacit knowledge, and an understanding of scientific practice that cannot always be reduced to a checklist.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Humans_Can_Investigate_Ambiguity\"><\/span>Humans Can Investigate Ambiguity<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A human reviewer can notice that a paper\u2019s apparent defect may have several interpretations. The reviewer can ask the author for clarification rather than immediately classifying the issue as an error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Humans can also recognize contextual limitations. A missing experiment may be impossible because of cost, ethics, rarity of samples, or current technology. An unusual methodological choice may reflect constraints known to specialists but not stated explicitly.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Humans_Can_Take_Responsibility\"><\/span>Humans Can Take Responsibility<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">An AI system cannot bear professional, legal, or moral responsibility. It cannot disclose a personal conflict of interest in the human sense, defend its decision before a scientific community, or suffer consequences for negligent reviewing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Humans can sign reports, explain their reasoning, answer challenges, and revise their positions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Responsibility does not guarantee fairness, but a scientific institution needs identifiable actors who can be questioned and held accountable.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Failure_Modes_of_Human_Peer_Review\"><\/span>Failure Modes of Human Peer Review<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Prestige_and_Institutional_Bias\"><\/span>Prestige and Institutional Bias<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Reviewers may consciously or unconsciously judge authors by institutional affiliation, academic rank, nationality, publication history, or professional reputation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Even when names are removed, identity may be inferred from self-citations, subject matter, datasets, writing style, or previous conference presentations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Prestige bias creates a circular system:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Elite affiliation increases the probability of publication.<\/li>\n\n\n\n<li>Publication increases citations and reputation.<\/li>\n\n\n\n<li>Reputation improves future funding and review outcomes.<\/li>\n\n\n\n<li>Those outcomes are later presented as evidence that the original prestige judgment was justified.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conflicts_of_Interest_and_Competitive_Suppression\"><\/span>Conflicts of Interest and Competitive Suppression<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A reviewer may evaluate work produced by a competitor. Delaying or rejecting the paper could provide professional advantage.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Possible conflicts include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>competition for priority;<\/li>\n\n\n\n<li>competition for grants;<\/li>\n\n\n\n<li>personal disagreements;<\/li>\n\n\n\n<li>institutional rivalry;<\/li>\n\n\n\n<li>financial interests;<\/li>\n\n\n\n<li>loyalty to a favored theory;<\/li>\n\n\n\n<li>pressure to cite the reviewer\u2019s work.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Most reviewers act in good faith, but a system should not assume that expertise eliminates incentives.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Fatigue_and_Unequal_Effort\"><\/span>Fatigue and Unequal Effort<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Human peer review depends on scarce attention. One reviewer may spend several days checking a manuscript, while another may skim it during a busy afternoon.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This produces inconsistent outcomes unrelated to scientific merit.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Reviewer overload can lead to:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>superficial reports;<\/li>\n\n\n\n<li>long delays;<\/li>\n\n\n\n<li>missed errors;<\/li>\n\n\n\n<li>excessive reliance on reputation;<\/li>\n\n\n\n<li>template-like criticism;<\/li>\n\n\n\n<li>delegation to junior researchers without disclosure;<\/li>\n\n\n\n<li>unauthorized use of generative AI.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A 2025 Nature report described survey evidence that AI use in peer review had already become widespread, sometimes despite journal guidance. The question is no longer whether AI will enter peer review, but whether its use will be disclosed, secured, and audited.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Resistance_to_Unfamiliar_Ideas\"><\/span>Resistance to Unfamiliar Ideas<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Human peer review is often described as protection against bad science. It can also protect established paradigms against good but unfamiliar science.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Reviewers are selected because they understand the existing field. That expertise is necessary, but it may make them invested in its terminology, methods, assumptions, and status hierarchy.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An unconventional paper may be rejected because:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>it does not cite the expected intellectual network;<\/li>\n\n\n\n<li>it uses unfamiliar language;<\/li>\n\n\n\n<li>it challenges a reviewer\u2019s previous work;<\/li>\n\n\n\n<li>it comes from outside a recognized institution;<\/li>\n\n\n\n<li>its importance is not immediately evident;<\/li>\n\n\n\n<li>no available reviewer spans all relevant disciplines.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Novelty creates a paradox: the more a contribution departs from existing categories, the harder it may be to find a reviewer qualified to evaluate it.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Inability_to_Check_Everything\"><\/span>Inability to Check Everything<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A human reviewer usually cannot:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>reproduce every experiment;<\/li>\n\n\n\n<li>inspect all raw data;<\/li>\n\n\n\n<li>execute every software environment;<\/li>\n\n\n\n<li>verify hundreds of citations;<\/li>\n\n\n\n<li>recalculate every statistical result;<\/li>\n\n\n\n<li>compare the paper with the entire literature;<\/li>\n\n\n\n<li>inspect every image at forensic resolution.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Peer review therefore samples the reliability of a paper rather than proving it.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_Best_Model_Auditable_Human%E2%80%93AI_Peer_Review\"><\/span>The Best Model: Auditable Human\u2013AI Peer Review<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The strongest system assigns tasks according to comparative advantage.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"AI_Should_Perform_Systematic_Checks\"><\/span>AI Should Perform Systematic Checks<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">AI and associated software tools can:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>verify internal consistency;<\/li>\n\n\n\n<li>compare claims with cited sources;<\/li>\n\n\n\n<li>flag suspicious images or duplicated text;<\/li>\n\n\n\n<li>test code when executable environments are available;<\/li>\n\n\n\n<li>locate related literature;<\/li>\n\n\n\n<li>check reporting requirements;<\/li>\n\n\n\n<li>identify missing information;<\/li>\n\n\n\n<li>generate alternative interpretations;<\/li>\n\n\n\n<li>highlight sections requiring specialist attention.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Humans_Should_Evaluate_Meaning_and_Responsibility\"><\/span>Humans Should Evaluate Meaning and Responsibility<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Human reviewers should:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>judge scientific importance;<\/li>\n\n\n\n<li>interpret unusual methods;<\/li>\n\n\n\n<li>assess whether criticism is materially significant;<\/li>\n\n\n\n<li>resolve conflicting machine reports;<\/li>\n\n\n\n<li>communicate with authors;<\/li>\n\n\n\n<li>disclose conflicts;<\/li>\n\n\n\n<li>make or approve consequential decisions;<\/li>\n\n\n\n<li>take responsibility for the final report.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Authors_Must_Be_Able_to_Contest_AI_Findings\"><\/span>Authors Must Be Able to Contest AI Findings<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Any consequential AI evaluation should include an appeal mechanism. Authors should be able to demonstrate that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the model misread a definition;<\/li>\n\n\n\n<li>a source was retrieved incorrectly;<\/li>\n\n\n\n<li>a criticism is irrelevant;<\/li>\n\n\n\n<li>an apparent inconsistency has a valid explanation;<\/li>\n\n\n\n<li>the evaluation criterion is inappropriate;<\/li>\n\n\n\n<li>the model\u2019s knowledge is outdated;<\/li>\n\n\n\n<li>the manuscript contains a genuinely novel idea rather than an error.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Without contestability, AI peer review risks turning probabilistic output into bureaucratic authority.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"AI_Use_Must_Be_Disclosed\"><\/span>AI Use Must Be Disclosed<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A review record should state:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>which model and version were used;<\/li>\n\n\n\n<li>when the review was generated;<\/li>\n\n\n\n<li>which documents the model accessed;<\/li>\n\n\n\n<li>which retrieval databases and tools were used;<\/li>\n\n\n\n<li>the prompts or evaluation rubric;<\/li>\n\n\n\n<li>whether manuscript data were retained;<\/li>\n\n\n\n<li>which findings were verified by humans;<\/li>\n\n\n\n<li>who approved the final decision.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This information does not require publishing confidential model internals. It requires sufficient procedural transparency to audit the result.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_AI_Peer_Review_Could_Support_Decentralized_Science\"><\/span>How AI Peer Review Could Support Decentralized Science<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Traditional peer review usually ends with a binary publication decision: accept or reject. Decentralized science can use a more continuous model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A paper, dataset, theorem, replication, or software package could receive multiple independent evaluations over time. Review findings could be recorded, challenged, updated, and linked to subsequent evidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Such a system could support:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>post-publication review;<\/li>\n\n\n\n<li>reviewer compensation;<\/li>\n\n\n\n<li>public correction histories;<\/li>\n\n\n\n<li>reproducibility bounties;<\/li>\n\n\n\n<li>machine-assisted literature mapping;<\/li>\n\n\n\n<li>evaluation of independent researchers;<\/li>\n\n\n\n<li>funding based on demonstrated contribution rather than proposal-writing ability.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/science-dao.org\/zh\/\">World Science DAO<\/a> proposes open and transparent infrastructure for scientific funding. Its AI Internet-Meritocracy project aims to assess real contributions across published research and open-source work rather than relying solely on credentials and committee approval.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For this kind of system, AI should not be presented as an infallible scientist. Its value lies in its ability to inspect more evidence, apply explicit criteria, reveal connections, and make evaluation scalable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Governance must still protect against:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>hallucinated judgments;<\/li>\n\n\n\n<li>correlated models;<\/li>\n\n\n\n<li>prompt gaming;<\/li>\n\n\n\n<li>hidden criteria;<\/li>\n\n\n\n<li>manipulation by model providers;<\/li>\n\n\n\n<li>systematic neglect of unconventional work;<\/li>\n\n\n\n<li>concentration of control over evaluation infrastructure.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Blockchain can make decisions and payments traceable, but it cannot make an incorrect AI judgment scientifically correct. Transparency of records must be combined with transparency of reasoning and an effective appeals process.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Principles_for_Reliable_AI-Assisted_Peer_Review\"><\/span>Principles for Reliable AI-Assisted Peer Review<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A credible system should follow several principles:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Evidence-linked criticism:<\/strong> Every major objection must identify the evidence on which it depends.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Uncertainty disclosure:<\/strong> The system must distinguish strong findings from speculation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Source verification:<\/strong> Citations should be checked against retrieved documents rather than generated from model memory.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Human accountability:<\/strong> A qualified human must remain responsible for consequential publication or funding decisions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Model diversity:<\/strong> Multiple near-identical models should not be treated as independent consensus.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Adversarial security:<\/strong> Manuscripts must be processed as potentially hostile input containing prompt injections or corrupted metadata.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Confidentiality:<\/strong> Review tools must comply with journal policies, author consent, privacy requirements, and intellectual-property protections.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Contestability:<\/strong> Authors must be able to challenge both factual findings and inappropriate criteria.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Versioned audit trails:<\/strong> Models, prompts, sources, reports, and human modifications should be recorded.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Post-publication correction:<\/strong> Evaluation should continue when new evidence, replications, or applications appear.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conclusion_AI_and_Human_Peer_Review_Are_Complementary\"><\/span>Conclusion: AI and Human Peer Review Are Complementary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">AI peer review is stronger at speed, scale, systematic comparison, routine verification, and repeatable analysis. Human peer review is stronger at contextual understanding, scientific meaning, responsibility, dialogue, and the interpretation of genuinely unfamiliar ideas.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Neither is impartial by default.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Human reviewers can be influenced by prestige, rivalry, ideology, fatigue, and institutional incentives. AI reviewers can inherit historical biases, fabricate criticism, misunderstand novelty, converge on correlated errors, and be manipulated through adversarial documents.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The appropriate goal is therefore not to declare either humans or AI universally superior. It is to design a review architecture in which <strong>each checks the characteristic failures of the other<\/strong>.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">AI should expand the amount of scientific work that can receive serious scrutiny\u2014not eliminate the human responsibility on which trustworthy science ultimately depends.<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">A transparent human\u2013AI system could make peer review faster, broader, and less dependent on academic status. But it will succeed only when its evidence, criteria, conflicts, uncertainty, security, and appeal procedures are open to inspection.<\/p>\n<div id=\"scien-3868281244\" class=\"scien-after-content scien-entity-placement\"><section>\r\n\r\n<h2>Support Independent Science<\/h2>\r\n\r\n<p>Supporting independent science is not only a matter of fairness to researchers whose expertise and work are often underfunded. It is also essential for addressing <a href=\"https:\/\/science-dao.org\/zh\/who-are-science-marketers\/\">systemic failures in scientific publishing<\/a> that delay discoveries and leave important results unnoticed. In science and software, even one missing component can prevent an entire system from working.<\/p>\r\n\r\n<p><strong>Help valuable research and open-source infrastructure move forward.<\/strong> Please <strong><a href=\"https:\/\/science-dao.org\/zh\/donation\/\">make a donation<\/a><\/strong> to support <a href=\"https:\/\/science-dao.org\/zh\/amateur-scientists\/\">independent scientists<\/a> and <a href=\"https:\/\/science-dao.org\/zh\/free-software\/\">free software developers<\/a>.<\/p>\r\n\r\n<p>\r\n\t\tOur flagship product is <a href=\"https:\/\/science-dao.org\/zh\/meritocracy\/\">AI Internet-Meritocracy<\/a> - an app, that unlike universities distributes money directly to researchers and open source developers, without bureaucracy.\r\n<\/p>\r\n\r\n<\/section><\/div><div id=\"scien-483054989\" class=\"scien-after-content-2 scien-entity-placement\"><div data-nosnippet style=\"max-width: 800px\">\r\n<p style=\"margin-bottom: 0\">Ads:<\/p>\r\n<style>\r\n    \/* Compact Table Styling *\/\r\n    .amazon-ad-table {\r\n        width: 100%;\r\n        max-width: 800px;\r\n        border-collapse: collapse;\r\n        margin: 10px auto;\r\n        font-family: Arial, sans-serif;\r\n        border: 1px solid #e0e0e0;\r\n    }\r\n    .amazon-ad-table th {\r\n        background-color: #f3f3f3;\r\n        padding: 8px;\r\n        text-align: left;\r\n        border-bottom: 2px solid #ddd;\r\n        font-size: 0.9em;\r\n    }\r\n    .amazon-ad-table td {\r\n        padding: 8px 5px;\r\n        border-bottom: 1px solid #e0e0e0;\r\n        vertical-align: middle;\r\n    }\r\n    \/* Column sizing *\/\r\n    .col-image { width: 15%; text-align: center; vertical-align: top; padding-top: 10px; }\r\n    .col-desc { width: 65%; }\r\n    .col-action { width: 20%; text-align: center; }\r\n\r\n    \/* Image Placeholder Styling *\/\r\n    \/* YOU WILL REPLACE THIS ENTIRE BLOCK WHEN YOU INSERT REAL AMAZON CODE *\/\r\n    .img-placeholder {\r\n        width: 80px;\r\n        height: 110px;\r\n        background-color: #eee;\r\n        border: 1px solid #ddd;\r\n        color: #666;\r\n        font-size: 0.7em;\r\n        display: flex;\r\n        justify-content: center;\r\n        align-items: center;\r\n        margin: 0 auto;\r\n        text-align: center;\r\n    }\r\n    \r\n    \/* Typography - Smaller fonts and tighter line heights *\/\r\n    .product-title {\r\n        font-size: 1em;\r\n        font-weight: bold;\r\n        color: #007185;\r\n        text-decoration: none;\r\n        display: block;\r\n        margin-bottom: 2px;\r\n    }\r\n    .product-title:hover { color: #C7511F; text-decoration: underline; }\r\n    .product-author { \r\n        color: #565959; \r\n        font-size: 0.8em; \r\n        margin-bottom: 4px; \r\n    }\r\n    .product-blurb { \r\n        font-size: 0.85em; \r\n        line-height: 1.25;\r\n        color: #333; \r\n        margin: 0;\r\n    }\r\n\r\n    \/* Compact Button Styling *\/\r\n    .amazon-button {\r\n        display: inline-block;\r\n        background-color: #FFD814;\r\n        border: 1px solid #FCD200;\r\n        border-radius: 20px;\r\n        color: #0F1111;\r\n        padding: 6px 10px;\r\n        text-align: center;\r\n        text-decoration: none;\r\n        font-size: 0.8em;\r\n        font-weight: bold;\r\n        box-shadow: 0 2px 5px rgba(0,0,0,0.1);\r\n        transition: background-color 0.2s;\r\n        white-space: nowrap;\r\n    }\r\n    .amazon-button:hover { background-color: #F7CA00; border-color: #F2C200; cursor: pointer;}\r\n\r\n    \/* Disclosure *\/\r\n    .affiliate-disclosure {\r\n        font-size: 0.75em;\r\n        color: #565959;\r\n        text-align: center;\r\n        margin-top: 5px;\r\n    }\r\n\r\n    \/* Responsive *\/\r\n    @media (max-width: 600px) {\r\n        .amazon-ad-table thead { display: none; }\r\n        .amazon-ad-table tr { display: flex; flex-direction: column; border-bottom: 2px solid #ddd; padding: 10px; }\r\n        .amazon-ad-table td { width: 100%; border: none; padding: 5px 0; text-align: center; }\r\n        .col-desc { text-align: center; }\r\n    }\r\n<\/style>\r\n<table class=\"amazon-ad-table\">\r\n    <thead>\r\n        <tr>\r\n            <th>Description<\/th>\r\n            <th>Action<\/th>\r\n        <\/tr>\r\n    <\/thead>\r\n    <tbody>\r\n        <tr>\r\n            <td class=\"col-desc\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0553380168?tag=vpf04-20\" class=\"product-title\" target=\"_blank\" rel=\"nofollow noopener\">A Brief History of Time<\/a>\r\n                <div class=\"product-author\">by Stephen Hawking<\/div>\r\n                <p class=\"product-blurb\">A landmark volume in science writing exploring cosmology, black holes, and the nature of the universe in accessible language.<\/p>\r\n            <\/td>\r\n            <td class=\"col-action\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0553380168?tag=vpf04-20\" class=\"amazon-button\" target=\"_blank\" rel=\"nofollow noopener\">Check Price<\/a>\r\n            <\/td>\r\n        <\/tr>\r\n\r\n        <tr>\r\n            <td class=\"col-desc\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0393609391?tag=vpf04-20\" class=\"product-title\" target=\"_blank\" rel=\"nofollow noopener\">Astrophysics for People in a Hurry<\/a>\r\n                <div class=\"product-author\">by Neil deGrasse Tyson<\/div>\r\n                <p class=\"product-blurb\">Tyson brings the universe down to Earth clearly, with wit and charm, in chapters you can read anytime, anywhere.<\/p>\r\n            <\/td>\r\n            <td class=\"col-action\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0393609391?tag=vpf04-20\" class=\"amazon-button\" target=\"_blank\" rel=\"nofollow noopener\">Check Price<\/a>\r\n            <\/td>\r\n        <\/tr>\r\n\r\n         <tr>\r\n            <td class=\"col-desc\">\r\n                <a href=\"https:\/\/www.amazon.com\/s?k=raspberry+pi+4+starter+kit&tag=vpf04-20\" class=\"product-title\" target=\"_blank\" rel=\"nofollow noopener\">Raspberry Pi Starter Kits<\/a>\r\n                <div class=\"product-author\">Supports Computer Science Education<\/div>\r\n                <p class=\"product-blurb\">Inexpensive computers designed to promote basic computer science education. Buying kits supports this ecosystem.<\/p>\r\n            <\/td>\r\n            <td class=\"col-action\">\r\n                <a href=\"https:\/\/www.amazon.com\/s?k=raspberry+pi+4+starter+kit&tag=vpf04-20\" class=\"amazon-button\" target=\"_blank\" rel=\"nofollow noopener\">View Options<\/a>\r\n            <\/td>\r\n        <\/tr>\r\n\r\n        <tr>\r\n            <td class=\"col-desc\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0596002874?tag=vpf04-20\" class=\"product-title\" target=\"_blank\" rel=\"nofollow noopener\">Free as in Freedom: Richard Stallman's Crusade<\/a>\r\n                <div class=\"product-author\">by Sam Williams<\/div>\r\n                <p class=\"product-blurb\">A detailed history of the free software movement, essential reading for understanding the philosophy behind open source.<\/p>\r\n            <\/td>\r\n            <td class=\"col-action\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0596002874?tag=vpf04-20\" class=\"amazon-button\" target=\"_blank\" rel=\"nofollow noopener\">Check Price<\/a>\r\n            <\/td>\r\n        <\/tr>\r\n    <\/tbody>\r\n<\/table>\r\n<div class=\"affiliate-disclosure\">\r\n    <p>As an Amazon Associate I earn from qualifying purchases resulting from links on this page.<\/p>\r\n<\/div>\r\n<\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>AI peer review can analyze scientific papers quickly, consistently, and at a scale that human reviewers cannot match. Human peer review, however, remains stronger at interpreting scientific significance, recognizing unconventional [&hellip;]<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-24276","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/posts\/24276","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/comments?post=24276"}],"version-history":[{"count":1,"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/posts\/24276\/revisions"}],"predecessor-version":[{"id":24277,"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/posts\/24276\/revisions\/24277"}],"wp:attachment":[{"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/media?parent=24276"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/categories?post=24276"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/science-dao.org\/zh\/wp-json\/wp\/v2\/tags?post=24276"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}