{"id":24290,"date":"2026-07-17T05:15:44","date_gmt":"2026-07-17T05:15:44","guid":{"rendered":"https:\/\/science-dao.org\/?p=24290"},"modified":"2026-07-17T05:15:55","modified_gmt":"2026-07-17T05:15:55","slug":"certainty","status":"publish","type":"post","link":"https:\/\/science-dao.org\/ru\/certainty\/","title":{"rendered":"AI Confidence Is Not Scientific Certainty: Designing Safer Funding Decisions"},"content":{"rendered":"<div id=\"scien-1315349458\" class=\"scien-before-content scien-entity-placement\"><style>\r\n.amazon-support-link {\r\n    display: inline-flex;\r\n    align-items: baseline;\r\n    gap: 0.3rem;\r\n    padding: 0.35rem 0.55rem;\r\n    color: inherit;\r\n    font-size: 0.88rem;\r\n    line-height: 1.2;\r\n    text-decoration: none;\r\n    opacity: 0.72;\r\n    white-space: nowrap;\r\n    transition: opacity 0.2s ease;\r\n}\r\n\r\n.amazon-support-link:hover,\r\n.amazon-support-link:focus-visible {\r\n    opacity: 1;\r\n    text-decoration: underline;\r\n}\r\n\r\n.amazon-support-link small {\r\n    font-size: 0.65rem;\r\n    opacity: 0.7;\r\n}\r\n\r\n@media (max-width: 900px) {\r\n    .amazon-support-link small {\r\n        display: none;\r\n    }\r\n}\r\n<\/style>\r\n<a class=\"amazon-support-link\"\r\n   href=\"https:\/\/www.amazon.com\/?tag=vpf04-20\"\r\n   target=\"_blank\"\r\n   rel=\"nofollow sponsored noopener\"\r\n   aria-label=\"Shop on Amazon and support World Science DAO\">\r\n    Shop on Amazon <span aria-hidden=\"true\">\u2197<\/span>\r\n    <small>affiliate link<\/small>\r\n<\/a><\/div>\n<p class=\"wp-block-paragraph\"><strong>Meta description:<\/strong> <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Artificial intelligence can evaluate research proposals, compare scientific contributions, identify missing evidence, and estimate the probable impact of a project. However, a confident AI answer is not the same thing as a scientifically certain conclusion.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An AI system may assign a proposal a score of 92 out of 100, describe its reasoning fluently, and produce a precise funding recommendation. None of these features proves that the underlying assessment is correct.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"><strong>AI confidence measures a property of a model\u2019s output. Scientific certainty depends on evidence, reproducibility, logical validity, and continued scrutiny.<\/strong><\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">Safer AI-assisted funding systems must therefore treat confidence as one input into a decision\u2014not as permission to distribute money automatically. They should expose uncertainty, compare independent evaluations, preserve human and community oversight, and make funding reversible or incremental when evidence remains weak.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#What_Does_AI_Confidence_Actually_Mean\" >What Does AI Confidence Actually Mean?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Why_Scientific_Certainty_Is_a_Different_Concept\" >Why Scientific Certainty Is a Different Concept<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#The_Three_Layers_of_Uncertainty_in_AI_Funding\" >The Three Layers of Uncertainty in AI Funding<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#1_Uncertainty_in_the_Scientific_Claim\" >1. Uncertainty in the Scientific Claim<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#2_Uncertainty_in_the_Available_Information\" >2. Uncertainty in the Available Information<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#3_Uncertainty_in_the_AI_Evaluator\" >3. Uncertainty in the AI Evaluator<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Why_High_Confidence_Can_Still_Produce_a_Bad_Funding_Decision\" >Why High Confidence Can Still Produce a Bad Funding Decision<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#The_Proposal_Resembles_Successful_Training_Examples\" >The Proposal Resembles Successful Training Examples<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#The_Model_Confuses_Presentation_With_Substance\" >The Model Confuses Presentation With Substance<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Several_Models_May_Share_the_Same_Error\" >Several Models May Share the Same Error<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#The_Model_May_Be_Confident_Outside_Its_Competence\" >The Model May Be Confident Outside Its Competence<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Calibration_Testing_Whether_Confidence_Means_Anything\" >Calibration: Testing Whether Confidence Means Anything<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Outcomes_May_Take_Years_to_Observe\" >Outcomes May Take Years to Observe<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Success_Is_Multidimensional\" >Success Is Multidimensional<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Past_Funding_Decisions_Are_Biased_Training_Data\" >Past Funding Decisions Are Biased Training Data<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#A_Safer_Architecture_for_AI-Assisted_Funding\" >A Safer Architecture for AI-Assisted Funding<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#1_Produce_an_Uncertainty_Report_Not_Just_a_Score\" >1. Produce an Uncertainty Report, Not Just a Score<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#2_Separate_Scientific_Merit_From_Funding_Confidence\" >2. Separate Scientific Merit From Funding Confidence<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-19\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#3_Use_Staged_Funding\" >3. Use Staged Funding<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-20\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#4_Require_Independent_and_Adversarial_Evaluations\" >4. Require Independent and Adversarial Evaluations<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-21\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#5_Record_Model_Disagreement\" >5. Record Model Disagreement<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-22\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#6_Test_Decisions_Against_Counterfactual_Inputs\" >6. Test Decisions Against Counterfactual Inputs<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-23\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#7_Preserve_Traceability\" >7. Preserve Traceability<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-24\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#8_Make_Appeals_Evidence-Based\" >8. Make Appeals Evidence-Based<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-25\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#9_Combine_Prospective_and_Retroactive_Funding\" >9. Combine Prospective and Retroactive Funding<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-26\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#10_Keep_the_Funding_System_Contestable\" >10. Keep the Funding System Contestable<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-27\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#How_AIIM_Can_Use_Confidence_Safely\" >How AIIM Can Use Confidence Safely<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-28\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#A_Practical_Funding_Decision_Template\" >A Practical Funding Decision Template<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-29\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Project\" >Project<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-30\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Scientific_Value\" >Scientific Value<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-31\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Evidence\" >Evidence<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-32\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Model_Recommendation\" >Model Recommendation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-33\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Confidence_and_Calibration\" >Confidence and Calibration<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-34\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Known_Uncertainties\" >Known Uncertainties<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-35\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Adversarial_Findings\" >Adversarial Findings<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-36\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Bias_and_Sensitivity_Tests\" >Bias and Sensitivity Tests<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-37\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Funding_Structure\" >Funding Structure<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-38\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Governance_Decision\" >Governance Decision<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-39\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Follow-Up\" >Follow-Up<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-40\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#The_Role_of_Human_Judgment\" >The Role of Human Judgment<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-41\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Scientific_AI_Must_Remain_Open_to_Refutation\" >Scientific AI Must Remain Open to Refutation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-42\" href=\"https:\/\/science-dao.org\/ru\/certainty\/#Conclusion\" >Conclusion<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_Does_AI_Confidence_Actually_Mean\"><\/span>What Does AI Confidence Actually Mean?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The phrase <strong>AI confidence<\/strong> can refer to several different things:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the probability assigned by a statistical classifier;<\/li>\n\n\n\n<li>the relative strength of one answer compared with alternatives;<\/li>\n\n\n\n<li>the consistency of an answer across repeated model runs;<\/li>\n\n\n\n<li>the agreement of several AI evaluators;<\/li>\n\n\n\n<li>a model\u2019s own verbal statement that it is \u201chighly confident\u201d;<\/li>\n\n\n\n<li>a separately calculated estimate of prediction reliability.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">These measures are not interchangeable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A model saying, \u201cI am 95% confident,\u201d does not necessarily mean that similarly labelled answers are correct 95% of the time. Unless the system has been tested and calibrated on a representative set of real funding decisions, the number may have little operational meaning.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Language models present an additional problem: confident language is part of their communication style. A model can express a false conclusion clearly, consistently, and persuasively.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The <a href=\"https:\/\/www.nist.gov\/itl\/ai-risk-management-framework\" target=\"_blank\" rel=\"noopener\">NIST AI Risk Management Framework<\/a> consequently treats AI reliability as a risk-management problem involving governance, measurement, monitoring, documentation, and context\u2014not merely a question of obtaining a high model score.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Why_Scientific_Certainty_Is_a_Different_Concept\"><\/span>Why Scientific Certainty Is a Different Concept<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Scientific certainty is rarely absolute. Scientific conclusions normally exist on a spectrum ranging from speculation to strongly replicated knowledge.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A scientific claim becomes more credible through processes such as:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>theoretical consistency;<\/li>\n\n\n\n<li>transparent methodology;<\/li>\n\n\n\n<li>valid data collection;<\/li>\n\n\n\n<li>statistical or mathematical analysis;<\/li>\n\n\n\n<li>independent review;<\/li>\n\n\n\n<li>replication;<\/li>\n\n\n\n<li>successful prediction;<\/li>\n\n\n\n<li>survival under attempted refutation.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">An AI evaluation may inspect some of these properties, but it does not create them.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, an AI may judge that a proposed experiment has a sound design. That judgment does not prove that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the experiment will be performed correctly;<\/li>\n\n\n\n<li>the measurements will be reliable;<\/li>\n\n\n\n<li>the hypothesis is true;<\/li>\n\n\n\n<li>the result will replicate;<\/li>\n\n\n\n<li>the project will produce the expected social or scientific value.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This distinction is especially important in research funding because proposals concern <strong>future work<\/strong>. The strongest possible evaluation cannot directly observe results that do not yet exist.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_Three_Layers_of_Uncertainty_in_AI_Funding\"><\/span>The Three Layers of Uncertainty in AI Funding<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A safer funding system should distinguish at least three layers of uncertainty.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"1_Uncertainty_in_the_Scientific_Claim\"><\/span>1. Uncertainty in the Scientific Claim<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The proposed hypothesis may be wrong. The theory may contain a hidden contradiction, the preliminary evidence may be misleading, or the experiment may fail.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This uncertainty is intrinsic to research. Funding only projects whose success is already certain would largely eliminate genuine discovery.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"2_Uncertainty_in_the_Available_Information\"><\/span>2. Uncertainty in the Available Information<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The evaluator may not have access to all relevant information. Important data may be unpublished, poorly documented, inaccessible, or outside the model\u2019s training material.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A proposal may also omit essential details unintentionally\u2014or strategically.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"3_Uncertainty_in_the_AI_Evaluator\"><\/span>3. Uncertainty in the AI Evaluator<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The model itself may:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>misunderstand a technical argument;<\/li>\n\n\n\n<li>rely on obsolete information;<\/li>\n\n\n\n<li>overweight conventional terminology;<\/li>\n\n\n\n<li>penalize unfamiliar research areas;<\/li>\n\n\n\n<li>reproduce prestige or geographic biases;<\/li>\n\n\n\n<li>mistake polished writing for scientific value;<\/li>\n\n\n\n<li>fail to detect fabricated citations;<\/li>\n\n\n\n<li>generate an internally plausible but incorrect analysis.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">These layers should not be compressed into one apparently precise number.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A \u201c90% confidence\u201d result can obscure whether the uncertainty comes from experimental noise, incomplete evidence, disagreement among models, or the evaluator\u2019s lack of competence in the relevant field.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Why_High_Confidence_Can_Still_Produce_a_Bad_Funding_Decision\"><\/span>Why High Confidence Can Still Produce a Bad Funding Decision<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_Proposal_Resembles_Successful_Training_Examples\"><\/span>The Proposal Resembles Successful Training Examples<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Models often perform well on inputs similar to examples represented in their training or evaluation data. A conventional proposal from a well-established field may therefore receive a confident assessment.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A genuinely novel proposal may be harder to classify. Its unusual vocabulary, methodology, or conceptual structure can reduce model confidence even when the idea is important.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This creates a risk of <strong>algorithmic conservatism<\/strong>: predictable research receives high scores while unconventional research is treated as unreliable.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_Model_Confuses_Presentation_With_Substance\"><\/span>The Model Confuses Presentation With Substance<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Grant proposals are persuasive documents. They are designed to make uncertain future work appear coherent and fundable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An AI evaluator may reward:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>professional formatting;<\/li>\n\n\n\n<li>familiar research narratives;<\/li>\n\n\n\n<li>fashionable terminology;<\/li>\n\n\n\n<li>extensive citation lists;<\/li>\n\n\n\n<li>confident forecasts;<\/li>\n\n\n\n<li>institutionally conventional project plans.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">These features can correlate with quality, but they are not quality itself. An independent researcher with a strong result and weak presentation may be more valuable than a well-resourced team with an elegant but incremental proposal.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Several_Models_May_Share_the_Same_Error\"><\/span>Several Models May Share the Same Error<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Using multiple AI agents is useful, but agreement does not automatically establish independence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Models may share:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>similar training corpora;<\/li>\n\n\n\n<li>related architectures;<\/li>\n\n\n\n<li>common evaluation benchmarks;<\/li>\n\n\n\n<li>the same dominant scientific assumptions;<\/li>\n\n\n\n<li>identical missing information;<\/li>\n\n\n\n<li>correlated safety or instruction tuning.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Ten related models can therefore repeat one error ten times.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is why <a href=\"https:\/\/science-dao.org\/ru\/ai-alignment\/\">AI systems should not be treated as completely independent judges merely because they produce separate outputs<\/a>. Meaningful evaluation diversity requires different models, prompts, evidence sources, roles, and adversarial objectives.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_Model_May_Be_Confident_Outside_Its_Competence\"><\/span>The Model May Be Confident Outside Its Competence<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Scientific competence is domain-specific. A model that performs well on molecular biology abstracts may perform poorly on category theory, experimental archaeology, or an emerging interdisciplinary field.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A funding system should therefore ask not only:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">How confident is the model?<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">It should also ask:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">What evidence shows that this model is competent for this particular kind of decision?<\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Calibration_Testing_Whether_Confidence_Means_Anything\"><\/span>Calibration: Testing Whether Confidence Means Anything<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A model is <strong>calibrated<\/strong> when its confidence scores correspond reasonably well to observed outcomes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose an evaluator labels 100 decisions as having 80% confidence. If the relevant judgment is later found to be correct in approximately 80 cases, the score may be considered reasonably calibrated for that setting.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, scientific funding creates difficult calibration problems.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Outcomes_May_Take_Years_to_Observe\"><\/span>Outcomes May Take Years to Observe<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Research influence can emerge long after a grant ends. A theoretical result initially considered obscure may later become foundational.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Success_Is_Multidimensional\"><\/span>Success Is Multidimensional<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A project can fail at its original objective but still produce:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>useful negative results;<\/li>\n\n\n\n<li>open datasets;<\/li>\n\n\n\n<li>reusable software;<\/li>\n\n\n\n<li>improved methods;<\/li>\n\n\n\n<li>trained researchers;<\/li>\n\n\n\n<li>unexpected discoveries.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A binary \u201csuccessful or unsuccessful\u201d label is therefore inadequate.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Past_Funding_Decisions_Are_Biased_Training_Data\"><\/span>Past Funding Decisions Are Biased Training Data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Historical grant decisions reflect institutional preferences, prestige hierarchies, national priorities, and disciplinary fashions. Training an AI to reproduce past decisions may reproduce these biases rather than identify scientific merit.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Calibration must consequently be performed against carefully defined outcomes\u2014not simply against agreement with earlier funding panels.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"A_Safer_Architecture_for_AI-Assisted_Funding\"><\/span>A Safer Architecture for AI-Assisted Funding<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A robust system should separate evaluation, decision, payment, auditing, and appeal. No single confidence score should control the entire process.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"1_Produce_an_Uncertainty_Report_Not_Just_a_Score\"><\/span>1. Produce an Uncertainty Report, Not Just a Score<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Every recommendation should include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the proposed funding score;<\/li>\n\n\n\n<li>the main supporting evidence;<\/li>\n\n\n\n<li>the most important counterarguments;<\/li>\n\n\n\n<li>missing information;<\/li>\n\n\n\n<li>domain limitations;<\/li>\n\n\n\n<li>sensitivity to assumptions;<\/li>\n\n\n\n<li>disagreement between evaluators;<\/li>\n\n\n\n<li>conditions that would change the recommendation.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For example:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong> Fund a small replication grant.<br><strong>Confidence:<\/strong> Moderate.<br><strong>Main uncertainty:<\/strong> Preliminary results have not been independently reproduced.<br><strong>Failure risk:<\/strong> The reported effect may depend on an undocumented preprocessing choice.<br><strong>Next evidence required:<\/strong> Reproduction using preregistered analysis and external data.<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">This is more informative than \u201cScore: 87\/100.\u201d<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"2_Separate_Scientific_Merit_From_Funding_Confidence\"><\/span>2. Separate Scientific Merit From Funding Confidence<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A proposal may have high potential value but low confidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These are different dimensions:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Scientific potential<\/th><th>Confidence in evaluation<\/th><th>Appropriate response<\/th><\/tr><\/thead><tbody><tr><td>High<\/td><td>High<\/td><td>Substantial funding with normal monitoring<\/td><\/tr><tr><td>High<\/td><td>Low<\/td><td>Small exploratory grant or independent review<\/td><\/tr><tr><td>Low<\/td><td>High<\/td><td>Reject or deprioritize with documented reasons<\/td><\/tr><tr><td>Low<\/td><td>Low<\/td><td>Request evidence or defer the decision<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This distinction protects high-risk, high-reward science. Low confidence should not automatically mean rejection.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"3_Use_Staged_Funding\"><\/span>3. Use Staged Funding<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Instead of treating funding as a one-time irreversible decision, allocate it in stages:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Verification grant:<\/strong> Confirm identity, evidence, code, data, or preliminary claims.<\/li>\n\n\n\n<li><strong>Pilot grant:<\/strong> Test whether the method works on a small scale.<\/li>\n\n\n\n<li><strong>Expansion grant:<\/strong> Increase funding after defined milestones.<\/li>\n\n\n\n<li><strong>Retroactive reward:<\/strong> Pay for demonstrated outputs and public value.<\/li>\n\n\n\n<li><strong>Long-term support:<\/strong> Fund maintenance, replication, and continued development.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Staged funding converts some uncertainty into observable evidence before the largest financial commitment is made.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It also reduces the consequences of a mistaken AI judgment.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"4_Require_Independent_and_Adversarial_Evaluations\"><\/span>4. Require Independent and Adversarial Evaluations<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">At least one evaluator should be assigned to challenge the recommendation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The adversarial evaluator should search for:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>unsupported assumptions;<\/li>\n\n\n\n<li>fabricated or irrelevant citations;<\/li>\n\n\n\n<li>methodological weaknesses;<\/li>\n\n\n\n<li>conflicts of interest;<\/li>\n\n\n\n<li>signs of proposal gaming;<\/li>\n\n\n\n<li>alternative explanations;<\/li>\n\n\n\n<li>reasons the project may be undervalued;<\/li>\n\n\n\n<li>reasons the dominant consensus may be wrong.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This is not the same as asking the original evaluator to \u201cdouble-check.\u201d A separate role, context, and ideally model should be used.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">World Science DAO has proposed applying this principle through the <a href=\"https:\/\/science-dao.org\/ru\/adversarial\/\">adversarial testing of AI-assisted funding systems<\/a>. Red-team processes can reveal vulnerabilities before those vulnerabilities control large financial allocations.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"5_Record_Model_Disagreement\"><\/span>5. Record Model Disagreement<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Disagreement should not be silently averaged away.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose four evaluators produce the following recommendations:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Model A: 91;<\/li>\n\n\n\n<li>Model B: 88;<\/li>\n\n\n\n<li>Model C: 43;<\/li>\n\n\n\n<li>Model D: insufficient evidence.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The average score is 74, but \u201c74\u201d hides the most important information: one evaluator found a major problem and another considered the evidence inadequate.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The system should identify the source of disagreement:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Did one model detect a false citation?<\/li>\n\n\n\n<li>Did the models interpret the objective differently?<\/li>\n\n\n\n<li>Does one evaluator have more relevant technical competence?<\/li>\n\n\n\n<li>Is the proposal genuinely controversial?<\/li>\n\n\n\n<li>Are the evaluation criteria underspecified?<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Disagreement is evidence about uncertainty.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"6_Test_Decisions_Against_Counterfactual_Inputs\"><\/span>6. Test Decisions Against Counterfactual Inputs<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A funding system should be checked for irrelevant sensitivity.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Evaluators can receive controlled versions of the same proposal with changes to:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>author name;<\/li>\n\n\n\n<li>institution;<\/li>\n\n\n\n<li>country;<\/li>\n\n\n\n<li>academic title;<\/li>\n\n\n\n<li>writing style;<\/li>\n\n\n\n<li>project popularity;<\/li>\n\n\n\n<li>citation count;<\/li>\n\n\n\n<li>demographic signals;<\/li>\n\n\n\n<li>order of information.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Large score changes caused by irrelevant alterations indicate bias or instability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, a mathematical proof should not become more logically valid merely because the author is associated with a famous university.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"7_Preserve_Traceability\"><\/span>7. Preserve Traceability<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A funding decision should record:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the submitted materials;<\/li>\n\n\n\n<li>model versions;<\/li>\n\n\n\n<li>evaluation prompts;<\/li>\n\n\n\n<li>external tools used;<\/li>\n\n\n\n<li>retrieved sources;<\/li>\n\n\n\n<li>individual evaluator outputs;<\/li>\n\n\n\n<li>confidence and uncertainty reports;<\/li>\n\n\n\n<li>voting results;<\/li>\n\n\n\n<li>payment conditions;<\/li>\n\n\n\n<li>subsequent corrections.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Traceability enables audits, appeals, and learning from mistakes. It also makes it harder to conceal arbitrary intervention.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The European Union\u2019s AI governance framework similarly emphasizes documentation, logging, transparency, robustness, risk management, and appropriate human oversight for systems used in consequential contexts.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Scientific funding may not always fall within a particular legal classification, but the underlying governance principles remain relevant.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"8_Make_Appeals_Evidence-Based\"><\/span>8. Make Appeals Evidence-Based<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Applicants should be able to challenge:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>factual errors;<\/li>\n\n\n\n<li>incorrect citations;<\/li>\n\n\n\n<li>misunderstood methods;<\/li>\n\n\n\n<li>missing evidence;<\/li>\n\n\n\n<li>conflicts between evaluator outputs;<\/li>\n\n\n\n<li>apparent bias;<\/li>\n\n\n\n<li>misuse of evaluation criteria.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">An appeal should not merely rerun the same prompt through the same model. It should introduce new evidence or an independent evaluation path.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Successful appeals should also improve the system. Repeated error patterns can become adversarial test cases for future model versions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"9_Combine_Prospective_and_Retroactive_Funding\"><\/span>9. Combine Prospective and Retroactive Funding<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Prospective grants fund work before results exist. They are necessarily uncertain.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Retroactive funding rewards work after outputs can be examined. It can consider:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>publications;<\/li>\n\n\n\n<li>proofs;<\/li>\n\n\n\n<li>datasets;<\/li>\n\n\n\n<li>source code;<\/li>\n\n\n\n<li>replication;<\/li>\n\n\n\n<li>dependencies;<\/li>\n\n\n\n<li>downstream scientific use;<\/li>\n\n\n\n<li>documented social benefit.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Neither method is sufficient alone.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Purely prospective funding may reward persuasive promises. Purely retroactive funding can exclude researchers who cannot afford to work without prior support.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A safer system combines:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>small prospective grants for access and experimentation;<\/li>\n\n\n\n<li>milestone payments for verified progress;<\/li>\n\n\n\n<li>retroactive rewards for demonstrated impact.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This hybrid model reduces reliance on speculative confidence scores.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"10_Keep_the_Funding_System_Contestable\"><\/span>10. Keep the Funding System Contestable<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A funding algorithm should not become an unquestionable scientific authority.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Researchers, donors, reviewers, and governance participants must be able to inspect its criteria and challenge its conclusions. The system should clearly distinguish:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>model recommendation;<\/li>\n\n\n\n<li>governance rule;<\/li>\n\n\n\n<li>factual verification;<\/li>\n\n\n\n<li>scientific judgment;<\/li>\n\n\n\n<li>final payment authorization.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">AI can organize information and scale evaluation. It should not make its own uncertainty disappear through institutional authority.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_AIIM_Can_Use_Confidence_Safely\"><\/span>How AIIM Can Use Confidence Safely<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/merit.science-dao.org\/\">AI Internet-Meritocracy<\/a> proposes using AI to evaluate scientific and open-source contributions and help allocate funding according to merit.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For such a system, the correct objective is not to construct an infallible artificial grant officer. No evaluator\u2014human or artificial\u2014is infallible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Therefore AI output is to be confirmed by <a href=\"https:\/\/science-dao.org\/ru\/voting-in-ai-internet-meritocracy-aiim\/\" data-type=\"post\" data-id=\"23661\">human voting<\/a>, who take the final decision on banning\/unbanning a user.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The objective should be to build a funding process that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>detects and reports uncertainty;<\/li>\n\n\n\n<li>compares multiple perspectives;<\/li>\n\n\n\n<li>rewards verifiable contributions;<\/li>\n\n\n\n<li>allows adversarial challenges;<\/li>\n\n\n\n<li>records decisions transparently;<\/li>\n\n\n\n<li>limits the damage caused by individual errors;<\/li>\n\n\n\n<li>improves as outcomes become observable.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">AIIM can therefore treat confidence as part of a broader <strong>decision-risk model<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A high AI score might justify a larger payment only when several conditions are also satisfied:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the evidence is accessible;<\/li>\n\n\n\n<li>the evaluators are sufficiently independent;<\/li>\n\n\n\n<li>no unresolved critical objection remains;<\/li>\n\n\n\n<li>the model has demonstrated competence in the domain;<\/li>\n\n\n\n<li>the decision passes applicable governance rules;<\/li>\n\n\n\n<li>an audit trail is preserved;<\/li>\n\n\n\n<li>the payment remains proportionate to uncertainty.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For uncertain but potentially transformative work, the appropriate response may be a small exploratory payment rather than rejection.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"A_Practical_Funding_Decision_Template\"><\/span>A Practical Funding Decision Template<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">An AI-assisted funding decision could use the following structure:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Project\"><\/span>Project<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Name, authors, discipline, requested amount, and proposed outputs.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Scientific_Value\"><\/span>Scientific Value<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">What problem does the project address, and why might solving it matter?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Evidence\"><\/span>Evidence<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">What claims, results, code, data, publications, or prior work can be verified?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Model_Recommendation\"><\/span>Model Recommendation<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">What does each evaluator recommend?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Confidence_and_Calibration\"><\/span>Confidence and Calibration<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">How reliable has each evaluator been on comparable tasks?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Known_Uncertainties\"><\/span>Known Uncertainties<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">What information is missing, disputed, novel, or difficult to evaluate?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Adversarial_Findings\"><\/span>Adversarial Findings<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">What are the strongest reasons not to fund the project?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Bias_and_Sensitivity_Tests\"><\/span>Bias and Sensitivity Tests<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Would irrelevant changes to identity, affiliation, or presentation alter the recommendation?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Funding_Structure\"><\/span>Funding Structure<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Should the project receive verification funding, pilot funding, milestone funding, or a retroactive reward?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Governance_Decision\"><\/span>Governance Decision<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Who approved the allocation, under which rules, and with what appeal mechanism?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Follow-Up\"><\/span>Follow-Up<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">What observable evidence will be used to reassess the decision?<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_Role_of_Human_Judgment\"><\/span>The Role of Human Judgment<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Human oversight is necessary, but \u201ckeep a human in the loop\u201d is not a complete solution.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Human reviewers can also be:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>biased;<\/li>\n\n\n\n<li>overconfident;<\/li>\n\n\n\n<li>inattentive;<\/li>\n\n\n\n<li>politically influenced;<\/li>\n\n\n\n<li>impressed by prestige;<\/li>\n\n\n\n<li>hostile to unfamiliar ideas;<\/li>\n\n\n\n<li>unable to evaluate every specialist field.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The purpose of human participation is not to certify that the AI is correct. It is to introduce accountability, contextual judgment, and an additional path for detecting error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Likewise, AI should not merely automate human prejudices. It should be used to expose inconsistencies, compare evidence at scale, and challenge decisions that depend too strongly on institutional reputation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The safest design is therefore not <strong>AI versus humans<\/strong>, but a structured system in which different evaluators can correct one another.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Scientific_AI_Must_Remain_Open_to_Refutation\"><\/span>Scientific AI Must Remain Open to Refutation<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Recent work on AI systems for scientific discovery emphasizes both their potential and the need for rigorous verification, peer review, and reproducibility safeguards. AI-generated scientific reasoning can accelerate hypothesis generation, but uncritical use may also multiply low-quality or irreproducible outputs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This principle applies equally to funding.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A scientific funding AI should be designed as though every recommendation may later be proven wrong. That assumption leads naturally to:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>limited initial exposure;<\/li>\n\n\n\n<li>transparent reasoning;<\/li>\n\n\n\n<li>independent verification;<\/li>\n\n\n\n<li>adversarial evaluation;<\/li>\n\n\n\n<li>reversible procedures;<\/li>\n\n\n\n<li>continuous monitoring;<\/li>\n\n\n\n<li>public correction mechanisms.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">In other words, scientific funding should follow the logic of science itself: claims are provisional, evidence is contestable, and confidence must remain open to revision.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">AI can make research funding faster, broader, more consistent, and potentially less dependent on institutional prestige. But a precise score or confident explanation does not convert an uncertain scientific judgment into a fact.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"><strong>The goal of safe AI funding is not to eliminate uncertainty. It is to represent uncertainty honestly and prevent uncertain judgments from exercising unlimited financial power.<\/strong><\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">A trustworthy funding architecture should combine calibrated evaluation, explicit uncertainty reports, multiple independent agents, adversarial testing, staged payments, transparent records, appeals, and retroactive assessment.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">AI confidence can help determine which evidence deserves attention. It should never be mistaken for scientific certainty.<\/p>\n<div id=\"scien-973903583\" class=\"scien-after-content scien-entity-placement\"><section>\r\n\r\n<h2>Support Independent Science<\/h2>\r\n\r\n<p>Supporting independent science is not only a matter of fairness to researchers whose expertise and work are often underfunded. It is also essential for addressing <a href=\"https:\/\/science-dao.org\/ru\/who-are-science-marketers\/\">systemic failures in scientific publishing<\/a> that delay discoveries and leave important results unnoticed. In science and software, even one missing component can prevent an entire system from working.<\/p>\r\n\r\n<p><strong>Help valuable research and open-source infrastructure move forward.<\/strong> Please <strong><a href=\"https:\/\/science-dao.org\/ru\/donation\/\">make a donation<\/a><\/strong> to support <a href=\"https:\/\/science-dao.org\/ru\/amateur-scientists\/\">independent scientists<\/a> and <a href=\"https:\/\/science-dao.org\/ru\/free-software\/\">free software developers<\/a>.<\/p>\r\n\r\n<p>\r\n\t\tOur flagship product is <a href=\"https:\/\/science-dao.org\/ru\/meritocracy\/\">AI Internet-Meritocracy<\/a> - an app, that unlike universities distributes money directly to researchers and open source developers, without bureaucracy.\r\n<\/p>\r\n\r\n<\/section><\/div><div id=\"scien-1782179596\" class=\"scien-after-content-2 scien-entity-placement\"><div data-nosnippet style=\"max-width: 800px\">\r\n<p style=\"margin-bottom: 0\">Ads:<\/p>\r\n<style>\r\n    \/* Compact Table Styling *\/\r\n    .amazon-ad-table {\r\n        width: 100%;\r\n        max-width: 800px;\r\n        border-collapse: collapse;\r\n        margin: 10px auto;\r\n        font-family: Arial, sans-serif;\r\n        border: 1px solid #e0e0e0;\r\n    }\r\n    .amazon-ad-table th {\r\n        background-color: #f3f3f3;\r\n        padding: 8px;\r\n        text-align: left;\r\n        border-bottom: 2px solid #ddd;\r\n        font-size: 0.9em;\r\n    }\r\n    .amazon-ad-table td {\r\n        padding: 8px 5px;\r\n        border-bottom: 1px solid #e0e0e0;\r\n        vertical-align: middle;\r\n    }\r\n    \/* Column sizing *\/\r\n    .col-image { width: 15%; text-align: center; vertical-align: top; padding-top: 10px; }\r\n    .col-desc { width: 65%; }\r\n    .col-action { width: 20%; text-align: center; }\r\n\r\n    \/* Image Placeholder Styling *\/\r\n    \/* YOU WILL REPLACE THIS ENTIRE BLOCK WHEN YOU INSERT REAL AMAZON CODE *\/\r\n    .img-placeholder {\r\n        width: 80px;\r\n        height: 110px;\r\n        background-color: #eee;\r\n        border: 1px solid #ddd;\r\n        color: #666;\r\n        font-size: 0.7em;\r\n        display: flex;\r\n        justify-content: center;\r\n        align-items: center;\r\n        margin: 0 auto;\r\n        text-align: center;\r\n    }\r\n    \r\n    \/* Typography - Smaller fonts and tighter line heights *\/\r\n    .product-title {\r\n        font-size: 1em;\r\n        font-weight: bold;\r\n        color: #007185;\r\n        text-decoration: none;\r\n        display: block;\r\n        margin-bottom: 2px;\r\n    }\r\n    .product-title:hover { color: #C7511F; text-decoration: underline; }\r\n    .product-author { \r\n        color: #565959; \r\n        font-size: 0.8em; \r\n        margin-bottom: 4px; \r\n    }\r\n    .product-blurb { \r\n        font-size: 0.85em; \r\n        line-height: 1.25;\r\n        color: #333; \r\n        margin: 0;\r\n    }\r\n\r\n    \/* Compact Button Styling *\/\r\n    .amazon-button {\r\n        display: inline-block;\r\n        background-color: #FFD814;\r\n        border: 1px solid #FCD200;\r\n        border-radius: 20px;\r\n        color: #0F1111;\r\n        padding: 6px 10px;\r\n        text-align: center;\r\n        text-decoration: none;\r\n        font-size: 0.8em;\r\n        font-weight: bold;\r\n        box-shadow: 0 2px 5px rgba(0,0,0,0.1);\r\n        transition: background-color 0.2s;\r\n        white-space: nowrap;\r\n    }\r\n    .amazon-button:hover { background-color: #F7CA00; border-color: #F2C200; cursor: pointer;}\r\n\r\n    \/* Disclosure *\/\r\n    .affiliate-disclosure {\r\n        font-size: 0.75em;\r\n        color: #565959;\r\n        text-align: center;\r\n        margin-top: 5px;\r\n    }\r\n\r\n    \/* Responsive *\/\r\n    @media (max-width: 600px) {\r\n        .amazon-ad-table thead { display: none; }\r\n        .amazon-ad-table tr { display: flex; flex-direction: column; border-bottom: 2px solid #ddd; padding: 10px; }\r\n        .amazon-ad-table td { width: 100%; border: none; padding: 5px 0; text-align: center; }\r\n        .col-desc { text-align: center; }\r\n    }\r\n<\/style>\r\n<table class=\"amazon-ad-table\">\r\n    <thead>\r\n        <tr>\r\n            <th>Description<\/th>\r\n            <th>Action<\/th>\r\n        <\/tr>\r\n    <\/thead>\r\n    <tbody>\r\n        <tr>\r\n            <td class=\"col-desc\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0553380168?tag=vpf04-20\" class=\"product-title\" target=\"_blank\" rel=\"nofollow noopener\">A Brief History of Time<\/a>\r\n                <div class=\"product-author\">by Stephen Hawking<\/div>\r\n                <p class=\"product-blurb\">A landmark volume in science writing exploring cosmology, black holes, and the nature of the universe in accessible language.<\/p>\r\n            <\/td>\r\n            <td class=\"col-action\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0553380168?tag=vpf04-20\" class=\"amazon-button\" target=\"_blank\" rel=\"nofollow noopener\">Check Price<\/a>\r\n            <\/td>\r\n        <\/tr>\r\n\r\n        <tr>\r\n            <td class=\"col-desc\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0393609391?tag=vpf04-20\" class=\"product-title\" target=\"_blank\" rel=\"nofollow noopener\">Astrophysics for People in a Hurry<\/a>\r\n                <div class=\"product-author\">by Neil deGrasse Tyson<\/div>\r\n                <p class=\"product-blurb\">Tyson brings the universe down to Earth clearly, with wit and charm, in chapters you can read anytime, anywhere.<\/p>\r\n            <\/td>\r\n            <td class=\"col-action\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0393609391?tag=vpf04-20\" class=\"amazon-button\" target=\"_blank\" rel=\"nofollow noopener\">Check Price<\/a>\r\n            <\/td>\r\n        <\/tr>\r\n\r\n         <tr>\r\n            <td class=\"col-desc\">\r\n                <a href=\"https:\/\/www.amazon.com\/s?k=raspberry+pi+4+starter+kit&tag=vpf04-20\" class=\"product-title\" target=\"_blank\" rel=\"nofollow noopener\">Raspberry Pi Starter Kits<\/a>\r\n                <div class=\"product-author\">Supports Computer Science Education<\/div>\r\n                <p class=\"product-blurb\">Inexpensive computers designed to promote basic computer science education. Buying kits supports this ecosystem.<\/p>\r\n            <\/td>\r\n            <td class=\"col-action\">\r\n                <a href=\"https:\/\/www.amazon.com\/s?k=raspberry+pi+4+starter+kit&tag=vpf04-20\" class=\"amazon-button\" target=\"_blank\" rel=\"nofollow noopener\">View Options<\/a>\r\n            <\/td>\r\n        <\/tr>\r\n\r\n        <tr>\r\n            <td class=\"col-desc\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0596002874?tag=vpf04-20\" class=\"product-title\" target=\"_blank\" rel=\"nofollow noopener\">Free as in Freedom: Richard Stallman's Crusade<\/a>\r\n                <div class=\"product-author\">by Sam Williams<\/div>\r\n                <p class=\"product-blurb\">A detailed history of the free software movement, essential reading for understanding the philosophy behind open source.<\/p>\r\n            <\/td>\r\n            <td class=\"col-action\">\r\n                <a href=\"https:\/\/www.amazon.com\/dp\/0596002874?tag=vpf04-20\" class=\"amazon-button\" target=\"_blank\" rel=\"nofollow noopener\">Check Price<\/a>\r\n            <\/td>\r\n        <\/tr>\r\n    <\/tbody>\r\n<\/table>\r\n<div class=\"affiliate-disclosure\">\r\n    <p>As an Amazon Associate I earn from qualifying purchases resulting from links on this page.<\/p>\r\n<\/div>\r\n<\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>Meta description: Artificial intelligence can evaluate research proposals, compare scientific contributions, identify missing evidence, and estimate the probable impact of a project. However, a confident AI answer is not the [&hellip;]<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-24290","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/posts\/24290","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/comments?post=24290"}],"version-history":[{"count":1,"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/posts\/24290\/revisions"}],"predecessor-version":[{"id":24291,"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/posts\/24290\/revisions\/24291"}],"wp:attachment":[{"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/media?parent=24290"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/categories?post=24290"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/science-dao.org\/ru\/wp-json\/wp\/v2\/tags?post=24290"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}