Why Prediction-Based Funding Misses Unpredictable Discoveries

Getting your Trinity Audio player ready...

Prediction-based research funding asks scientists to describe discoveries before they make them. This creates a structural contradiction: the more genuinely surprising a discovery would be, the harder it is to predict, explain, and defend in a grant proposal.

Forecasting has a legitimate role in research management. Funders should assess whether a project is feasible, ethical, and scientifically coherent. But predicted impact should not be treated as a reliable measure of future scientific value. A funding system optimized for convincing forecasts will often select research that is easy to describe rather than research that changes what can be described.

The central problem is simple:

A proposal can predict the completion of planned work. It cannot reliably predict the discovery of something that neither the applicant nor the reviewers currently understand.

This is why prediction-based funding systematically risks overlooking unconventional researchers, foundational work, new mathematical frameworks, exploratory experiments, scientific software, and results whose applications emerge only years later.

What Is Prediction-Based Research Funding?

Prediction-based funding allocates money according to claims about future work. Applicants are normally expected to specify:

• the question they will answer;
• the methods they will use;
• the results they expect;
• the project’s likely impact;
• a timeline, budget, and set of deliverables.

These requirements are reasonable when the task is primarily engineering or implementation. A funder commissioning a telescope component, clinical database, or measurement campaign needs evidence that the work can be completed.

Fundamental research is different. Its purpose is often to discover which questions, concepts, or methods are appropriate. The most valuable result may therefore be something that was absent from the proposal.

The distinction is not between planned and unplanned work. Serious researchers usually have plans. The distinction is between planning an investigation and predicting its scientific significance.

Transformative Discoveries Are Difficult to Recognize in Advance

A transformative discovery changes an existing field’s assumptions, methods, or conceptual structure. Before that change occurs, reviewers must evaluate the proposal using the assumptions and methods already accepted by the field.

This produces a recognition problem. The proposal is judged through the framework that the proposed research may eventually replace.

The US National Science Foundation defines transformative research as work capable of radically changing understanding or creating a new paradigm. Its Early-Concept Grants for Exploratory Research mechanism explicitly supports untested and potentially transformative ideas that may be considered “high risk, high payoff.” The existence of such a mechanism is itself an acknowledgment that ordinary selection procedures can undersupply exploratory work. (NSF: Transformative Research)

The US National Institutes of Health has reached a similar conclusion. Its High-Risk, High-Reward Research program supports unusually innovative research and does not generally require preliminary data. NIH also notes that transformative research is inherently difficult and risky, yet necessary for accelerating discovery.

These programs are valuable, but they remain special exceptions inside systems primarily organized around advance project evaluation. Labeling a small portion of funding “high risk” does not eliminate the underlying preference for predictable proposals elsewhere.

Proposal Review Rewards Legibility

A grant application must make a project understandable to reviewers within limited time. This favors proposals that have:

• familiar terminology;
• established methods;
• recognizable research questions;
• conventional publication plans;
• preliminary evidence;
• applications that can already be named;
• institutional support and respected collaborators.

These properties may correlate with competent research. They are not equivalent to future importance.

A researcher developing an unfamiliar theory faces a disadvantage even when the theory is correct and valuable. Reviewers must first learn enough of the new framework to evaluate it. By contrast, a proposal extending a familiar method can be evaluated through established criteria.

This creates a legibility premium: research receives more favorable treatment when it fits the evaluative categories already available to the funding system.

The result is not necessarily deliberate conservatism. Reviewers may honestly attempt to identify the best projects. But when evidence is incomplete, understandable proposals feel safer than difficult ones. Consensus forms more easily around work that resembles work the field already recognizes.

Prediction Selects Persuasive Narratives, Not Necessarily Important Results

Proposal writing requires applicants to turn uncertain research into a coherent story:

1. This problem is important.
2. This method is promising.
3. These activities will produce these results.
4. Those results will generate this impact.

Actual research rarely follows such a clean sequence. Experiments fail. Definitions change. A minor observation becomes central. A planned application proves impossible, while an unanticipated application appears elsewhere.

Nevertheless, applicants are rewarded for presenting uncertainty as a credible narrative. This can select for the ability to formulate convincing predictions rather than the ability to produce valuable knowledge.

The problem becomes stronger when proposals are evaluated according to projected social or economic impact. Applicants then have incentives to attach speculative applications to early-stage research. This does not mean that they are dishonest. They are responding rationally to a system that demands a prediction where reliable prediction may be impossible.

Research has also documented the use of promotional or “hyped” language in scientific communication, illustrating the broader pressure to present work as more consequential and certain than the evidence alone establishes.

A system should not force scientists to become prophets or marketers merely to obtain permission to investigate an uncertain question.

Review Scores Have Limited Predictive Precision

Peer review can reject proposals with obvious methodological defects. It can identify scientific competence and distinguish very weak applications from plausible ones. But this does not mean that it can precisely rank the future value of several strong proposals.

Research on NIH grant review has found conflicting evidence about how well peer-review rankings predict later research productivity. A long-term analysis by NIH researchers noted that the predictive value of percentile rankings was limited and difficult to interpret.

The problem is especially serious near a funding threshold. Several proposals may all be credible, yet small differences in reviewer assignment, interpretation, or scoring determine which researchers receive years of support.

Recognizing this uncertainty, some funders have introduced lotteries among applications that pass a quality threshold. The Swiss National Science Foundation, for example, has used random selection where proposals are judged to be of comparable quality. Statistical work supporting this model emphasizes low inter-reviewer reliability, particularly near the funding line.

A lottery does not predict discoveries better. Its advantage is epistemic honesty: it acknowledges that reviewers may be unable to justify a precise ranking among sufficiently good proposals.

Unexpected Combinations Often Produce High-Impact Research

Important discoveries frequently arise by combining concepts, methods, or research contexts that were not previously connected.

A large-scale study published in Nature Communications examined “surprising” combinations of research contents and contexts. It found an association between scientific impact and combinations that departed from expected patterns. This does not prove that every unconventional combination deserves funding, but it supports the broader point that major advances can emerge from intellectual configurations that existing expectations do not readily identify.

Prediction-based selection may suppress precisely these combinations. Interdisciplinary proposals can be difficult to assign to a panel. Each reviewer may understand only part of the project. An idea may appear marginal in every established discipline while being important at their intersection.

The same issue applies to researchers changing direction. A 2025 study in Nature identified a “pivot penalty”: the measured impact of researchers’ new work declined as they moved farther from their previous subjects, and this penalty had intensified over time. The finding has several possible interpretations, but it demonstrates that scientific systems do not treat intellectual movement neutrally.

A funding model based heavily on past specialization and forecast credibility can make such pivots still harder.

Applications Cannot Always Be Predicted from Basic Research

The social value of basic research often becomes apparent only after other discoveries, technologies, or historical circumstances emerge.

At the time a mathematical definition is introduced, nobody may know whether it will later support computer verification, physics, cryptography, or a different branch of mathematics. A biological mechanism may appear narrow until a disease makes it clinically important. A software library created for one research community may later become infrastructure for many others.

Demanding a predetermined application can therefore reverse the actual order of discovery. In many cases:

1. researchers develop a concept or tool;
2. other people learn what it makes possible;
3. applications emerge through reuse and combination;
4. its wider significance becomes visible.

Prediction-based funding asks applicants to describe stage four before stage one has been funded.

This favors projects whose applications are already visible. It disadvantages foundational work that expands the space of future possibilities without specifying which possibility will ultimately matter.

Failed Predictions Can Still Produce Valuable Science

A proposal may make a prediction that turns out to be wrong while producing valuable evidence, methods, datasets, or negative results.

Under a rigid project-based model, such work can appear unsuccessful because it did not deliver the promised conclusion. Scientifically, however, disproving an attractive hypothesis may be more useful than confirming an expected one.

The difference is fundamental:

Funding accountability should ask whether valuable, rigorous work was produced—not whether reality followed the proposal’s narrative.

Researchers should remain accountable for using resources responsibly. But accountability should be based on observable scientific conduct and output: experiments performed, software released, proofs checked, datasets documented, errors corrected, and results communicated.

Treating adherence to the original prediction as the principal success criterion encourages researchers to avoid risky questions or to reinterpret ambiguous results as confirmation.

Prediction-Based Funding Encourages Incrementalism

A project is easiest to predict when it is close to completed work. Researchers can cite preliminary results, use established methods, estimate publication targets, and explain how the project extends an accepted literature.

This makes incremental research administratively attractive.

There is nothing inherently wrong with incremental work. Most scientific progress depends on careful extensions, replications, measurements, and maintenance. The problem occurs when funding criteria systematically confuse predictability with merit.

An analysis of millions of papers and patents found that they have become less likely to disrupt previous scientific and technological directions. The authors linked this decline partly to a narrowing in the range of prior knowledge being used. The study does not establish that grant systems alone caused the decline, but its findings are consistent with concern about an increasingly conservative research environment.

A portfolio containing only predictable work may produce reliable publications while missing discoveries that create new fields.

Can Better Forecasting Solve the Problem?

Artificial intelligence, prediction markets, expert aggregation, and bibliometric models may improve some funding forecasts. They can help identify inconsistencies, estimate completion risks, compare publication records, or reveal overlooked relationships.

But better forecasting cannot eliminate fundamental uncertainty.

A model trained on past successes will learn patterns associated with discoveries already recognized. It may identify future work that resembles those successes. A genuinely new discovery may not resemble them in the relevant dimensions.

There is also a reflexivity problem. Once researchers know what a model rewards, they can adapt proposals to its signals. The system may then optimize scientific descriptions for predicted fundability rather than optimize research for truth or usefulness.

AI can support evaluation, but it should not be treated as an oracle capable of knowing the future scientific value of work that has not yet been done.

Fund Researchers, Exploration, and Demonstrated Contributions

A more robust funding ecosystem should not rely on one selection mechanism. It can combine several approaches:

Stable support for capable researchers

Person-based funding gives researchers room to change direction when evidence demands it. It reduces the need to pretend that every important step was foreseen in a proposal.

This model still requires evaluation, but the evaluation can focus on demonstrated scientific judgment, previous output, openness, reliability, and continued activity rather than a single projected result.

Small exploratory allocations

Many uncertain ideas do not initially require a large grant. Small, rapidly distributed amounts can allow researchers to test concepts, produce prototypes, gather initial evidence, or expose an idea to criticism.

Because each decision carries less financial risk, a fund can support a wider variety of approaches.

Partial lotteries among qualified proposals

Where expert review can establish a threshold but cannot reliably rank applications above it, random selection may be fairer than artificial precision.

Lotteries are not suitable for every funding decision. Projects involving major safety, infrastructure, or feasibility constraints still require detailed scrutiny. But randomization can be rational where uncertainty dominates small score differences.

Retrospective and continuous rewards

Instead of paying only for promised work, funders can reward contributions after they become observable. Relevant outputs include:

• research articles and monographs;
• verified proofs;
• scientific software;
• datasets and protocols;
• replications and negative results;
• peer review and error detection;
• maintenance of research infrastructure;
• explanations that make neglected work usable by others.

Retrospective evaluation cannot finance every project before work begins. Researchers still need advance support. But combining baseline or exploratory funding with rewards for demonstrated contributions reduces dependence on speculative impact narratives.

How AI Internet-Meritocracy Changes the Timing of Evaluation

AI Internet-Meritocracy proposes evaluating scientists and free-software contributors continuously, using public evidence of actual work rather than relying exclusively on promises made in grant applications.

The important change is not simply the use of AI. It is the timing and object of evaluation.

Traditional project funding asks:

How valuable do reviewers expect this proposed project to become?

AIIM instead aims to ask:

What valuable, attributable work has this person already produced, and are they continuing to contribute?

This does not remove uncertainty or eliminate the need for safeguards. Evaluators may misunderstand obscure research. Public records may be incomplete. AI systems may inherit prestige bias, overvalue measurable outputs, or be manipulated through promotional content. Human voting and appeal mechanisms are therefore needed to contest errors and defend the system against abuse.

Nevertheless, evaluating demonstrated work has an important advantage: a discovery does not need to be predictable before it can be recognized.

Researchers can follow evidence, change direction, build unanticipated tools, or introduce concepts whose applications are initially unknown. When the value becomes visible, the contribution can be incorporated into subsequent funding decisions.

Learn more about the broader project at World Science DAO.

Prediction Should Inform Funding, Not Dominate It

Funding agencies cannot ignore the future. Every allocation implies some expectation that useful work will follow. Feasibility estimates, risk analysis, and scientific plans remain necessary.

The mistake is to assume that the future importance of research can be ranked with the same confidence as the feasibility of a defined task.

Prediction works best for bounded questions:

• Can this team perform the proposed experiment?
• Is the budget plausible?
• Are the methods technically appropriate?
• Are ethical and safety controls adequate?

It works much less reliably for open-ended questions:

• Will this idea create a new field?
• Which theory will matter in 20 years?
• What unexpected use will this tool acquire?
• Which obscure researcher is developing a foundational concept?
• Which failed experiment will reveal the decisive anomaly?

A well-designed funding system should distinguish these questions rather than compressing them into one proposal score.

Conclusion

Prediction-based funding misses unpredictable discoveries because it rewards researchers for describing future value before that value can be known. It favors legible projects, familiar methods, established institutions, incremental extensions, and persuasive impact narratives.

Some forecasting is unavoidable. But scientific funding should be diversified across stable researcher support, exploratory grants, qualified lotteries, infrastructure funding, and continuous rewards for demonstrated contributions.

The objective should not be to eliminate uncertainty from science. That would eliminate much of what makes discovery possible.

The better goal is to build a funding system that can support valuable research before its significance is obvious—and recognize it once the evidence appears.