Why Citation Counts Cannot Measure the Value of Basic Mathematics

Getting your Trinity Audio player ready...

Citation counts can show that a mathematical paper has been noticed and used by other researchers. They cannot reliably show whether the work is true, profound, foundational, difficult, original, or likely to transform science decades later.

This distinction matters especially in basic mathematics, where valuable results may initially have few readers, belong to a small research community, require years of additional development, or become useful only after an unexpected connection is discovered.

A citation is evidence of scholarly attention. It is not a direct measurement of mathematical value.

What Do Citation Counts Actually Measure?

A citation count records how many indexed publications refer to a particular work. In favorable circumstances, citations provide a rough indication that a paper has influenced subsequent academic writing.

However, citation counts are shaped by many factors unrelated to scientific importance:

• the size of the research field;
• the number of active authors in that field;
• publication and citation customs;
• database coverage;
• the age and accessibility of the paper;
• the reputation and institutional network of the author;
• whether the result is easy to apply;
• whether the topic is currently fashionable;
• whether the paper is cited positively or critically.

The San Francisco Declaration on Research Assessment argues that research outputs should be assessed on their own merits rather than through journal-level proxies. DORA’s later guidance also stresses that citation indicators must be interpreted transparently, specifically, contextually, and fairly.

The Leiden Manifesto for Research Metrics similarly states that quantitative evaluation should support, rather than replace, qualitative expert assessment. It also warns against comparing fields without accounting for their different publication and citation practices.

These limitations apply throughout science, but they become particularly severe in foundational mathematics.

Basic Mathematics Often Has a Long Time Horizon

Applied results can sometimes generate immediate citations because other researchers can insert a method, dataset, or algorithm into current work.

Basic mathematics frequently follows a different timeline.

A new definition, abstraction, theorem, or proof technique may need to pass through several stages:

1. A small group of specialists understands it.
2. Other mathematicians simplify or generalize it.
3. Connections to established theories become visible.
4. Expository work makes it accessible to a wider community.
5. Applications emerge in mathematics, physics, computing, or engineering.

This process may take decades.

Citation-based evaluation therefore contains a strong short-term bias. It rewards work that fits an existing research program more quickly than work that creates a new program.

A result may be mathematically important precisely because it does not yet have a large community prepared to cite it.

Small Fields Produce Small Citation Counts

Citation totals depend partly on the number of people available to produce citations.

Suppose one paper is published in a field with 20,000 active researchers and another in a specialized field with 100 active researchers. Even when the second paper is central to its subject, its maximum realistic citation rate may be much lower.

This creates a basic denominator problem:

A paper with 30 citations in a field of 100 researchers may have greater field-level influence than a paper with 300 citations in a field of 20,000 researchers.

Field-normalized metrics try to address this problem, but they cannot fully solve it. The boundaries of mathematical fields are ambiguous, interdisciplinary papers are difficult to classify, and radically new work may not belong to a stable comparison group.

Basic mathematics is particularly resistant to normalization because a foundational contribution can redefine the field in which it would supposedly be evaluated.

Foundational Results May Disappear Into Standard Knowledge

Successful mathematics is often absorbed into the discipline.

A theorem may initially be associated with a particular paper, but later researchers may cite:

• a textbook;
• a survey;
• a software library;
• a standard name for the theorem;
• a later and more accessible proof;
• no source at all because the result has become common knowledge.

This produces a paradox: the more thoroughly a mathematical idea becomes part of the intellectual infrastructure, the less accurately direct citations may trace its influence.

The original work may become conceptually indispensable but bibliographically invisible.

Citation analysis is therefore better at tracing explicit references than at tracing the actual transmission of mathematical ideas.

Citations Measure Use More Easily Than Possibility

Some mathematical results are valuable because they solve a current problem. Others are valuable because they make entire classes of future problems expressible.

A foundational contribution may introduce:

• a new mathematical object;
• a more general definition;
• a bridge between previously separate theories;
• a language that removes unnecessary distinctions;
• a proof method that can be reused;
• a classification that reorganizes a subject;
• a formal framework from which later results follow.

The value of such work is partly option value: it expands what future researchers can formulate, investigate, or prove.

Citation counts primarily record realized academic use. They do not measure the range of discoveries that a framework makes possible.

This is similar to evaluating infrastructure only by counting the vehicles currently using it. A bridge to a newly developing region may initially have little traffic while still being strategically important.

Difficult Work Can Receive Fewer Citations

Citation systems often favor results that are easy to recognize, explain, and reuse.

A long or technically difficult mathematical work may receive fewer citations because:

• few researchers can verify it;
• the notation is unfamiliar;
• the prerequisite knowledge is extensive;
• the work has not yet been translated into standard terminology;
• journals and reviewers lack suitable specialists;
• applying it requires substantial additional research.

Low citation counts may therefore reflect a verification bottleneck, not low value.

This does not mean that every obscure or difficult manuscript is important. Complexity is not evidence of correctness or significance. It means only that citation scarcity cannot distinguish among several possibilities:

• the result is wrong;
• the result is correct but minor;
• the result is correct but inaccessible;
• the result is foundational but premature;
• the result is important but institutionally neglected.

A useful evaluation system must investigate these possibilities rather than treating them as equivalent.

Citations Do Not Establish Correctness

Papers can be cited because researchers agree with them, dispute them, correct them, compare against them, or mention them historically.

A highly cited result can still contain an error. Conversely, a correct theorem may receive little attention.

Citation counts do not directly test:

• whether a proof is valid;
• whether assumptions are necessary;
• whether definitions are coherent;
• whether the theorem is genuinely new;
• whether a claimed generalization is strict;
• whether the result has been independently checked.

For mathematics, proof verification is logically prior to impact measurement. A thousand citations cannot turn an invalid proof into a valid one.

Automated and human evaluation should therefore separate at least three questions:

1. Is the result correct?
2. What mathematical advance does it contain?
3. How widely has it already been used?

Citation metrics address only part of the third question.

Citations Reward Established Research Networks

Citation visibility is affected by social structure.

Researchers at prominent institutions generally have better access to conferences, collaborators, journals, seminars, and influential academic networks. Their work may circulate quickly even before its significance has been independently established.

Researchers outside established institutions may face the opposite problem. Their work can remain unread because potential evaluators use affiliation, reputation, or existing citation counts as preliminary filters.

This creates cumulative advantage:

1. Recognition produces citations.
2. Citations produce professional credibility.
3. Credibility produces visibility and funding.
4. Visibility produces more citations.

Research on cumulative advantage in science has documented how metric-based systems can reinforce existing inequalities rather than neutrally measuring quality.

In basic mathematics, where verification may require rare expertise, dependence on reputation can become especially strong.

Fashionable Problems and Foundational Problems Have Different Citation Profiles

A paper that improves a currently popular method may rapidly attract citations from a large active community.

A paper that questions the foundations, changes the available language, or creates a new domain may initially attract fewer citations because it does not fit existing workflows.

Citation optimization can therefore encourage researchers to:

• divide work into many papers;
• remain close to fashionable topics;
• produce incremental variations;
• cite strategically;
• avoid high-risk foundational research;
• choose problems with large existing audiences.

These incentives are rational under metric-driven career systems, but they are not necessarily aligned with the long-term needs of mathematics.

The result can be a portfolio imbalance: too much work optimized for measurable short-term attention and too little work aimed at creating new mathematical foundations.

Negative Citations and Routine Citations Inflate Counts

Not every citation represents substantial intellectual dependence.

A paper may be cited:

• as one item in a long literature review;
• because it uses a standard method;
• to identify a disputed claim;
• because citation norms require acknowledging nearby work;
• through repeated copying of references from other papers;
• because it provides a convenient survey rather than an original result.

Meanwhile, a theorem deeply embedded in another proof may be used without a direct citation to its original source.

Citation counts therefore mix weak, strong, positive, negative, historical, and ceremonial references into one number.

A better system would distinguish among different relationships, such as:

• direct theorem dependency;
• use of a definition;
• use of a proof technique;
• independent verification;
• correction or refutation;
• conceptual influence;
• software implementation;
• educational transmission;
• downstream scientific application.

Raw counts discard this structure.

The H-Index Makes the Problem Worse

The h-index combines publication volume and citation counts. A researcher has an h-index of h when h publications have each received at least h citations.

Although convenient, the metric cannot reliably compare researchers across fields, career stages, or publication cultures. It also ignores many qualitative contributions, including teaching, collaboration, software, proofs, and the actual intellectual content of research. DORA specifically highlights these limitations and warns against treating the h-index as a complete measure of researcher quality.

For foundational mathematicians, the h-index introduces several distortions:

• one major monograph may count less than numerous incremental papers;
• long-term work is penalized;
• highly specialized contributions receive lower ceilings;
• young or independent researchers appear systematically weaker;
• deeply interconnected results may be divided unnaturally to improve metrics.

The h-index measures a particular publication pattern. It does not measure mathematical depth.

Historical Importance Is Often Visible Only Retrospectively

The ultimate significance of a mathematical contribution may depend on developments that did not exist when the work was published.

Ideas from number theory, logic, geometry, algebra, and topology have repeatedly acquired applications long after their creation. But evaluation committees must make decisions before history reveals the full result.

This creates an unavoidable epistemic limitation: no evaluator can perfectly predict future usefulness.

The appropriate response is not to pretend that citation counts solve the prediction problem. Instead, funding systems should explicitly acknowledge uncertainty and maintain a diversified research portfolio.

That portfolio should include:

• immediately applicable mathematics;
• incremental development of established theories;
• rigorous work in small specialties;
• high-risk foundational research;
• preservation and exposition of difficult theories;
• independent proof checking and replication.

A robust scientific system does not require every contribution to demonstrate rapid popularity.

What Should Replace Citation Counts?

Citation counts do not need to be abolished. They can remain one limited source of evidence. The problem begins when they are treated as the primary definition of scientific value.

A better model would combine several dimensions.

1. Correctness

For mathematical research, evaluators should examine whether the definitions, statements, and proofs are valid.

Evidence may include:

• specialist review;
• independent proof reconstruction;
• formal verification;
• successful correction of identified gaps;
• consistency with established results.

2. Originality

The system should determine what is genuinely new:

• a theorem;
• a proof;
• a definition;
• a generalization;
• a counterexample;
• a unifying abstraction;
• a new research question.

Originality should not be inferred from publication venue or author reputation.

3. Depth and Difficulty

Some contributions solve problems that resisted sustained effort. Others reveal a simple observation that reorganizes a subject.

Difficulty alone is not value, but evaluators should recognize the intellectual work required to produce and verify a result.

4. Foundational Reach

A mathematical framework may deserve recognition when it:

• unifies separate theories;
• weakens unnecessary assumptions;
• extends methods to a larger domain;
• supplies reusable language;
• reveals previously hidden relations;
• supports multiple future research directions.

This dimension is often invisible to citation counts during the early life of a theory.

5. Dependency and Use

Instead of merely counting references, evaluation should examine how later work depends on the contribution.

A theorem used centrally in five major results may be more influential than a paper mentioned superficially in 500 introductions.

Dependency-aware evaluation could trace specific mathematical and computational relationships.

6. Verification and Replication

Researchers who verify, formalize, reproduce, explain, or correct mathematical results create value even when they do not produce highly cited new theorems.

Scientific funding should therefore recognize replication as research infrastructure rather than treating verification as unpaid secondary labor.

7. Long-Term Expert Reassessment

Evaluation should remain revisable.

A contribution that initially appears minor may later become central. Another that initially attracts attention may fail under scrutiny. Funding and reputation systems should be able to update their assessments without permanently binding scientific value to early popularity.

How AI-Assisted Evaluation Could Help

AI systems may eventually help analyze mathematical work at a finer level than bibliometric databases.

For example, an AI-assisted evaluation system could attempt to:

• map definitions and theorem dependencies;
• compare claims with prior literature;
• identify strict generalizations;
• locate equivalent formulations;
• detect unsupported claims;
• recommend qualified human reviewers;
• track corrections and verification status;
• trace use in proofs, software, and applications;
• update assessments as new evidence appears.

However, AI confidence is not mathematical certainty. Such systems must remain auditable, contestable, and subject to specialist review.

The purpose of AI should not be to replace citation counts with another opaque score. It should be to expose the evidence behind an evaluation.

How AIIM Could Evaluate Basic Mathematics More Fairly

The AI Internet Meritocracy, or AIIM, proposes evaluating scientific outputs directly rather than funding researchers mainly through promises, prestige, or conventional journal metrics.

For basic mathematics, this approach could separate rewards for:

• formulating a valuable problem;
• introducing a useful definition;
• proving a theorem;
• supplying a counterexample;
• verifying a proof;
• correcting an error;
• formalizing a result;
• writing an important exposition;
• connecting previously separate theories;
• demonstrating a downstream application.

This distinction is critical. A paper is not a single indivisible unit of value. It may contain several contributions with different levels of certainty, originality, and usefulness.

AIIM could also revisit assessments over time. Early funding might reward verified novelty, while later rewards could recognize newly discovered dependencies or applications.

Such a model would not eliminate expert judgment. It would make the grounds for judgment more explicit and reduce dependence on citations as a crude proxy.

Readers can explore related proposals in the discussion of funding research according to results rather than promises and merit-based science funding.

Citation Counts Are Evidence, Not Value

Citation metrics remain useful for limited questions:

• Has this work attracted academic attention?
• Which papers explicitly reference it?
• How has attention changed over time?
• In which research communities is it discussed?

They cannot independently answer the more important questions:

• Is the mathematics correct?
• Is it original?
• Does it solve an important problem?
• Does it create a useful conceptual framework?
• Will it become foundational?
• Was it unfairly ignored?
• How much future research will depend on it?

The central mistake is not counting citations. It is confusing the count with the thing being valued.

Conclusion

Citation counts cannot measure the full value of basic mathematics because mathematical value is multidimensional, field-dependent, structurally complex, and often visible only over long periods.

Foundational mathematics may have few immediate users, a small specialist audience, difficult notation, and no obvious application. Yet it can still reshape the language of mathematics, connect separate theories, or enable future discoveries.

Research evaluation should therefore treat citations as one contextual signal among many. Correctness, originality, foundational reach, dependency, verification, exposition, and long-term usefulness must be evaluated separately.

The future of mathematical assessment should not be a more elaborate popularity ranking. It should be an evidence-based system capable of recognizing contributions before popularity arrives.