Beyond Citations: Replacing H-Index Bottlenecks with Multidimensional AI Research Scoring

Getting your Trinity Audio player ready...

The H-index measures citation performance—not scientific merit in its entirety. It can indicate that a researcher has produced several frequently cited papers, but it does not directly measure whether those papers are reproducible, whether their data and code are usable, whether the researcher performs valuable peer review, or whether their work provides essential infrastructure for later discoveries.

A better research-assessment system should not merely count citations more accurately. It should evaluate multiple forms of verifiable scientific contribution.

Multidimensional AI scoring could help by examining reproducibility, code quality, data availability, peer-review work, replication results, methodological rigor, and the dependencies between scientific projects. Such a system would not eliminate human judgment. Its purpose would be to organize evidence, expose the reasons behind an assessment, and reduce dependence on a single bibliometric number.

This is the approach proposed by AI Internet-Meritocracy: funding science and open-source development according to demonstrable contribution rather than institutional status or one-dimensional citation metrics.

What Does the H-Index Measure?

Jorge Hirsch introduced the H-index in 2005 as a compact measure combining publication output and citation impact. A researcher has an H-index of h when h of their papers have each received at least h citations.

For example, a researcher has an H-index of 20 when at least 20 of their publications have received 20 or more citations each.

The metric has practical advantages:

• it is relatively easy to calculate;
• it is less dominated by one exceptionally cited publication than total citation count;
• it combines a measure of productivity with a measure of academic attention;
• it is more robust than some citation indicators against errors in the long tail of citation records.

These advantages explain why the H-index became influential. The problem begins when a useful bibliometric indicator is treated as a general score of researcher quality.

The H-index answers a narrow question: how many of a researcher’s publications have crossed a particular citation threshold? It does not answer whether the research is correct, reproducible, open, original, socially useful, or technically well implemented.

How the H-Index Creates Research Bottlenecks

It rewards accumulated visibility

Citations normally take time to accumulate. Established researchers therefore have a structural advantage over early-career researchers, independent scholars, and people entering a new field.

A young researcher may publish a rigorous and transformative result while retaining a low H-index for years. An older researcher may continue to benefit from citations to work completed decades earlier.

The index can therefore become a lagging measure of recognition rather than a timely measure of present contribution.

It differs radically between disciplines

Citation practices vary between mathematics, biology, physics, computer science, medicine, and the humanities. Large experimental fields may generate extensive co-authorship and citation networks, while highly abstract or specialized fields may have small research communities.

Directly comparing H-indices across fields can therefore produce misleading conclusions. The Leiden Manifesto explicitly recommends accounting for differences between fields and using quantitative evaluation to support, rather than replace, expert assessment.

It cannot distinguish use from approval

A citation may indicate that another paper:

• uses a result;
• criticizes it;
• attempts to replicate it;
• mentions it historically;
• copies a standard citation from related literature;
• cites a popular review rather than the original discovery.

Citation counts record links between documents, but usually do not encode the meaning of those links.

It under-rewards research infrastructure

Scientific work increasingly depends on software, datasets, documentation, benchmarks, ontologies, formal proofs, laboratory protocols, and maintenance.

A developer may create software used by thousands of researchers without receiving citations proportional to its importance. FORCE11’s Software Citation Principles were developed partly because research software is critical to scholarship but has historically received inadequate recognition within the publication system.

It does not test reproducibility

A highly cited result can later prove difficult to reproduce. Conversely, a replication study, negative result, or correction may be scientifically valuable while attracting fewer citations than an initially dramatic claim.

The Transparency and Openness Promotion Guidelines identify research and verification practices intended to make scientific claims more transparent and verifiable. The updated framework covers research practices, verification practices, and verification studies.

None of these practices is visible in an ordinary H-index.

It overlooks peer review and scientific maintenance

Peer review is necessary for detecting errors, improving exposition, checking methods, and protecting the scientific record. Yet careful reviewers commonly receive little formal recognition.

The same applies to:

• reproducing another team’s analysis;
• correcting published mistakes;
• maintaining scientific software;
• answering technical questions;
• improving documentation;
• curating datasets;
• connecting previously isolated theories.

A citation-only system can reward the final visible paper while overlooking much of the work that made it reliable.

From One Number to a Scientific Contribution Profile

The solution is not necessarily to create a more complicated universal ranking. Compressing every contribution into one number can recreate the same problem under a new name.

A multidimensional system should first produce a contribution profile. Any aggregate score used for funding or discovery should remain secondary, contextual, and explainable.

A research profile could contain dimensions such as the following.

Dimension	Possible evidence
Research influence	Field-normalized citations, substantive use by later work, downstream dependencies
Reproducibility	Independent replications, executable workflows, available environments, verified results
Methodological rigor	Appropriate controls, preregistration where relevant, statistical checks, formal verification
Code quality	Tests, documentation, maintainability, versioning, security review, issue resolution
Data quality	Provenance, metadata, licensing, validation, accessibility, reuse
Peer-review contribution	Review quality, corrections identified, author feedback, editorial verification
Openness	Accessible papers, code, data, protocols and research artifacts
Originality	Novel concepts, methods, proofs, datasets or research directions
Scientific infrastructure	Software, standards, databases, libraries, benchmarks and long-term maintenance
Dependency value	Importance of a contribution to later projects, even where direct citations are sparse

The output should explain why each dimension received its assessment and link to the supporting evidence.

How AI Could Evaluate Reproducibility

Reproducibility should not be inferred merely from whether a paper contains a link to a repository. The system could examine several levels of evidence.

Availability

Are the paper, code, data, configuration files, dependency versions and instructions accessible?

Executability

Can the computational workflow be executed in a documented environment? Are dependencies pinned? Are required inputs available?

Result consistency

Does executing the supplied workflow regenerate the reported tables, figures or numerical results within an appropriate tolerance?

Independent verification

Has another researcher reproduced the result using the original artifacts? Has anyone replicated the underlying claim using independently produced data or methods?

Response to failure

When reproducibility problems are reported, does the author investigate them, correct the materials and document the changes?

This produces a more meaningful assessment than a binary “code available” label. Recent research on computational reproducibility also indicates that documentation quality and standardized execution environments materially affect whether others can rerun published methods.

How AI Could Evaluate Research Code

Code quality is not identical to scientific correctness. Elegant software can implement an invalid method, while untidy code may still produce a correct result. The two questions must remain separate.

An AI-assisted assessment could inspect:

• automated test coverage;
• reproducible builds;
• documentation and examples;
• modularity and readability;
• issue handling;
• version control history;
• release practices;
• dependency management;
• licensing;
• static-analysis results;
• independent code review;
• correspondence between code and the published method.

Research-software guidance commonly emphasizes planning, readable implementation, version control, testing, modular design, documentation, reproducibility and long-term maintenance.

However, automated code metrics must not become rigid universal rules. Standards appropriate for safety-critical aerospace software, for example, are not automatically appropriate for a short exploratory mathematics script. Assessment should be calibrated to the artifact’s purpose and risk level.

How AI Could Recognize Peer Review

Peer review is difficult to score because much of it is confidential and because quantity does not imply quality.

Counting completed reviews would encourage shallow reviewing. A better system could recognize peer-review contribution through evidence such as:

• whether authors found the review useful;
• whether the review identified a verified error;
• whether it improved methodology or exposition;
• whether the reviewer supplied tests, proofs or replication evidence;
• whether editors confirmed the review’s relevance;
• whether later corrections validate its concerns;
• whether the review was timely and constructive.

Open reviews can be assessed directly. Confidential reviews could use privacy-preserving attestations from journals or editors without exposing manuscript content.

A reviewer who discovers a fundamental error may contribute more to science than someone who publishes several routine reports. Research assessment should be capable of representing this fact.

AI Scoring Should Analyze Scientific Dependencies

Citation graphs treat research as a network of papers. Science is actually a network of dependencies between contributions.

A later project may depend on:

• a theorem;
• a software library;
• a dataset;
• an experimental instrument;
• a classification system;
• a correction;
• an algorithm;
• a protocol;
• an informal technical explanation.

The foundational contribution may receive few citations, particularly when it is absorbed into standard practice or hidden inside software dependencies.

AI could help construct a richer contribution graph by analyzing papers, repositories, datasets, package manifests, documentation and verified statements from researchers. Funding could then recognize work that supports important downstream projects.

This idea is central to AI meritocracy in research funding: evaluate a broad, open contribution record and trace how scientific outputs support one another rather than allocating resources primarily through credentials, proposals or publication counts.

Why Multidimensional AI Scoring Is Not Automatically Fair

Replacing the H-index with an opaque AI score would not solve research assessment. It could make the problem worse.

Hidden weights can encode policy choices

How much should reproducibility count relative to originality? Should a widely used software package outrank a difficult theorem? There is no purely technical answer.

These are governance decisions and should be declared openly.

Different fields require different evidence

Reproducibility in pure mathematics means something different from reproducibility in clinical medicine. Code quality may be central to computational biology but irrelevant to a historical archive based on physical documents.

A legitimate system needs field-sensitive criteria without allowing disciplinary communities to exclude unconventional work automatically.

Automated scoring can be gamed

Once a metric controls funding, people adapt their behavior to maximize it. Researchers could manufacture superficial documentation, reciprocal reviews, artificial dependencies or low-value repository activity.

The Leiden Manifesto warns that indicators affect the systems they measure and should therefore be scrutinized and updated regularly.

AI can misinterpret technical work

An AI model may produce confident but incorrect judgments about mathematical proofs, experimental methods or specialized software. Automated assessment must distinguish between:

1. directly verified facts;
2. model-generated inferences;
3. human evaluations;
4. disputed claims.

Reputation feedback loops can persist

If an AI system trains mainly on historical citations, journal prestige and institutional signals, it may reproduce the same hierarchy it was intended to replace.

The system must not call itself multidimensional while using conventional prestige as a hidden proxy.

Requirements for a Legitimate AI Research-Assessment System

A defensible system should include:

Transparent criteria

Researchers must know which dimensions are assessed, what evidence is used and how funding decisions are derived.

Evidence-level explanations

Every score should link to the papers, tests, reviews, replications or repository records supporting it.

Separate dimensions

The system should preserve distinctions between originality, correctness, openness, software quality and influence rather than hiding them inside one unexplained number.

Human appeal

Researchers need a practical mechanism for contesting incorrect data, mistaken identity, misclassified contributions and invalid model conclusions.

Versioned models and weights

Changes to scoring criteria should be documented. Past decisions should remain auditable under the model version that produced them.

Anti-gaming analysis

The system should detect suspicious citation circles, coordinated reviews, fabricated activity and attempts to divide one contribution into many superficial artifacts.

Uncertainty disclosure

A score based on strong replication evidence should not be presented with the same confidence as a speculative AI interpretation.

Independent governance

The organization operating the model should not have unrestricted authority to change rankings or redirect funds secretly.

These requirements align with the broader movement toward responsible research assessment. DORA calls for improving how research outputs are evaluated, while CoARA promotes qualitative assessment supported by responsible use of quantitative indicators.

How AI Internet-Meritocracy Could Go Beyond Citations

AI Internet-Meritocracy proposes using software to evaluate scientific and open-source contributions and distribute donated funds according to measurable merit.

Within such a framework, citations can remain one source of evidence, but they should not dominate the assessment.

A possible workflow would be:

1. Collect public research artifacts.
   Identify papers, preprints, code repositories, datasets, reviews and documented contributions.
2. Resolve identities and authorship.
   Connect contributions to the correct researchers while recording uncertainty and disputed claims.
3. Build a contribution graph.
   Map citations, software dependencies, data reuse, methodological influence, replication and peer-review activity.
4. Assess multiple dimensions.
   Evaluate reproducibility, rigor, code, openness, originality and downstream utility separately.
5. Publish explanations.
   Show the evidence and reasoning behind each assessment.
6. Permit correction and appeal.
   Allow researchers and reviewers to challenge false or incomplete conclusions.
7. Allocate funds continuously.
   Distribute donations according to current evidence rather than forcing researchers to wait for occasional grant competitions.

This would change the core question from “How many citations has this researcher accumulated?” to “What verifiable contributions has this person made to the functioning and progress of science?”

Beyond Bibliometric Meritocracy

The H-index should not simply be abolished and replaced by an AI-generated super-index. Citations remain useful evidence of attention and influence. They become harmful when institutions mistake them for a complete representation of scientific value.

A multidimensional system can recognize that scientific progress depends on different kinds of work:

• proposing an original theory;
• proving a theorem;
• producing reliable data;
• writing reusable software;
• replicating an experiment;
• identifying an error;
• reviewing a manuscript carefully;
• maintaining research infrastructure;
• making another researcher’s work understandable and executable.

No one metric captures all of these contributions.

The appropriate goal is therefore evidence-based research assessment, not automated numerical authority. AI can read larger bodies of evidence than a temporary committee and detect relationships that ordinary citation counts miss. But its conclusions must remain transparent, contestable, field-aware and subject to human governance.

Science can move beyond citations without abandoning measurement. It must measure what actually makes research valuable.