Should Teaching, Reviewing, and Dataset Maintenance Count as Scientific Output?

Getting your Trinity Audio player ready...

Yes—teaching, peer review, and dataset maintenance should count as scientific output when they produce identifiable, assessable, and reusable value. They should not necessarily receive the same kind or amount of recognition as an original theorem, experiment, or discovery. However, excluding them entirely creates a distorted picture of how science actually works.

Scientific progress depends on more than publishing papers. Researchers also:

• train people who will produce future discoveries;
• detect errors before or after publication;
• maintain datasets that support hundreds of analyses;
• document methods and software;
• preserve research objects so that results remain reproducible;
• explain difficult ideas in forms that other people can use.

A credible research-assessment system should therefore distinguish between different kinds of scientific output rather than forcing every contribution into the category of “journal article.”

What Counts as Scientific Output?

A scientific output is a durable contribution that improves the creation, verification, communication, preservation, or application of reliable knowledge.

This definition is broader than “a paper written by a researcher,” but it is not unlimited. Routine activity does not automatically become scientific output merely because it happens inside a university.

A contribution should normally satisfy several conditions:

1. It has an identifiable product or result.
2. Its creator or maintainer can be identified.
3. Its quality can be evaluated.
4. It provides value beyond the act of performing a job.
5. Its use, influence, or dependency relationships can be documented.

Under this framework, a maintained dataset may qualify, while merely attending a data-management meeting would not. A rigorous public review may qualify, while clicking “accept” on a manuscript without analysis would not.

Why Papers Alone Cannot Represent Scientific Production

The journal article became the dominant unit of research assessment partly because it is convenient to count. That does not mean it captures every scientifically valuable contribution.

A paper may depend on:

• years of dataset collection and correction;
• software maintained by researchers who are not listed as authors;
• specialist reviewers who identify serious methodological errors;
• teachers who developed the researcher’s technical abilities;
• documentation that makes replication possible;
• curators who preserve earlier results and metadata.

When assessment recognizes only the final paper, it rewards the visible endpoint while neglecting much of the infrastructure that made the result possible.

This is inconsistent with contemporary open-science policy. UNESCO’s definition of open scientific knowledge includes not only publications but also research data, educational resources, software, source code, and hardware. Its Recommendation on Open Science specifically states that research data require regular curation and maintenance.

A more accurate system should represent science as a network of dependent contributions.

Should Teaching Count as Scientific Output?

Teaching should count when it creates transferable scientific value

Ordinary teaching is important professional work, but not every lecture should be treated as a separate research output. The relevant distinction is between delivering instruction and creating a durable intellectual or educational contribution.

Teaching-related scientific outputs may include:

• an original textbook or advanced set of lecture notes;
• a carefully designed open course;
• a new explanation of a difficult scientific concept;
• verified educational software or interactive demonstrations;
• a curriculum introducing an emerging field;
• problem collections, formal exercises, or laboratory protocols;
• educational resources that are reused by other institutions;
• mentorship that leads to documented scientific work.

UNESCO treats open educational resources as part of the wider open-science ecosystem, supporting the position that educational materials can be legitimate knowledge outputs rather than merely auxiliary university services.

Teaching should not be measured only by student popularity

Student evaluations can provide useful evidence, but they are vulnerable to factors unrelated to scientific or educational quality. A demanding instructor may receive lower ratings than an entertaining but less rigorous one. Ratings can also reflect class size, grading expectations, subject difficulty, and bias.

Teaching output should instead be evaluated through evidence such as:

• accuracy and intellectual depth;
• originality of presentation;
• adoption by other educators;
• measurable learning outcomes;
• accessibility and documentation;
• continuing use over time;
• influence on subsequent research or professional practice.

A widely reused explanation of a difficult theorem may contribute more to science than a minor publication that nobody uses. The assessment system should be capable of recognizing that possibility without assuming that all teaching is equivalent to research discovery.

Should Peer Review Count as Scientific Output?

Substantive review is a form of scientific quality control

Peer review can reveal:

• invalid assumptions;
• statistical errors;
• missing prior work;
• irreproducible methods;
• unsupported conclusions;
• ethical problems;
• unclear definitions;
• opportunities for stronger experiments or proofs.

A detailed review may materially improve a paper, prevent a false result from entering the literature, or identify a new research question. Such work creates scientific value even when the review itself remains unpublished.

Responsible research-assessment frameworks increasingly recognize peer review alongside teaching, mentoring, software engineering, and data curation as part of the diverse processes through which researchers contribute.

Review should be credited carefully

Counting every completed review equally would produce bad incentives. Researchers might submit superficial reports simply to accumulate points. Review should therefore be recognized according to its demonstrated substance and subsequent usefulness.

Possible evidence includes:

• editor or author assessments of the report;
• whether the review identified a confirmed problem;
• whether recommendations were adopted;
• the depth and specificity of the analysis;
• publication of an open review;
• later validation by other reviewers or replications;
• the reviewer’s record of accuracy over time.

Confidentiality must also be preserved where necessary. A system can record that a verified review occurred without immediately disclosing the manuscript, author, or report. Credit may be released later if the journal’s policy and participants permit it.

Reviews should be independently citable where possible

Open, signed reviews can become research objects with persistent identifiers. Even when a review is anonymous, the system could maintain a cryptographically or institutionally verified record proving that a contribution was made.

This would make reviewing visible without converting it into authorship. The reviewer would receive credit for evaluation, not for creating the reviewed result.

The distinction matters:

Authorship creates or reports a scientific contribution; reviewing evaluates, corrects, or validates one. Both can have value, but they are different kinds of output.

This broader model also supports the idea of professional science marketers: independent evaluators who identify valuable scientific work, explain its importance, and risk their own reputations by recommending it. Such people could perform part of the filtering traditionally associated with journals without needing to own the publication process.

Related reading: Who Should Pay Peer Reviewers—and for What Exactly? and A Reputation System for Scientific Reviewers.

Should Dataset Maintenance Count as Scientific Output?

Dataset maintenance is often indispensable scientific work

Creating a dataset is not a single event. Valuable datasets may require continuing work to:

• correct errors;
• remove duplicates;
• harmonize formats;
• document variables;
• track provenance;
• update classifications;
• manage consent or access restrictions;
• preserve compatibility with analytical tools;
• publish new versions;
• respond to reports from users;
• prevent link rot or file loss.

Without this maintenance, a dataset can gradually become misleading or unusable.

The FAIR Data Principles state that research data should be Findable, Accessible, Interoperable, and Reusable. These properties do not arise automatically. They depend on metadata, documentation, identifiers, standards, stewardship, and continuing technical work.

UNESCO likewise describes open research data as requiring good stewardship together with regular curation and maintenance.

Maintenance should receive version-specific credit

Dataset creators and maintainers should not be treated as one undifferentiated group. A useful record should specify roles such as:

• initial collection;
• validation;
• data cleaning;
• metadata design;
• ontology development;
• privacy review;
• infrastructure administration;
• error correction;
• version migration;
• long-term preservation.

The Contributor Roles Taxonomy, commonly called CRediT, already recognizes data curation as a distinct contributor role. This demonstrates that contribution records can be more precise than a binary author/non-author distinction.

Each important dataset version should have:

• a persistent identifier;
• a release date;
• a change log;
• named or pseudonymously verifiable contributors;
• machine-readable metadata;
• licensing and access conditions;
• links to previous and subsequent versions.

Data citation standards should make citations understandable both to people and to machines, allowing dataset use to become part of formal research attribution.

Different Outputs Require Different Evaluation Criteria

Teaching, reviewing, datasets, software, replications, and original discoveries should not be collapsed into one score without explanation.

Output type	Primary value	Appropriate evidence
Original research	Produces new knowledge	Validity, novelty, importance and later use
Teaching resource	Transfers scientific understanding	Adoption, accuracy, learning value and reuse
Peer review	Evaluates or improves research	Depth, correctness, detected problems and adopted changes
Dataset creation	Produces reusable evidence	Quality, documentation, uniqueness and scientific use
Dataset maintenance	Preserves reliability and accessibility	Corrections, updates, availability and downstream dependencies
Software maintenance	Keeps computational research operational	Reliability, fixes, reuse and dependent projects
Replication	Tests existing findings	Methodological rigor and effect on confidence
Scientific synthesis	Connects existing knowledge	Coverage, accuracy, explanatory power and subsequent use

A weighting system may conclude that a major discovery deserves a much larger reward than one routine dataset correction. That is reasonable. The correction should nevertheless receive some credit when it verifiably prevents errors or preserves downstream work.

The correct alternative to treating all outputs equally is differentiated evaluation, not exclusion.

Risks of Broadening the Definition of Scientific Output

Recognition systems can be manipulated. Expanding the list of outputs without improving verification could generate large quantities of low-value material.

Output inflation

Researchers might divide one contribution into many tiny records. Systems should therefore distinguish meaningful updates from trivial administrative changes.

Popularity replacing quality

Downloads, views, or student ratings may indicate attention without proving scientific value. Usage metrics should be supporting evidence, not final judgment.

Reciprocal reviewing

Groups could reward one another through favorable reviews. Conflict disclosures, reviewer-reputation models, independent checks, and anomaly detection would be necessary.

Maintenance monopolies

A maintainer might resist improvements that reduce dependence on their role. Open governance, documented procedures, interoperable formats, and succession plans can reduce this risk.

Excessive metric aggregation

A single numerical score can conceal essential differences. A scientist who produces excellent datasets and a scientist who proves theorems may both be valuable, but not in identical ways. Their contribution profiles should remain visible.

How AIIM Could Reward Multiple Forms of Scientific Output

AI Internet Meritocracy could represent scientific production as a dependency graph rather than as a list of papers.

In such a graph:

• a paper may depend on a dataset;
• the dataset may depend on long-term maintenance;
• an experiment may depend on software;
• a corrected paper may depend on a reviewer’s report;
• a researcher’s work may depend on a specialized educational resource;
• a replication may raise or lower confidence in an earlier result.

The system could assign separate rewards to each contribution according to its verified role and downstream importance.

For example, when a dataset supports many valuable studies, part of the resulting reward could flow to:

1. the dataset’s original creators;
2. contributors who corrected major errors;
3. maintainers who kept it usable;
4. metadata or ontology designers;
5. reviewers who verified its quality.

Teaching outputs could receive rewards when later research demonstrably depends on them. Reviewers could receive rewards when their assessments prove accurate or prevent unsupported claims from being funded.

AIIM would not need to declare every contribution valuable in advance. It could distribute small initial rewards and revise them as dependency evidence, evaluations, replications, and uses accumulate.

This is consistent with research funding based on demonstrated results rather than promises, while also recognizing that the “result” may be infrastructure, education, evaluation, or preservation—not only a conventional paper.

A Better Definition of Scientific Merit

Scientific merit should measure more than authorship. It should examine how a contribution affects the entire knowledge process.

A defensible assessment can ask:

• Did the contribution create new knowledge?
• Did it make existing knowledge understandable?
• Did it detect or correct an error?
• Did it make evidence reusable?
• Did it preserve an important scientific resource?
• Did other valuable work depend on it?
• Was the contribution performed rigorously and transparently?
• Can its effect be independently verified?

These questions recognize different scientific roles without pretending that all roles are interchangeable.

Conclusion

Teaching, reviewing, and dataset maintenance should count as scientific outputs when they leave verifiable products or effects that improve science.

Recognition should not mean awarding every activity the status of a major discovery. It should mean recording contributions accurately, evaluating each according to its proper criteria, and distributing credit in proportion to demonstrated value.

Science is not produced only by the person whose name appears first on a paper. It is produced by a network of creators, teachers, reviewers, curators, maintainers, replicators, and tool builders.

A scientific reward system becomes more accurate—not less—when it can see that network.