The Reproducibility Crisis

Overview

The reproducibility crisis (also called the replication crisis) is the finding that a large share of published scientific results cannot be reproduced when independent researchers repeat the experiment. It spans psychology, biomedicine, economics, and beyond, and it is the deepest of the "misaligned incentive" problems that Decentralized Science sets out to address. Where decentralized publishing targets how results are disseminated and DeSci funding targets how research is paid for, the reproducibility crisis is about a prior question: whether the published record is even true.

The crisis is not primarily about fraud. It is a systems problem: the incentives that govern careers, grants, and journals reward novel, positive, statistically-significant results and quietly penalise the replications, null results, and shared data that would keep the record honest.

How deep the crisis runs

Three landmark pieces of evidence anchor the modern debate:

Ioannidis (2005), "Why Most Published Research Findings Are False." The most-cited paper in PLoS Medicine argued from first principles that, given small studies, small effect sizes, flexible analysis, and publication bias, the majority of claimed research findings in many fields are likely false positives.
Reproducibility Project: Psychology (2015). The Open Science Collaboration directly replicated 100 studies from three top psychology journals. 97% of the originals had reported statistically-significant results; on replication only 36% did, and just 39% of effects were judged to have replicated — with replication effect sizes roughly half the originals (Science; project on OSF).
Baker's Nature survey (2016). A survey of over 1,500 scientists found that more than 70% had tried and failed to reproduce another scientist's experiments, and more than half had failed to reproduce their own — yet most had never published a failed replication ("1,500 scientists lift the lid on reproducibility").

Parallel large-scale replication efforts in experimental economics and cancer biology (the Reproducibility Project: Cancer Biology) reached similar conclusions: a meaningful fraction of headline findings simply do not hold up.

Why it happens: the incentive structure

The crisis is best understood as the predictable output of the incentives researchers actually face, not as individual carelessness:

Publish or perish. Hiring, tenure, and grants reward publication volume and novelty. Replications and null results rarely advance a career, so they rarely get done or published.
Publication bias (the file-drawer problem). Journals preferentially publish positive, significant findings. The negative results that would balance the record stay in the drawer, inflating the apparent strength of every published effect.
p‑hacking & researcher degrees of freedom. Flexible choices in data collection and analysis — which outliers to drop, which covariates to include, when to stop collecting — let researchers reach the p < 0.05 threshold from noise, often without conscious intent (Simmons, Nelson & Simonsohn, "False-Positive Psychology").
HARKing. "Hypothesising After the Results are Known" — presenting a post-hoc explanation of a chance finding as if it had been predicted in advance — turns exploratory noise into a confirmatory-looking story.
Closed data and code. When the underlying data and analysis code are not shared, results cannot be checked, re-run, or built on, and errors go undetected.

What fixes it: open, credible science

The reform agenda is largely about changing incentives and making the process transparent by default:

Preregistration. Recording hypotheses and the full analysis plan before seeing the data removes the flexibility that p‑hacking and HARKing exploit (Center for Open Science; Nosek et al., "The preregistration revolution").
Registered Reports. Journals peer-review and accept a study on the strength of its question and method before results exist, so publication no longer depends on the outcome (COS Registered Reports).
Open data & code. Sharing the dataset and analysis pipeline lets anyone re-run and verify a result — the foundation of the FAIR data principles.
Open, credited peer review. Transparent, attributed, and increasingly paid review realigns the one step meant to catch problems — see decentralized publishing & peer review.
Funding negative and confirmatory work. Deliberately paying for replications and null results — a natural fit for the retroactive and community-directed models covered in DeSci funding.

Where DeSci — and Caper — fit

Most of the reproducibility crisis is a problem of scientific method and culture, and no blockchain "solves" it. What decentralised infrastructure can change is the layer underneath the incentives: making the record permanent and the decisions transparent. On-chain timestamps let a hypothesis or preregistration be provably fixed before data collection; content-addressed storage makes data and code tamper-evident; and token incentives can be pointed at the work — replications, null results, shared datasets — that the traditional system underfunds. That is the connective tissue between this crisis and the rest of the DeSci agenda.

How Caper approaches this. Caper is a DAO protocol, not a lab or a journal — it does not run experiments or host papers. Its contribution is the funding-and-governance rail a research community would sit on. Every proposal, vote, and payout is an on-chain action with typed outcomes (a passing PAYOUT or INVEST vote is the transfer), so who funded which work is a permanent public record rather than an off-chain minute. And a caper's decisive governance weight is an earned, non-transferable soulbound record of participation, not a tradeable token — so influence over which research a treasury backs cannot simply be bought by the best-capitalised bidder. It is a modest, honest fit: Caper supplies transparent, incentive-aligned rails; it does not adjudicate whether a given result replicates.

Topic	The reproducibility crisis
What it is	The widespread failure of published scientific findings to replicate when re-tested
Landmark evidence	Ioannidis (2005) · Reproducibility Project: Psychology (2015) · Baker's Nature survey (2016)
Root causes	Publication bias, p‑hacking, HARKing, publish‑or‑perish incentives, closed data & code
Proposed fixes	Preregistration, registered reports, open data/code, open peer review, realigned funding
Why DeSci cares	Blockchains can make method, data, and funding decisions transparent and permanent — attacking the incentives that reward irreproducible work

Overview

How deep the crisis runs

Three landmark pieces of evidence anchor the modern debate:

Ioannidis (2005), "Why Most Published Research Findings Are False." The most-cited paper in PLoS Medicine argued from first principles that, given small studies, small effect sizes, flexible analysis, and publication bias, the majority of claimed research findings in many fields are likely false positives.

Reproducibility Project: Psychology (2015). The Open Science Collaboration directly replicated 100 studies from three top psychology journals. 97% of the originals had reported statistically-significant results; on replication only 36% did, and just 39% of effects were judged to have replicated — with replication effect sizes roughly half the originals (Science; project on OSF).

Baker's Nature survey (2016). A survey of over 1,500 scientists found that more than 70% had tried and failed to reproduce another scientist's experiments, and more than half had failed to reproduce their own — yet most had never published a failed replication ("1,500 scientists lift the lid on reproducibility").

Why it happens: the incentive structure

The crisis is best understood as the predictable output of the incentives researchers actually face, not as individual carelessness:

Publish or perish. Hiring, tenure, and grants reward publication volume and novelty. Replications and null results rarely advance a career, so they rarely get done or published.

Publication bias (the file-drawer problem). Journals preferentially publish positive, significant findings. The negative results that would balance the record stay in the drawer, inflating the apparent strength of every published effect.

p‑hacking & researcher degrees of freedom. Flexible choices in data collection and analysis — which outliers to drop, which covariates to include, when to stop collecting — let researchers reach the p < 0.05 threshold from noise, often without conscious intent (Simmons, Nelson & Simonsohn, "False-Positive Psychology").

HARKing. "Hypothesising After the Results are Known" — presenting a post-hoc explanation of a chance finding as if it had been predicted in advance — turns exploratory noise into a confirmatory-looking story.

Closed data and code. When the underlying data and analysis code are not shared, results cannot be checked, re-run, or built on, and errors go undetected.

What fixes it: open, credible science

The reform agenda is largely about changing incentives and making the process transparent by default:

Preregistration. Recording hypotheses and the full analysis plan before seeing the data removes the flexibility that p‑hacking and HARKing exploit (Center for Open Science; Nosek et al., "The preregistration revolution").

Registered Reports. Journals peer-review and accept a study on the strength of its question and method before results exist, so publication no longer depends on the outcome (COS Registered Reports).

Open data & code. Sharing the dataset and analysis pipeline lets anyone re-run and verify a result — the foundation of the FAIR data principles.

Open, credited peer review. Transparent, attributed, and increasingly paid review realigns the one step meant to catch problems — see decentralized publishing & peer review.

Funding negative and confirmatory work. Deliberately paying for replications and null results — a natural fit for the retroactive and community-directed models covered in DeSci funding.

Where DeSci — and Caper — fit