The Reproducibility Crisis

Overview

The reproducibility crisis (also called the replication crisis) is the finding that a large share of published scientific results cannot be reproduced when independent researchers repeat the experiment. It spans psychology, biomedicine, economics, and beyond, and it is the deepest of the "misaligned incentive" problems that Decentralized Science sets out to address. Where decentralized publishing targets how results are disseminated and DeSci funding targets how research is paid for, the reproducibility crisis is about a prior question: whether the published record is even true.

The crisis is not primarily about fraud. It is a systems problem: the incentives that govern careers, grants, and journals reward novel, positive, statistically-significant results and quietly penalise the replications, null results, and shared data that would keep the record honest.

How deep the crisis runs

Three landmark pieces of evidence anchor the modern debate:

• Ioannidis (2005), "Why Most Published Research Findings Are False." The most-cited paper in PLoS Medicine argued from first principles that, given small studies, small effect sizes, flexible analysis, and publication bias, the majority of claimed research findings in many fields are likely false positives. The paper itself carries a formal correction, published 25 August 2022, seventeen years after the original: a set of parentheses was missing from the Table 2 equation for the false-positive cell (Research Finding = Yes, True Relationship = No) under bias. PLOS Medicine surfaces it on the article page, and Crossref records it on the 2005 DOI as an updated‑by relation.
• Reproducibility Project: Psychology (2015). The Open Science Collaboration directly replicated 100 studies from three top psychology journals. 97% of the originals had reported statistically-significant results; on replication only 36% did, and just 39% of effects were judged to have replicated – with replication effect sizes roughly half the originals (Science; project on OSF).
• Baker's Nature survey (2016). A survey of over 1,500 scientists found that more than 70% had tried and failed to reproduce another scientist's experiments, and more than half had failed to reproduce their own – yet most had never published a failed replication ("1,500 scientists lift the lid on reproducibility").

Parallel large-scale replication efforts in experimental economics and cancer biology (the Reproducibility Project: Cancer Biology) reached similar conclusions: a meaningful fraction of headline findings simply do not hold up.

Why it happens: the incentive structure

The crisis is best understood as the predictable output of the incentives researchers actually face, not as individual carelessness:

• Publish or perish. Hiring, tenure, and grants reward publication volume and novelty. Replications and null results rarely advance a career, so they rarely get done or published.
• Publication bias (the file-drawer problem). Journals preferentially publish positive, significant findings. The negative results that would balance the record stay in the drawer, inflating the apparent strength of every published effect.
• p‑hacking & researcher degrees of freedom. Flexible choices in data collection and analysis – which outliers to drop, which covariates to include, when to stop collecting – let researchers reach the p < 0.05 threshold from noise, often without conscious intent (Simmons, Nelson & Simonsohn, "False-Positive Psychology").
• HARKing. "Hypothesising After the Results are Known" – presenting a post-hoc explanation of a chance finding as if it had been predicted in advance – turns exploratory noise into a confirmatory-looking story.
• Closed data and code. When the underlying data and analysis code are not shared, results cannot be checked, re-run, or built on, and errors go undetected.

What fixes it: open, credible science

The reform agenda is largely about changing incentives and making the process transparent by default:

• Preregistration. Recording hypotheses and the full analysis plan before seeing the data removes the flexibility that p‑hacking and HARKing exploit (Center for Open Science; Nosek et al., "The preregistration revolution").
• Registered Reports. Journals peer-review and accept a study on the strength of its question and method before results exist, so publication no longer depends on the outcome (COS Registered Reports).
• Open data & code. Sharing the dataset and analysis pipeline lets anyone re-run and verify a result – the foundation of the FAIR data principles.
• Open, credited peer review. Transparent, attributed, and increasingly paid review realigns the one step meant to catch problems – see decentralized publishing & peer review.
• Amendments that travel with the citation. Correcting the record only helps if the correction reaches the reader. Crossref records a post-publication amendment on the original DOI as a machine-readable updated‑by relation – the metadata behind Crossmark – and in September 2023 the Retraction Watch database (~43,000 records, alongside Crossref's own ~14,000) was acquired and released as a public dataset through the same keyless API. The Ioannidis correction above is a worked example: one unauthenticated request against the 2005 DOI returns it. A citation that points at a publisher's article path instead of a DOI inherits none of this.
• Funding negative and confirmatory work. Deliberately paying for replications and null results – a natural fit for the retroactive and community-directed models covered in DeSci funding.

Where DeSci – and Caper – fit

Most of the reproducibility crisis is a problem of scientific method and culture, and no blockchain "solves" it. What decentralised infrastructure can change is the layer underneath the incentives: making the record permanent and the decisions transparent. On-chain timestamps let a hypothesis or preregistration be provably fixed before data collection; content-addressed storage makes data and code tamper-evident; and token incentives can be pointed at the work – replications, null results, shared datasets – that the traditional system underfunds. That is the connective tissue between this crisis and the rest of the DeSci agenda.

How Caper approaches this. Caper is a DAO protocol, not a lab or a journal – it does not run experiments or host papers. Its contribution is the funding-and-governance rail a research community would sit on. Every proposal, vote, and payout is an on-chain action with typed outcomes (a PAYOUT or INVEST proposal that clears both its ballot and its market window is the transfer), so who funded which work is a permanent public record rather than an off-chain minute.

And a caper's decisive governance weight multiplies a member's token holdings by an earned, non-transferable soulbound record of participation – a record minted by a member's own ballots, one token per ranked ballot cast and none for trading, and transferable to nobody, so influence over which research a treasury backs cannot be bought off an existing holder. Holdings still count: t is a multiplier, not a rounding error. It is a modest, honest fit: Caper supplies transparent, incentive-aligned rails; it does not adjudicate whether a given result replicates.