Project summary

We’re a volunteer group aiming to document and quantify the replication crisis in the social and behavioural sciences. You can see our initial output here. We plan to expand greatly this year.

Replications are essential to accumulate knowledge and accelerate scientific progress. Despite their relevance, replication studies are under-used and undercited, leading to a biased view of the scientific literature. To address this issue and promote the broader use of replication studies in research, education, and policy, we propose to (1) create and maintain a comprehensive database that catalogs replication efforts across various fields; (2) develop two user-friendly Shiny apps for finding, exploring, and visualizing replication data, enabling field-specific meta-scientific analyses; and (3) conduct outreach to encourage contributions to the database, test the apps, and teach researchers, educators, and policymakers how to best engage with our developed resources.

You can see our crowdsourced doc here and a presentation of the future of our project here. We recently expanded this into a proper machine-readable dataset and did double-entry, extracting raw stats for 1319 experiments (paper forthcoming). We are also partnering with ReD to make this the largest collection of replication studies in the world (~7000).

How will this funding be used?

• Train coders to add and validate entries to our database

• Conduct online events (hackathons) that incentivize replication entries in disciplines that are missing in our database 

• Program the educational Shiny app, democratizing access to our database and results 

• Conduct training sessions for the general public

• Maintain and curate a user-friendly platform that is accessible to the public  

• Conduct research on the moderators and causes of successful replications

• Sustain the FORRT community and its educational programmes
  

Who is on your team and what's your track record on similar projects?

An array of early career researchers, mostly psychology professors, postdocs, or PhD candidates.

Flavio Azevedo (asst. prof at Groningen), Helena Hartmann (postdoc at University Hospital Essen), Lukas Röseler, Lukas Wallrich, Max Korbmacher, Gavin Leech, Rían O Mahoney (did half of the 1300 double checks on the current dataset), Mahmoud Elsherif, Bethan Illey… and many more.

We've received awards and commendations from SIPS (one of the main reform orgs in psychology), JISC, NASA, HiddenREF, and the UKRN. We were also finalists for the Einstein Foundation Early Career Award.

What are the most likely causes and outcomes if this project fails? (premortem)

We are not likely to fail in the modest goal (of collating more and merging efforts). But downstream problems (people not using the database, a rearguard action waiting us out, waiting for the crisis to blow over as it has before) are quite likely.

What other funding are you or your project getting?

None. (Staff are salaried elsewhere but not for this work.)