Last week forty researchers across six countries gathered online for the 2025 UKRN Replication Games with the Institute for Replication (I4R).
Guided by I4R’s Lenka Fiala and Abel Brodeur, eleven teams worked to replicate papers from several journals including Nature Human Behaviour, Psychological Science and European Economic Review. After a brief welcome from UKRN’s Diane Hird, Lenka explained the programme of events, and both Lenka and Abel were available to provide help and advice throughout the day. The teams worked to develop a pre-analysis plan, list discrepancies, find documentation errors, reproduce the results of the paper and summarise their process, find coding errors and fix them, and identify and perform robustness checks.
It was a very successful day, all the teams managed to replicate their papers although some had to work hard to unearth the information enabling them to do so, ‘the authors didn’t make it easy to reproduce their work’. Several papers did not include raw data or the code used to clean the raw data and teams took a long time to figure out how to interpret the data. One team successfully recreated the paper’s analysis step by step in R. As always, looking at other people’s work makes you think more carefully about your own research and research writing, ‘it was a lesson on what not to do.’
The teams now have a couple of weeks to submit their reports and any code they created to I4R who will then liaise with the authors of the original papers. I4R will also use the reports to compile meta-analysis papers.
The I4R team usually incorporate a singalong at the start of their in-person events and to accommodate the online format of these Replication Games, Lenka played videos of familiar songs with reworked with replication-themed lyrics, ‘When your p-values dance I hear a p-hackers tambourine’… maybe you had to be there.
Huge thanks to the I4R team members Lenka Fiala, Abel Brodeur and Derek Mikola for organising the event and thanks to all our Replication Game team members. See you next year!
‘It was a really pleasant day for me and our team’