Here are details of an initiative that I presented during the recent LNL retreat at LSE. If you would like more information please contact me.
One of the main reasons that researchers give for not adopting open research practices is that they don’t have the time for an extra process and don’t understand where it would fit in the lifecycle of a research project. To the first part of that reason: fair enough. Time stretched too thin will be a familiar sounding story to many academics. The second could potentially be resolved by pegging open research practices to steps that are already familiar. One way that we might be able to do both at once is to integrate study preregistration into existing research ethics processes.
We recently implemented this as a pilot project for student dissertations in the Department of Psychology at Edge Hill University. Our final year and taught MSc programmes operate an internal research ethics process in line with guidance from the British Psychological Society. Students complete a standardised ethics form, which is reviewed by their supervisor and another researcher independent of the project. The ethics application is an important pedagogical step for the student and, ideally, a stimulus to kickstart project planning.
What we tried
Our approach was to reorganise and streamline our existing ethics forms to collect questions about methodology and reframe them following the format of an osf.io standard study preregistration. Really, that should be plenty of information for ethics reviewers to understand what you intend to do to your human participants and their data. Because our department sees quantiative and qualitative dissertation projects in roughly equal proportion we organised the form so that it should work just as well for preregistering projects of either type. Because this pilot is designed for student projects, we also include lots of specific instructions on what each prompt should document. You can find all of our materials here. Having completed a research ethics form in this format one should be able to copy/paste their way to 90% of a completed study preregistration!
What we didn’t try
There are a handful of other queries that you might expect to see in a study preregistration that aren’t strictly necessary for an ethics form. I am aware of some programmes that have gone full hog on making research ethics identical with preregistration. That sounds like it is probably more hardcore than what we’ve attempted here. One misgiving that I have about that approach is that it could add a little more work where it isn’t strictly needed. I am more interested in building efficiencies as a way to reduce barriers to open research practices than making them mandatory. For the purposes of this pilot, where there is a need to scale across large cohorts of students and the exercise is mainly pedagogical, this seemed particularly prudent. I might yet be convinced that a version of this implemented for professional research ethics could go further, but you’ll have to knock me off my efficiency high horse first. If you notice a problem I haven’t or have an idea to improve this process, please let me know!
Thoughts on implementing it yourself
If you don’t have responsibility for your local dissertation module (which I didn’t) you’ll need cooperation from someone who does (which I did, hat tip to Dr. Alex Balani for his work on this too). It might also be a good idea to seek input from someone intimately familiar with the predilections of any accrediting board or institutional boards that are likely to take an interest (hat tip to Prof. Derek Heim for his invaluable mentorship).
What if they just don’t wanna? I’ve encountered a few arguments that might help reluctant colleagues to see that even if they aren’t persuaded to try this for the intrinsic value of improving research integrity, there may be extrinsic reasons to do it anyway.
The first reason is to evidence pedagogical best practices. We happened to have a suite of programme reaccreditations following directly after this pilot and were able to include this initiative as evidence of our programmes’ innovative approaches to teaching and learning. If your reluctant colleagues haven’t innovated much lately themselves, perhaps they might think again about your offer of an easy win?
The second reason is to strengthen your local research Environment. And I mean that with a capital E as in the People, Culture, Environment portion of the Research Excellence Framework (REF) that will see a larger than usual share of how university research is evaluated weighted to how institutions are organised to support said excellence. How exactly that will be measured is not yet set in stone, but one of the candidate indicators that the REF have published is assessment level mechanisms for supporting the reproducibility of research (hat tip to University of Lancaster’s LNL Tom Morley for pointing this out). Somewhere in or nearby your department there is probably a research lead who should have this sort of thing on their mind. Maybe this person could be a good ally?
The sneaky part (shhhh! Don’t tell my colleagues!)
Alongside the changes to the main ethics form that constitute the substance of this initiative, we also introduced a handful of other time saving measures. These included things like more concise participant documents (which are easier to write and review), a lot of very very explicit guidance for our student researchers, and a bit of admin busting automation to shave time off everyone’s workload. These don’t have much to do with open research per se, but the point was to mitigate the perception that open research takes too much time. No harm in confounding that experience with a little extra time savings, eh?
Deploying to institutional research ethics
I have aspirations to build this process out for wider implementation at my institution. There is still a good distance to travel yet, but early discussions have been promising (perhaps an advantage of a smaller and nimbler university?). At some point this will want consultation with researchers from a wider range of epistemologies to make sure it is fit for purpose for all use cases. For example, discussing this proposal at the recent UKRN LNL retreat it was pointed out that it would need to be adapted to accommodate secondary data analysis style preregistrations. I imagine there may be other variants to consider as well, and I also imagine that this should be doable. Wish me luck?
LNL Michel Belyk, Edge Hill University michelbelyk.com