UK Reproducibility Network
  • About
  • Community
    • Steering Group
    • Local Network Leads
    • Institutional Leads
    • External Stakeholders
    • Advisory Board
    • UKRN Academy
    • International
  • Resources
    • Terms of Reference
    • Code of Conduct
    • Common Statements
    • Primers
    • Open Research across Disciplines
    • Shared resources
    • Working Papers
    • Videos
  • Activities
    • Events
      • Open Research Calendar
    • Open Research Programme
      • Partner Institutions
      • Other Partners
      • Training
      • Evaluation
      • Sharing
      • Reward and recognition
    • Initiatives
    • News
  • Join UKRN
  • Contact
Select Page

Contents

  • 1 Open Research Programme
  • 2 Workstream 2: Evaluation
    • 2.1 Lead
    • 2.2 Aim and objectives
    • 2.3 Approach
      • 2.3.1 Evaluation of open research practices overall
      • 2.3.2 Evaluation of specific interventions
      • 2.3.3 Outputs
        • 2.3.3.1 Training
        • 2.3.3.2 Evaluation
        • 2.3.3.3 Sharing

Open Research Programme

A five-year programme supported by Research England, UKRN institutional members and project partners

Workstream 2: Evaluation

Lead

The lead for this workstream in 2022-23 is Malcolm Macleod, University of Edinburgh.

Aim and objectives

We will develop and use effective ways to assess the changes the programme seeks to make among both researchers and institutions.

We will do this by:

  • Developing and regularly deploying a high-quality survey of researchers’ open research practices, and their views on the support they have in adopting those. This survey serves three purposes:
    • It provides a baseline for the programme, and a regular progress measure
    • It gives participating institutions reliable insight to inform their planning
    • It informs the programme in both the training and the reward/recognition areas
  • Developing effective ways to assess the effects of specific programme interventions, notably those on training and reward/recognition

Approach

Practically, the steps we are taking are as follows:

Evaluation of open research practices overall

We have designed and piloted a survey instrument on 14 specific categories of open research practice, which covers how important the practice is, how common, and how well supported.  We are now planning how best to deploy the survey to institutions in the programme in a way that ensures the findings are as representative as possible.  The next step will be to analyse survey responses at an aggregated level and at the level of each individual institution. We will disseminate anonymised, aggregated results openly, and institution results to the relevant institution. The results will inform the training programme (workstream 1), as well as a discussion in early 2023 with professional and technical staff on how they support open research. We plan to repeat the survey, perhaps twice during the programme.  You can find the survey instrument and further information on the OTRP repository via https://doi.org/10.48420/22094216.v1 

Evaluation of specific interventions

This work will begin shortly.  We will design non-survey methods for the evaluation of interventions, including evaluation of training at both individual and institutional levels.  We will put things in place for that evaluation ahead of training roll-out in 2023, working with partner institutions.  We will explore novel evaluation methods that may provide better evidence of change than those currently used.

Outputs

WG4 Project Plan



Training

Enabling UK institutions to improve the skills in open research practices

i

Evaluation

Developing effective ways to assess changes among researchers and institutions



Sharing

Enabling and encouraging institutions to share good practice

About UKRN

UKRN is a peer-led consortium that aims to ensure the UK research system is outstanding in conducting and promoting rigorous and transparent research. It is currently led by Marcus Munafò (Bristol), Chris Chambers (Cardiff), Alexandra Collins (Imperial), Laura Fortunato (Oxford), Etienne Roesch (Reading), and Malcolm Macleod (Edinburgh).

Upcoming Events

Oct 19
1:00 pm - 3:00 pm

Contributions to research by human and AI produced lay summaries

View Calendar

Recent posts

  • UKRN Response to REF28 initial decisions and Research Environment
  • Octopus.ac One Year In
  • UKRN ORCC releases three primers on Open Research

Subscribe by email

  • Twitter
© UK Reproducibility Network. All rights reserved.