UK Reproducibility Network
  • About
  • Community
    • Steering Group
    • Local Network Leads
    • Institutional Leads
    • External Stakeholders
    • Advisory Board
    • UKRN Academy
    • International
  • Resources
    • Terms of Reference
    • Code of Conduct
    • Common Statements
    • Speaker Directory
    • Primers
    • Open Research across Disciplines
    • Presentations
    • Materials
  • Activities
    • Events
      • Open Research Calendar
    • Open Research Programme
      • Partner Institutions
      • Training
      • Evaluation
      • Sharing
      • Reward and recognition
    • Initiatives
    • News
  • Join UKRN
  • Contact
Select Page

Contents

  • 1 Open Research Programme
  • 2 Workstream 2: Evaluation
    • 2.1 Lead
    • 2.2 Aim and objectives
    • 2.3 Approach
      • 2.3.1 Evaluation of open research practices overall
      • 2.3.2 Evaluation of specific interventions
        • 2.3.2.1 Training
        • 2.3.2.2 Evaluation
        • 2.3.2.3 Sharing

Open Research Programme

A five-year programme supported by Research England, UKRN institutional members and project partners

Workstream 2: Evaluation

Lead

The lead for this workstream in 2022-23 is Malcolm Macleod, University of Edinburgh.

Aim and objectives

We will develop and use effective ways to assess the changes the programme seeks to make among both researchers and institutions.

We will do this by:

  • Developing and regularly deploying a high-quality survey of researchers’ open research practices, and their views on the support they have in adopting those. This survey serves three purposes:
    • It provides a baseline for the programme, and a regular progress measure
    • It gives participating institutions reliable insight to inform their planning
    • It informs the programme in both the training and the reward/recognition areas
  • Developing effective ways to assess the effects of specific programme interventions, notably those on training and reward/recognition

Approach

Practically, the steps we are taking are as follows:

Evaluation of open research practices overall

We have designed and piloted a survey instrument on 14 specific categories of open research practice, which covers how important the practice is, how common, and how well supported.  We are now planning how best to deploy the survey to institutions in the programme in a way that ensures the findings are as representative as possible.  The next step will be to analyse survey responses at an aggregated level and at the level of each individual institution. We will disseminate anonymised, aggregated results openly, and institution results to the relevant institution. The results will inform the training programme (workstream 1), as well as a discussion in early 2023 with professional and technical staff on how they support open research. We plan to repeat the survey, perhaps twice during the programme.

Evaluation of specific interventions

This work will begin shortly.  We will design non-survey methods for the evaluation of interventions, including evaluation of training at both individual and institutional levels.  We will put things in place for that evaluation ahead of training roll-out in 2023, working with partner institutions.  We will explore novel evaluation methods that may provide better evidence of change than those currently used.



Training

Enabling UK institutions to improve the skills in open research practices

i

Evaluation

Developing effective ways to assess changes among researchers and institutions



Sharing

Enabling and encouraging institutions to share good practice

About UKRN

UKRN is a peer-led consortium that aims to ensure the UK retains its place as a centre for world-leading research. It is led by Marcus Munafò (Bristol), Chris Chambers (Cardiff), Alexandra Collins (Imperial), Laura Fortunato (Oxford), Etienne Roesch (Reading), and Malcolm Macleod (Edinburgh).

Recent posts

  • Dorothy Bishop Prize 2023 Winners
  • Indicators of Open Research: UKRN call for priorities
  • Open Research Programme welcomes new Institutional Partner

Subscribe by email

  • Twitter
© UK Reproducibility Network. All rights reserved.