Gorilla Experiment Builder is part of the UKRN Stakeholder Engagement Group. Gorilla’s CEO, Jo Evershed, was recently named one of Computer Weekly’s Rising Star Women in Tech. We interview Jo about Gorilla, its place in the open research landscape, and the benefits of working with UKRN.
Jo Evershed is the Founder CEO of Gorilla Experiment Builder, a powerful, flexible and intuitive platform for running behavioural research online that allows researchers to go far beyond surveys. She also convenes and hosts the Behavioural Science Online Conference. Connect with Jo on Twitter, LinkedIn or BlueSky.
Congratulations on being named one of Computer Weekly’s Rising Star Women in Tech! What does this recognition mean to you?
I’ve been leading the team at Gorilla Experiment Builder for 8 years now; we serve over 1000 universities worldwide, whose researchers have collected data from well over 1 million participants.
In the lab, collecting data used to take months. With Gorilla, it can now be done in hours or days. The team and I are incredibly proud of the impact we have had on researchers’ lives and how we’ve transformed the experiment building and data gathering landscape.
The recognition from Computer Weekly is humbling, but also invigorating for me and the team. As I’m sure you can imagine, it takes constant research, development and innovation to make cutting edge research software. When the COVID pandemic hit we were a team of just 6, and we found ourselves suddenly supporting a huge number of researchers moving online. The COVID years were exhilarating, but also exhausting. The award highlights the importance of leveraging technology to investigate human behaviour, and motivates us to go further in our quest to transform the behavioural science industry.
As a team, we are setting our sights on our next goals, and this recognition serves as a very welcome pat on the back that we are on the right track and are doing a good job! Ultimately we hope that, with better data, we humans can redesign our health, wealth, and education systems to be more human-centric, based on sound and rigorous evidence.
For me personally it’s also a validation of a decision I made to go back to university as a mature student, and create my dream job. It’s been far harder than I could ever have imagined, but also far more rewarding. I’ve had to learn, reflect, and grow a lot, and there’s still a long way to go! But I’m proud of my past self for making the leap and putting in the work.
Can you tell us what the Gorilla Experiment Builder is, and how it came about?
Gorilla is a browser based platform for creating and running behavioural studies online. In a nutshell, participants go in and gorgeous behavioural data comes out. Our tools allow you to create surveys, reaction time tasks, navigation tasks, games, shops, multiplayer tasks and more – and then combine them all together into a controlled experiment, all without needing to code!
It was my own struggles collecting data at UCL that inspired me to create Gorilla. When I was a student, we had to learn MATLAB, which was fun, but didn’t seem like a great use of time. Then – to overcome the challenges of slow data collecting in the lab – I convinced the director of the Science Museum to let me collect data from visitors. In just 3 weeks I had collected data from 200 parent-child dyads. It was exhilarating!
After leaving UCL, I was inspired to create Gorilla so that other behavioural researchers could create behavioural protocols quickly and easily and then collect data at scale online. Ultimately, leaving researchers with more time for science and discovery. Our guiding principles have been to make our tools “Easy enough for students and powerful enough for professionals.”
Why ‘Gorilla’? It all began when I casually remarked to UCL’s Prof Daniel Richardson, “So, you’re envisioning a bigger and better Survey Monkey?” Without skipping a beat, Dan enthusiastically replied, “Exactly! I want a Survey Gorilla!” And just like that, the name stuck. He’s gone on to win awards for his innovating teaching of research methods using Gorilla.
“What we’ve managed to do with Gorilla is give students the tools and templates to make their own experiments, with minimal supervision. Students can go out and test a hypothesis via social media. Results flood in from all over the world, and they’re creating this incredible range of studies. It’s a much richer learning experience.” Professor Daniel Richardson, UCL
So, there you have it – Gorilla, breaking barriers and paving the way for hassle-free, scalable data collection, all while keeping the spirit of scientific exploration alive!
How does Gorilla support Reproducibility and Open Research?
Supporting open and reproducible workflows is a core design principle at Gorilla. We do this in several ways including collaboration features, version control features, open materials and our upcoming data studio.
Gorilla is Open Access software (see more in the next section) which means anyone can sign up and use our experiment design tools for free. This supports collaboration across teams and universities, as everyone working on the project can have an account at no cost. On top of this, our built in version control with commit notes means that you can see who has worked on what and what needs doing next. You can think of it as providing the benefits of Github, but without the learning curve.
If you’re doing a pre-registration or Registered Report, you can use Gorilla’s Open Materials Repository to share your methods with ease (publicly or privately via link-only access) before you’ve even collected the data. This allows those reviewing your report to preview and inspect your protocol in detail at the click of a button.
Once you’ve finished your experiment and you’re ready to publish or post a preprint, you can update or create an Open Materials page and share your exact experiment as part of your project’s research record. Link to it directly from your paper, and also your Open Science Framework project page! This means that anyone reading your paper can preview, inspect and clone your protocol in just a few clicks. This ensures protocols don’t get lost at the back of a drawer and removes the barrier of having to ask for the configuration files, together this makes it so makes it much easier to run replications and extensions.
There’s more to come though! Data Studio is due to be released in the next few months. With Data Studio, you can pre-process your raw data directly within Gorilla and keep track of the decisions you made along the way to help keep your data analysis process transparent. Future You will be grateful to know exactly what Past You did!
Our 2024 roadmap includes further improvements to Data Studio and Open Materials, so that it’s even easier for users to do reproducible and open research.
Why did you choose to make your software Open Access and not Open Source?
The challenges associated with Open Source software have been well documented. For instance:
- Lack of funding
- Limited documentation
- Usability (UI/UX) challenges
- Security concerns
- Sustainability
Together these often (but not always) lead to feature-limited and hard-to-use software that unfortunately ‘dies’ when the original creator can no longer maintain it.
The legacy I wanted to leave the behavioural science industry was self-sustaining, powerful, easy to use, well supported, secure software that behavioural researchers could rely on. I wanted to make it easy enough for students – which meant overcoming usability challenges – and powerful enough for researchers – which meant hiring world class developers. And secure enough for storing identifiable medical data, which meant funding for expensive lawyers and IT security professionals.
I didn’t have faith that I could achieve this aim with an Open Source model. Plenty of people had tried. They are all talented and knowledgeable people and had made impressive systems, but I wanted to go further. So I decided to do it differently, because when something isn’t working, you don’t just try the same thing again. You run a different experiment. And that’s why I chose a closed source model.
Nevertheless, I am hugely keen on Open Science, so I wanted to remove as many barriers as possible. That’s why our tools are Open Access. It means any researcher anywhere in the world can sign up to Gorilla for free forever and collaborate with any other researcher anywhere else in the world. You can use all our experiment design tools without paying any money, you can even test your protocol and see your data locally. All at zero cost.
I believe this gives our users the best of both worlds.
Accessible – Enabling worldwide science collaboration with free accounts.
Powerful – So that science isn’t constrained by technical capacity
Easy-to-use – So you save time and can focus on the science, not the programming
Supported – So you’re never lost or alone
Which neatly spells APES! Just what you’d expect from a Gorilla team 🙂
There’s an assumption that Open Source means you can add your own customisation and Closed Source means you can’t. But that’s an over-simplification. Open Access software can be extensible and customisable too. See the video at the end of the next section where we demonstrate creating new components and then using them through the normal user interface!
What do you think is important when creating research infrastructure software?
In order to establish a dependable platform for behavioural experiments, I have thought deeply about how to create successful research infrastructure.
The first insight is that for research infrastructure software – that can be used globally – longevity is key. Why the global scope? Because that’s how you get the greatest economies of scale and how you make the best use of precious public funding.
So what is necessary for longevity? Longevity requires a self-sustaining funding source. Historically, a lot of research infrastructure has been funded by grants or funded with academics time (and salaries). Unfortunately, grants run out or academics move to a new role and then infrastructure projects die.
So how can you get a self-sustaining funding source? That’s where a subscription model works well. If all users pay a fair and transparent subscription fee, according to the value they derive, then that creates a permanent funding source. This funding can then be used to pay for development, maintenance and support, and all the other costs associated with research infrastructure. And that’s what we’ve done at Gorilla, and that’s one reason why it’s been so successful.
The second insight is the need to balance flexibility and standardisation. One challenge with cutting edge science is that many researchers are testing novel research paradigms; this requires flexibility. And yet it’s essential that the underlying infrastructure supporting these paradigms are validated and trusted; this requires standardisation. The way we’ve solved this problem at Gorilla is to create an extremely powerful component system where researchers can create an astonishingly wide range of paradigms out of standardised components that we’ve tested and validated.
On top of that, with our scripting tools we’ve given users the ability to augment Gorilla by combining novel code with our standardised and tested components to create entirely new effects. This gives Research Software Engineers the best of both worlds – the power, usability and validation of infrastructure software, with the extensibility of writing code.
What are the benefits of being part of UKRN’s Stakeholder Engagement Group, and working with the UKRN community?
At Gorilla we don’t believe in waiting for change to happen: we want to be in the conversation and helping to make that change happen. UKRN harmonises those efforts and connects like-minded people and organisations together.
Our product manager – Dr Jade Pickering – has been part of several UKRN activities, particularly during her PhD. She was instrumental in setting up the Open Research Working Group at her alma mater (University of Manchester), was on the global committee for the ReproducibiliTea organisation which is closely linked to the UKRN, and submitted evidence to the UK Parliament’s inquiry on reproducibility and research integrity. She even represented her university at the United Nations to discuss open science’s role in achieving the UN’s Sustainable Development Goals.
So, with the support of communities such as the UKRN, she’s worked to influence policy at the university, national, and international level – and she brings those principles and passion to Gorilla. Without the UKRN, her journey wouldn’t have been possible. Now, at Gorilla, the UKRN enables her to continue that journey from a different perspective and enables us to build self-sustaining research infrastructure that advances the research landscape.
Our close relationship with the UKRN allows us to benefit from the wider insights of the research community on how better software infrastructure can support researchers to do the best science they can. We can then build workflows into our products that make doing the right thing not only possible, but also easy – for example, allowing our users to create an Open Materials repository to share their protocol in a few clicks. So that more researchers can do reproducible science, even when they are busy.
That’s why we hugely appreciate our relationship with the UKRN community. It ensures Gorilla is not only powerful and easy to use by the initial researchers, but also for those who build on the science that’s come before.