An impact assessment playbook

Europeana is a digital platform for Europe’s cultural heritage with the ambitious aim of transforming the world with culture. In practice, this means that we work with thousands of European archives, libraries and museums to share their cultural heritage for fun, education and research. And as we enter the European Year of Cultural Heritage, we have been grappling with a number of questions around the theme of impact.

We wanted to create a common language for the cultural (heritage) sector to use when talking about impact

Did our interventions improve the way our children are educated? Did they result in a stronger, more cohesive society? Did they enable artists to create ground-breaking new works? What positive change in society have we contributed to? In short, what is our impact?

A manageable process

Undertaking an impact assessment can be a daunting and complex task, so we started a project to make that process more manageable. We wanted to create a common language for the cultural (heritage) sector to use when talking about impact, so that we can all prove, understand and refine our contribution to the wellbeing of society. To achieve this, we agreed on the following design principles:

• The method should be easy to follow but tailored to our sector and equally accessible for small and large institutions.
• It should contain case studies that demonstrate how impact assessment can bring new insights into our work.
• It should be co-created by our community and the conversation should be open for anyone with an interest in the topic to join.

The methodology, at its core, is based around academic research by Simon Tanner of King’s College, London called the Balanced Value Impact Model. We turned that model into a practical playbook, a script that anyone with basic group facilitation skills could use. We worked with a team of cultural heritage professionals and impact designers at every step of the way, applying the methodology to our own projects as well as others.

The process we followed enabled us to validate our work as we progressed. We tested each step in the method on our stakeholders and, as we ran workshops, we observed what worked well and where we needed to make adjustments and tweaks. And then we ran them all over again, learning at each step.

One of the insights we gained was that we should not look at impact assessments as a nice report you write at the end of a project. Rather we should look at them as a continuous process that helps us to refine and improve our impact, iteratively. In our playbook we distinguish four interrelated phases: (i) design, (ii) assessment, (iii) narration and (iv) evaluation.

The starting point is to get your principal stakeholders aligned on the metrics that best indicate the impact relevant for your project. This sounds rather simple, and it is, provided that you can manage the conversation in a structured way. This is exactly what the playbook is designed for. If you do this part well you will get everyone to buy into the concept and achieve much better results from it.

Testing it out

We tested this method on our own work. Being an organisation with a clear European remit, we decided to investigate our contribution to a shared European identity through our First World War project, Europeana 1914-1918. Not an easy assignment.

The results of our survey showed us that participants in our project indeed experienced a strong sense of community. It really did not matter much what side of the trenches you fought during the war: the significance of the war for the participants in our project was that it created a sense of shared experience that contributed to a sense of community in Europe – identity if you will.

We were very pleased to be able to conclude this, but we also gained other valuable insights. For example, that users did not learn as much as they hoped from their experience with our service. This understanding led us to significantly modify our project, which will hopefully increase our relevance to these communities in the future.

Start now

You can start designing your impact assessments right now using the playbook. In the meantime, we will continue developing the methodology by adding new sections that will help you to collect more relevant data, make more sense of it and tell your story in a more engaging way.

Harry Verwayen is Deputy Director of Europeana.
pro.europeana.eu