Community Focus - The Celtic Neighbours approach

As the funding pot shrinks, arts organisations will need more than ever to prove their value. Meic Llewellyn, argues that its the voices of the people that really do the telling.
I know arts and culture make a contribution to health, education, to crime reduction, to strong communities, to the economy and to the nations well-being, but I dont always know how to evaluate it or describe it. We have to find a language and a way of describing its worth, commented Estelle Morris, Arts Minister, in November 2003.

Its probably true to say that evaluation is not yet a word that sets our pulses racing. To many of us, busy in the cultural activities that we love, and working every hour of the day to strengthen our communities, evaluation often seems a tedious chore, undertaken reluctantly and very often at the last minute. Get it done, tell them what they want to hear, rush it to the post and stuff all the raw material on that bottom shelf. Not surprising really, given the often mechanistic, instrumental evaluation methods and predictive outcomes that seem to over ride and even substitute for thought, image and creativity.

And yet not doing it properly is a singularly depressing and self-defeating strategy. Not only does it leave us with the feeling of a job badly done, but in accepting the lowest common denominator so unquestioningly, were also missing an important chance to draw our funders and partners much more completely into the work and its real value the interaction and interplay of human beings and the world. What we want to hear, a number of funding agencies have told us, is the voices of the people involved. Kate Braithwaite, Director of the Carnegie Rural Commission, put a similar point very well a few weeks ago, We want to hear about your failures as well as your successes, Meic, she told me. If you dont have some failures to tell us about youre not working near enough the edge!

This suggests that a more truthful and holistic approach to evaluation would be one that stimulates dialogue and inclusivity, and that we need to find the right language to capture and share the true results of our artistic endeavors the joy, confidence, positive change, connection and co-operation, independence and autonomy. We need to open up a space in which all those involved can share their stories and compare their impressions and their achievements, excitements, disappointments and understandings. And, if we are to draw in those participants whove been evaluated till the pips squeak in the past without ever seeing any results (its a bit like being disappeared, someone commented not long ago) we have to ensure that the results of our evaluation are made available in as accessible and engaging a form as possible.

Effective evaluation, therefore, is an open and continuing dialogue, and a process that can take place in any number and combination of forms depending on the stage of the project, participants, artforms and circumstances. It works most effectively through a network of people and resources that are identified within a project from day one, and which then quietly inform the production process throughout the project, thereby ensuring that things are working, and that they keep working: feedback and constructive criticism received half way through a project offer an opportunity to remedy and adapt things; evaluation compiled at the end can be fed into the development of new projects. And as this continuous checking, learning and adapting process gathers pace, so the production group, and all those in the community supporting it, gain skills and confidence.

Being the lead partner in Celtic Neighbours, an action research project linking cultural organisations in Welsh-speaking Wales, the Irish Gaeltacht and the Hebrides in Scotland, has presented Voluntary Arts Wales with the ideal opportunity to create just such an inclusive and engaging approach to evaluation. Experience has shown us that the losses to everyone artists, communities, and funders are huge if we accept a limited and lifeless approach to the evaluation process. Every participant and partner has a story to tell, experiences to re-live, eyes to see and ears to hear. It is vital that these stories the interpretations that give the art its meaning can be told and are then made accessible to everyone.

Our toolkit is now filled with friendly, fun, quick to apply, easy to learn and interesting qualitative evaluation methods, all of which rely on the conversation and dialogue of the people involved, and the words and language they use themselves to convey meaning and detail. It uses a wide variety of media, including verbal, visual, aural, individual and collaborative techniques, and includes unexpected elements such as sensory shapes, mindmaps, a journey to now and philosophical worms. Whilst not rocket science, the toolkit has been specifically created to be as adaptable as possible so as to become the property of all participants. After all, every enterprise is different, and all good projects need to allow people to devise new patterns of measurement and reporting in order to capture the essence of their work.

A good example of a tool we use is an archery target that encourages straightforward, gut-feeling responses, and attracts the sort of people who might never write a comment in a visitors book or who would discard a long questionnaire. It can also be a surprisingly sophisticated tool divide your target into a number of segments, and you can evoke a graded pattern of responses to different elements of any experience. Another is can I have a word? a questionnaire that overcomes fear and loathing by simply asking for a one word response (two if you really have to&) to five or six key questions. Simple, friendly, requiring minimal time to complete and helpful in engaging people who might turn away from a crowded page of tick-boxes. And sensory shapes a pool of coloured shapes: light bulbs, hearts, thunderclouds, question marks, parcels liberates responses from people who may not see themselves as particularly articulate. Yes, we do ask people to write us reports, but we also take photographs, give out disposable cameras, draw cartoons and write songs. Our long-standing challenge is a string quartet!

Additionally, and vitally, these tools enable us, the process designers, to ask the most important question of all: what is it we really, this time, want to know? In this way the potential for a mismatch between the information required and the priorities and criteria of our funders and partners is short-circuited. Everyone wins!

The arts offer an interesting journey in the search to find and communicate our experiences of life. Participants and audiences are expected to re-examine and reflect on the often mysterious elements concerned with human emotion and human spirit the things that give life its meaning but which arent easily, if ever, replicable or simple to measure. But if we employ the right tools, those that fit the purpose, the evaluation process can be a very powerful way of sharing ideas, experience, problems and power. When inclusive and creative, and developed collaboratively, evaluation can lead to a more dynamic, informed and equal relationship between all those taking part in any undertaking. So, lets lift evaluation out of its grey straightjacket and metamorphose it into something vibrant, proactive and empowering. And fun! Come on - how about that string quartet?