How have arts organisations been able to gauge whether their New Audiences projects have succeeded? How do they define their success, and how is it measured? In a world of targets, policies, aims and objectives, it can be as difficult to recognise success as to identify failure.
New Audiences was committed to testing and researching approaches to audience development through practical projects. In such a context, success can be perceived on several levels: artistic success, positive outcomes for participants and audience members, whether goals and targets were achieved and what unexpected outcomes have arisen. If the project failed artistically, was a negative experience for participants and achieved none of its goals, it might still be said to have succeeded in testing an approach to audience development – and finding it a failure. Even in a world where it is important to appear successful in the eyes of funders, instructive failure should be recognised as a possible outcome of a risk-taking approach.
Defining success
Although many evaluations of New Audiences projects identify the factors which contributed to success, the way in which different organisations define and articulate success seems more problematic. Hilary Smallwood was the outreach worker for Cre8, a project with young people aged 11-25 in Ross-on-Wye. She recorded what had happened and what had been said at each event, together with her personal impressions. “That’s how you could see it was actually working,” she says. “To some extent it is subjective, and I don’t know how you get away from that.” She also found integrity important. “There are those who would say that you yourself want the project to be a success, and you have a personal vested interest in showing that it went well.”
Organisations must also accept that ‘success’, as it might be defined by day-to-day norms, is not a given in the context of action research. Converting results into a useful learning experience requires time and space for reflection: a qualified success on one level can still provide valuable information on another. Conversely, a project that achieves surface success may have been ineffective in developing the audience.
Setting the targets against which the project’s achievement will be measured is also a challenge. Particularly in the early stages of New Audiences, managers found it difficult to visualise how ‘success’ might manifest itself. They were not initially able to compare their targets, nor indeed their results, with those of other projects. This is now possible to a greater extent through the New Audiences website.
Success indicators
A range of indicators showing how managers view success is emerging through research commissioned by Arts Council England. These tend to fall into the following categories:
Audiences, including
• attracting a large number of people
• reaching a specific target group of people
Organisations, including
• organisational change
• making an impact on programming
Finance, including
• increasing work for the organisations
• success in levering new money
Marketing, including
• creation of a database of new attenders
• discovering a new communication medium
Professional relationships, including
• creation of new partnerships
• cementing existing relationships
Contributing to national policy, including
• ability to share or disseminate knowledge
• gaining recognition for an overlooked area.
Data and culture
Issues also arise over ‘soft’ and ‘hard’ data, and whether one is more valuable or indeed more valid than another. Hard statistics – box office data, participation rates, response rates to ticket offers, etc – are more easily collected. Gathering evidence of people’s personal responses to the quality of an experience is time-consuming and requires both a different set of skills and a different attitude to the results. Favourable qualitative responses could be used to mask a lack of success in gaining quantifiable targets – but the reverse could also be true. To give a thousand people a dull, off-putting experience is probably less cost-effective in the long run than giving twenty people a wonderful time. The idea that subjective observations are not valid evidence needs to be re-examined.
Sarah Bedell, an evaluator for New Audiences, says that the qualitative/ quantitative divide is a reality. “People think that if results aren’t expressed as numbers, figures or ticket yields, then it’s woolly and it will not stand up. Marketing people in particular are driven by quantitative targets. It’s that balance of the financial, the social and the artistic.” Evaluator Helen Jermyn has pointed out that, “artists, when asked how they judged the success of a project, often referred to the small things they had witnessed which they felt were evidence of a successful outcome.”
Long-range success
Success often extends beyond the timescale of the original project, laying foundations for more sustainable work, influencing future plans or building on a legacy such as the creation of a new post. Many organisations have used New Audiences as a springboard to further activity and development. Arts Intelligence has been carrying out follow-up research on a range of New Audiences-funded organisations, and has uncovered many instances of post-project success which could not have been measured within the lifetime of the original initiative.
The New Audiences Programme as a whole allows arts organisations to examine the concepts of success: setting targets to measure it; finding factors that promote it; defining and recognising it; and translating current successes into future achievement.
Beyond the Page
For further information, including a more detailed list of success criteria and downloadable reports, go to http://www.newaudiences.org.uk and click on Essential Audiences.
Feedback to Essential Audiences can be sent to audiences@artsprofessional.co.uk
Essential Audiences is compiled and written by Catherine Rose. For more information about the New Audiences Programme, contact Arts Council England, 14 Great Peter Street, London, SW1P 3NQ. t: 020 7973 6497; f: 020 7973 6791; e: newaudiences@artscouncil.org.uk; textphone: 020 7973 6564