Research - The problem with research?

Let's face it, audience research is always a bit of a problem for arts organisations. Why? Because the whole science of research is based on the principle that if we select a few representative individuals to answer our questions, their answers will give us a strong indication as to how all the rest of our audiences and visitors feel. But this process of selection poses a series of tricky dilemmas for arts researchers.

No such thing as average

Who exactly are 'representative' arts attenders, and how can they be identified? Unlike retail stores or consumer goods manufacturers, theatres, galleries, arts centres and theatre production companies deliver ever-changing series of artistic products to their visitors and audiences. It is the nature of the artistic product at a particular point in time that will largely determine the nature of the audience; so a large-scale receiving venue, which in any given week might be staging a combination of Jo Brand, Motörhead, Postman Pat, Romeo and Juliet and the Moscow Philharmonic Orchestra, would have its work cut out to identify all those comedy-loving heavy metal feminists with pre-school children and a penchant for the classics. Sensibly, they don't generally even try. Less extreme programming also poses problems, though. To what extent is the half-term visitor at a museum representative of the year-round audience; or the 'August' visitor to a gallery in Edinburgh; or the matinee attender at a repertory theatre? Each has its own peculiarities, and there's no point trying to pretend that anyone in each of those categories represents the 'average attender'. 

Box office statistics

So what are the current options for arts organisations that want to know how well they are serving their many and varied customers? Well, they can look at their box office data (assuming they have a box office system - sorry touring companies). This tells them a lot about what people do - how many tickets they buy for what; where they sit; how often they come back (most of them not very often); whether they booked for a group, etc. But what it doesn't tell them is probably far more important for marketing purposes: for example, why people booked their tickets; where they heard about the event; what they thought of the experience (the venue or the art); how old they are; what their ethnic origin is, etc. So, box office data provides an important but incomplete solution to an arts marketer's data needs.

Inadequate methodologies

That leaves us with surveys and focus groups. Let's deal with focus groups first. Beloved by arts organisations everywhere, focus groups were never intended to paint an accurate picture of a situation, but rather to provide an intuitive understanding of it. They are great for revealing the in-depth motivations and feelings of a small number of people, but hopeless as a basis for making decisions that will have financial consequences, as they provide no indication of the extent to which these motivations and feelings are shared by audiences or visitors as a whole. Surveys, on the other hand, are scientific instruments that are specifically intended to reflect the views of entire audiences. By asking the right questions of sufficient quantities of the right people, statisticians assure us that we can confidently predict how everyone thinks and feels about an issue. The devil is in the detail, though. Because of the diversity of audiences, described earlier, it is generally only possible for surveys to capture representative information from small pockets of audiences or attenders. Building up a bigger picture is extraordinarily expensive. You have to ask questions of an awful lot of people to be sure that a full range of views is captured.

Benchmarking revelations

Finally, let's suppose, having done an all-singing, all-dancing, methodologically rigorous survey comprising many thousands of people over a long period of time, an arts organisation finds that two-thirds of its attenders described their experience of that organisation as being 'good or very good'. Just how good is good? Should that organisation crack open the champagne and give itself a big pat on the back, or should it be putting on its sackcloth and ashes, on the grounds that a third of the audience didn't think much of it? A scenario similar to this arose a few years ago when a client (who must remain nameless, for reasons that will become apparent) actually conducted such a survey and found that 75% of its customers had given it a 'good or very good' rating. They chose the champagne route. What they didn?t know was that findings from an almost identical survey conducted by a similar organisation just 12 months before, had recorded a 95% 'good or very good' rating from its customers. It's only in this global context that the first organisation would know to stock up on sackcloth and cancel the order with the wine merchant. So, in summary, box office data is limited in its scope; small-sample ad hoc surveys are poor predictors of the views of whole audiences; focus groups aren't representative enough; and research only becomes really useful when its findings can be considered in a wider context. What arts marketers really need is access to continuous research amongst all their customers, linked in with the information from their box office records. Watch this space.

Stuart Nicolle is a Director of Purple Seven, and Liz Hill is a Director of Arts Intelligence.

Purple Seven and Arts Intelligence are currently piloting a new audience research system that will link email survey data with box office data to provide arts venues and touring companies with online access to a wide range of continuous audience feedback. To register your interest in this developing project, e: emma@artsintelligence.co.uk