A failure of scholarship in the field of arts and health

A DCMS report asserting that ‘the arts can,’ ‘music can,’ and ‘dance can’ improve arts and health includes highly flawed studies that should not be relied upon to guide policy, says Stephen Clift.

News of a DCMS-commissioned study proclaiming the existence of strong evidence on arts and health that "can be trusted to guide policy" may be reassuring to some, but glance beneath its cover and you will find all is not as it appears. While this report includes an assessment of the ‘quality’ of the evidence on arts and health interventions, it signally fails to provide a critical review of available research.

Academic studies are organised according a standard hierarchy of evidence, with meta-analyses and systematic reviews said to provide the highest standards of evidence, followed by randomised controlled trials (RCTs) and then observational and qualitative studies. But categorising them in this way does not guarantee that the claims made provide strong evidence of validity, generalisability and impact, which can guide policy.

Earlier this year – a good three months before the appearance of this new DCMS report – I published a critical review of the recent WHO Scoping Review that underpins it, and followed that with a series of blogs elaborating on my concerns. Yet there is no evidence that any account has been taken of the substantial issues the review and blogs raised.

Here are some telling examples of studies quoted favourably in the report, which are highly questionable:

A systematic review of RCTs on dance for older people is cited, which includes at least one study which is highly flawed in failing to report results on falls. On this basis the review should have been excluded.
An RCT controlled trial on music and prejudice is cited which is not concerned with health outcomes, and has no place in an arts and health review. Even in its own terms, it does not show impacts on prejudice in any meaningful sense.
An observational study of creativity and older people is cited as showing that singing can improve health and reduce falls, even though careful analysis shows that the study is flawed. The claim regarding falls is implausible and has never been replicated.

The report consistently reifies the arts, repeatedly stating that ‘the arts can,’ ‘music can,’ ‘dance can’ without acknowledging any of the specifics of country, context, timing, quality of the arts experience, and people involved. It is claimed, for example, that ‘The evidence base on arts and physical decline’ provides ‘an overall grade of recommendation of A: the evidence on the use of the arts to reduce physical decline in older age is strong and can be trusted to guide policy development.’ But the studies on which this conclusion is based were undertaken in Brazil, China, Germany, Thailand, Turkey and the United States, to name but a few of the countries involved – all with different cultural and artistic traditions and health and social care systems. One wonders how policy in the UK can possibly be formulated on a body of evidence drawn together from all corners of the globe no matter how good the quality?

I call on all researchers, health practitioners, policy analysts and artists actively involved in the field of arts and health to critically assess the value and limitations of this report. I may be wrong in my assessment, but to me it signals a worrying failure of critical scholarship.

Stephen Clift is Professor Emeritus at Canterbury Christ Church University and Visiting Professor at York St John University

Link to Author(s):

Stephen Clift

Opinion

Health and wellbeing

Published: 05-10-2020

Comments

Mouthful replied on 08 Oct 2020 Permalink

Arts and Policy - never great bedfellows?

Thanks for opening this particular Pandora's box Stephen - I hope we can get the lid back on it! The points you raise in this post and your earlier responses to the WHO Scoping Review are uncomfortable and unsettling - as well as being valid - which is I guess what the point of critique is. The methodical way in which you’ve gone through your own area of specialist interest in the scoping review and highlighted areas of inconsistency highlights how easy it is to take things at face value rather than digging beneath the headlines. It also raises some thorny issues which I think may always arise whenever the utility of the Arts is championed in political terms. The ‘logic chain’ of causality which policy appears to need in order to be able to act does put pressure on the Arts to be able to articulate simple, clear solutions for what are really rather intractable problems of human society. I agree that we do need to be careful as a sector not to believe our own hype, and imagine the Arts as a panacea for whatever political problem the government of the day wishes to address. As we found when strapping the Arts to the social impact policy agenda twenty years ago, the issues themselves are more complex and multi-dimensional than can be explained by simple ‘input-output’ logic. At best, the Arts can rarely be more than complementary causes when addressing social, health or other inequalities. By and large, I think we’re better than we were at highlighting these complexities nowadays, and the specific ways in which the Arts might address them – thanks in no small part to research centres like your own as well as some of the authors of the report that you criticise – but it’s good to be reminded not to become complacent. It’s much better for us to spot weaknesses in our own thoughts, values and beliefs before those same weaknesses manifest in failed policy initiatives which fall short of our promises of what the Arts can do. The other problem of course with getting too cosy to policy is that it perhaps becomes less obvious to us how policy itself may be disguising more uncomfortable truths. What is it that has been driving social and health inequalities in the first place? And how can we be sure that in implementing policy, we’re not diverting the catalytic power of the Arts into supporting political systems which perpetuate structural inequalities of all kinds, when our artistic energy might be better spent imagining and bringing to life more promising and sustainable alternatives? Given that many artists in the UK will now be liberated from policy as a result of being #excluded from government support, now may be a good time to put our knowledge and skills to service in other ways. If the connections which bind Arts to policy weaken and dissolve – as they already appear to be, if the current haemorrhaging of artists from cultural institutions is anything to go by - there will be an even greater need for robust critical understanding of what the Arts are good for. Dave Camlin

jonathan Barnes replied on 10 Oct 2020 Permalink

Arts and health research or passion?

Dear Stephen thanks for opening this vital debate. This is a debate on many levels - what do we mean by research ? (Is it just systematic enquiry made public ? What do we understand by systematic ?) Should researchers always place the limitations of their study up front to avoid the temptation of over claiming? Is there a sense that RCTs may be inappropriate ways of measuring the infinitely complex nature of the human mind and body? Is there a hierarchy of research and if so, are RCTs at the top simply because they can so easily fit phenomena into boxes and spreadsheets? Is our increasing domination by figures a result of the gradual loss of spiritual/ humanistic values ? My feeling is that indeed many of the Arts and health studies I have read (and indeed completed) try to avoid fitting people into the constraints of 'normal' research, because the issues are such a complex, and untangleable mix of wordless feeling, crudely worded fact and opinion. You are right that the over-broad ( and overboard) claims of general effectiveness are indeed inappropriate and must be called out - but many arts and health evaluations whilst not pharmal-style research, serve to prepare the ground for bigger scale and repeat studies which will indeed take us forward.