Scottish Government

Main Findings

Social Impact

• Participation in cultural and sporting activities has been shown to result in the gaining of new skills, improve informal and formal learning, increase self-confidence, self-esteem and a feeling of self-worth, improve or create social networks, enhance quality of life, promote social cohesion, personal and community empowerment, and improve personal and local image, identity and a sense of well-being.
• Although no research has demonstrated a causal relationship between participation in arts, culture and sport activities and a reduction in offending behaviour, national and international research and evaluation has demonstrated a link between the two.
• Research (including clinical research) has shown that participation in cultural or sporting activities (including physical activity) has led to improved physical and mental health (eg reduced stress levels, reduction in anxiety and blood pressure, reduction in visits to GP etc). Non clinical outcomes have included improved communication skills, new skills and confidence, improved interpersonal skills and increased social networks and enhanced sense of well-being.
• Robust and longitudinal research studies have shown that there is an association between cultural possessions in the home/culture in family background and educational performance; that there is a link between cultural participation and increased literacy and that participation in music and visual arts is linked to being above average in reading, maths and behaviour; and that sports and arts schools tend to perform better over time and at a given point in time than maintained schools.
• Various studies have shown that certain groups are under-represented in participation in culture, the arts and sport - low socio-economic groups; young people with low educational attainment; disabled and mobility impaired people (particularly those living in rural areas); ethnic minority groups; young males post-education (in the arts); teenage girls (in sport); and school children and teenagers (visiting heritage organisations).

Economic Impact

• Various economic appraisals and evaluations of the establishment of a major Scottish arts centre, theatre in London, the cultural industries such as museums, libraries, galleries and exhibitions, and major cultural events have demonstrated direct and indirect economic impacts on local areas and beyond.
• Economic evaluations of sporting events have demonstrated an economic impact on the local area by way of additional expenditure, the generation of additional employment, and the potential for long term economic gains, particularly when a major sporting event has been the catalyst for regeneration of an area.

Introduction

The recent Partnership Agreement for a Better Scotland (PABS, May 2003) made several high level commitments to promote cultural life, including sport, as "inclusive and accessible", where participation brings real benefits to communities and individuals. PABS also commits to maximising the contribution that sport, culture and the arts can play in promoting Scotland and in attracting international events.

However, at present there is no strategic overview of the evidence base that indicates that sport, culture and the arts do indeed bring real economic and/or social benefits to society. This Research Findings paper summarises a Literature Review that draws together recent research findings that provide firm, robust evidence on the links between participation in culture and sport and the social and economic impact on communities and individuals. This is particularly important in light of further investment to be made in these areas, ensuring that good value for money is obtained in supporting the government's key objectives.

The Review is a first step in developing a longer term research framework to inform policy development, implementation and evaluation in the areas of culture and sport. By providing a coherent social research evidence mapping, it highlights gaps in research and will underpin future collaborative efforts between the various cultural and sport agencies to provide robust evidence on which to base cultural and sport policy.

Recent key findings therefore, on the social and economic impact of culture and sport, including various major events that have taken place both in Scotland and elsewhere, are drawn together in this Review. The Review also covers information set out in policy documents relevant to the PABS priorities, and in corporate strategies and action plans of the different agencies, plus conference proceedings on relevant topics.

The Review is not comprehensive or exhaustive as there is a wealth of research available. However, it focuses on the main or most relevant research and evaluations that have been carried out in recent years, that relate to the National Cultural and Sport Strategies to increase participation of socially excluded groups, youth, ethnic minorities and disabled people, and that improve health, personal development, educational attainment and well-being.

A literature search of various recent research studies and evaluations (most from about 1998) found the following:

Social Impact

Personal and Community Development

Participation in cultural and sporting activities has been shown to result in the gaining of new skills, improve informal and formal learning, increase self-confidence, self-esteem and a feeling of self-worth, improve or create social networks, enhance quality of life, promote social cohesion, personal and community empowerment, improve personal and local image, identity and a sense of well-being.

For young people in particular, participation can reduce truancy/bad behaviour at school, reduce the propensity to offend and lead to better educational/employment prospects. For ethnic minority groups, all of the above personal and social aspects can occur, and in addition participation in cultural activities relating to their own culture can result in an enhanced sense of pride in, and 'empowerment' of, the ethnic community; and for disabled people, participation can reduce isolation, increase social networks and enhance quality of life.

Social Justice

Although no research has demonstrated a causal effect between participation in arts, culture and sport activities and a reduction in offending behaviour, national and international research and evaluation has demonstrated a link between the two. Where sport and arts activities have been targeted at young people at risk, or actual offenders, there has occurred a significant reduction in crime figures in these areas and a reduction in re-offending. Programmes of activities can not only create a 'diversion' from criminal behaviour, but can facilitate key skills in learning, develop personal and social skills and provide routes into further education and employment, all of which can lead to the propensity not to re-offend or participate in criminal behaviour.

Health

Research (including clinical research) has shown that participation in cultural or sporting activities (including physical activity) has led to improved physical and mental health (eg reduced stress levels, reduction in anxiety and blood pressure, reduction in visits to GP etc). Clinical, hospital based research has provided hard, undisputed facts on the improvement of health. However, 'softer' more qualitative outcomes have been shown to include improved communication skills in those with special needs; 'carers' having developed new skills and confidence; and improved interpersonal skills and increased social networks having led to an improved sense of well-being amongst the target population.

Education

Robust and longitudinal research studies have shown that there is an association between cultural possessions in the home/culture in family background and educational performance; that there is a link between cultural participation and increased literacy and that participation in music and visual arts is linked to being above average in reading, maths and behaviour; and it is believed by educators that arts activities and creativity in education have a positive educational impact on the majority of pupils.

Research has also demonstrated that sports and arts schools perform better over time and at a given point in time than maintained schools. Participation in arts education can lead to not only the development of arts and knowledge skills, but to increased confidence and the development of communication skills, an understanding of diversity and transferable skills for future employment.

Participation and Under-represented Groups

Various quantitative research projects have provided information on the level and extent of participation by the general population in culture, the arts and sport. Such studies have shown that certain groups are under-represented in participation in every area - low socio-economic groups, young people with low educational attainment, disabled and mobility impaired people (eg the elderly in particular and those living in rural areas), ethnic minority groups, young males post-education (in the arts), teenage girls (in sport) and school children and teenagers (visiting heritage organisations).

Barriers to participation

Research has investigated the reasons for under-representation of these groups and has found that the main, common barriers are: lack of time and money (particularly the cost issue for families); availability and location; lack of, or inaccessible, public transport (particularly for those living in rural or peripheral areas); lack of information, lack of understanding and perception that the activity is 'not for them'; irrelevance to their everyday lives; and 'management ethos'. In addition to the above, for ethnic minority groups there are barriers such as lack of diversity in representation of the arts and in sport; lack of their own culture portrayed; language barriers; fear of racism; and social constraints. For disabled and mobility impaired people, lack of appropriate access to and at facilities; lack of programmes geared to their needs; lack of emotional and physical support; and a sense of 'feeling different' constrain their participation in cultural and sport activities. For the lesbian, gay, bisexual and transgender community, the small number of plays and arts events that reflect their own culture, despite the fact that a considerable number of people in the arts world are from their communities; the lack of books on subjects related to their communities in libraries; and the lack of privacy in changing rooms in leisure centres.

Audience Development

Research into audience development carried out on a variety of cultural and art activities, has mainly been quantitative (combined with qualitative) in nature and provides robust findings on the type of audience that attends and the potential audience that does not at present attend. Research has demonstrated that some venues lack understanding of box office computer systems, have poor data collection systems, and events are based on an inaccurate picture of audiences; that partnership working and engaging with the local community can increase audience attendance at the theatre, particularly when the programmes reflect art forms that attract the target audience; and that psychological barriers are foremost in deterring young people from attending cultural venues and innovative methods of marketing are required to increase attendance by this group.

Research on gallery audiences has shown that evening opening of galleries can catch a group of people who normally cannot find time for such visits during the day; can stimulate perception of art galleries as a social destination; and there is potential to augment the gallery experience with special events, music, cafes etc; and that Sunday openings and social events in galleries would encourage more attenders. Similarly, research on museums found that people would like to see evening opening hours, to have more 'innovative' displays and to be more child friendly. Findings also suggest that arts and cultural facilities need to devote more time and resources to making existing programmes more user-friendly and psychologically and intellectually accessible, and should provide more and easily accessible information to raise awareness. It has been found that specifically targeting a particular group and organising an event with educational content can create future interest in and attendance at art exhibitions. However, research has also found that arts provision (eg the number of galleries in an area) does not necessarily equate to high attendance.

Economic Impact

The economic evaluations covered by this Review have demonstrated additionality by way of additional expenditure and employment, direct and indirect impact, and revealing 'hidden' costs and taking them into account when providing the net outcome.

Various economic appraisals and evaluations of the establishment of Dundee Contemporary Arts and its multi-functional provision, West End theatre in London, cultural industries and museums, libraries, galleries, exhibitions and major cultural events have demonstrated direct and indirect economic impacts on local areas and beyond. Evaluations have shown that additional expenditure and employment, both locally and nationally, have often been the result of a cultural centre or major event having been staged. Social impact has also resulted from such cultural facilities or events, including the creation of a 'cultural sector' in the city, enhancing the image of an area and providing the opportunity to promote the area on the international stage.

Such events and cultural facilities also highlight the importance of cultural tourism to the national economy and evaluation of international visitor attractions (such as the Guggenheim Museum) and international events (eg Winter Olympics in the US) have demonstrated the economic and cultural impact on the local community and the host nation.

Similarly, the economic evaluations of sporting events have demonstrated an economic impact on the local area by way of additional expenditure and the generation of additional employment and the potential for long-term economic gains, particularly when a major sporting event has been the catalyst for regeneration of an area. This can also result in the promotion of a city or area on the global stage, and attract visitors to a particular area they may never have visited in the past.

Evaluations of major cultural and sporting events have demonstrated that strong and close partnership working has contributed to the success of the event. For example, the evaluation of the Dundee Contemporary Arts centre concludes that other Scottish cities can learn from this example, where the use of a partnership approach in identifying a strategy using culture as a potential source of regeneration, was successful in its aims. Other examples of successful partnership working are the annual Ceolas week-long festival in South Uist, and the Stirling Initiative Cultural Programme, both of which have a definite economic and social impact on the community and have become popular events for locals and visitors alike; and the Manchester 2002 Commonwealth Games, where there was a coalition between the private sector, national and local government to provide a world class event.

Towards a common evaluation framework

Methods used for impact evaluation by the various agencies and organisations consist of a variety of designs and approaches. Large scale organisational surveys of museums, archives and public libraries provide the quantitative measures of the extent of service provision and financial and attendance data. Some studies have explored social inclusion and under-represented groups through focus groups, individual interviews, and consultation; and household surveys have been carried out to establish the number of participants and non-participants in culture, the arts and sport. Sample sizes range from very small numbers to several thousands. Traditional data collection tools such as self-completion questionnaires, face to face and telephone interviews, focus/discussion groups and case studies have been used in combination or as a single method. Surveys and interviews with cultural and sport staff and project workers have been carried out, as well as user groups.

However, it has to be recognised that the social impact of programmes, initiatives, individual projects etc, is a difficult concept to objectively measure. The Review has examined numerous research studies and evaluations, and it would seem that very few have used all the characteristics required to measure accurately a clear impact on individuals or communities eg large representative and unbiased samples, control groups for comparison purposes and adequate time spans to evaluate a longer term impact. Some studies have consisted of large representative samples, some small qualitative pieces of research have been carried out in a rigorous way using control groups or before and after data, and some have been longitudinal in nature and examined the longer term impact of participation. Nevertheless, there is a substantial number of research projects and evaluations 'out there' that provide good quality, robust evidence gathered in a rigorous manner (although focused on the specific initiative being assessed) that can be used as evidence of the social and economic impact of cultural activities and sport.

There is a widespread acceptance amongst the research community of the value of qualitative data to understand and assess social impact, particularly in the arts field, where self-reported evidence from participants provides a valid testimony to the immediate, and sometimes more long-term, impact on their lives. As long as the data are gathered within a rigorous research framework then qualitative information can provide reliable information on immediate impact. However, many are of the view that many arts projects are too small-scale and too modestly resourced to yield robust results that can be quantified and validated, and that good measurement tools are required that can be both sensitive to the particular area of work, but robust enough to define outcomes.

Related to the issue of specificity of research, it would appear that sometimes organisations involved in researching issues relevant to the service they provide, may focus too narrowly on specific aspects of that service, and might therefore fail to capitalise on the potential of the research to gather better-founded information on wider, related aspects of their service, thereby providing the 'bigger picture'. If the scope of research is sufficiently wide at the outset, it can provide a more robust set of findings in policy terms than if it only relates to single issues; it will also produce data that may be helpful to the organisation and other bodies, and more influential in policy development in the longer term.

Similarly, in relation to economic impact evaluation, critiques of conventional methodology in this area have been made, including that many do not take into account the 'immeasurable' (in economic terms) social impacts such as enhanced quality of life, sense of place, improved image of the host area and thus its ability to attract investment and employment as a result. Such critiques of economic impact evaluations point out that often the negative costs of hosting a mega event are not highlighted, and that a way of evaluating both the social benefits and social costs of major events should be found, examining the negative as well as the positive effects on the community.

Gaps in Evidence

One of the objectives of this Review was to identify gaps in evidence that, if filled, would inform and assist sport, arts and culture policy development, particularly in light of the National Cultural Strategy, Sport 21 and the Partnership Agreement for a Better Scotland. On examining the available evidence, and from discussing gaps in research with the various NDPBs, it would appear that there exist gaps in evidence in the following areas:

• Data collection: research on participation (and therefore non-participation), and views and attitudes to culture and sport requires to be carried out on a large-scale, consistent basis, so that information can be disaggregated at least to local authority level
• Longitudinal studies: very few longitudinal studies are available to inform the assessment of the social and economic impact of cultural and sport initiatives, programmes and major events over the medium to long term. There is a need to measure beyond immediate impact
• Under-represented groups: more research needs to be carried out, particularly at a national level on attitudes held and behavioural reasons behind non-participation by some particular groups in cultural and physical activities and guidance for the cultural and sport agencies on how to promote inclusion can be drawn from examples of good practice
• Well-being and Quality of Life: a) there needs to be more research done on the definition, analysis and measurement of the concepts of 'well-being' and 'quality of life', as often they are cited as the outcomes of participation in arts, culture and sport activities; and b) on the measurement of the relative contribution of arts, culture and sport to well-being and quality of life
• Physical activity: there has been little large-scale research on programmes promoting children's physical activity, or that has focused on socially excluded children; there has been little evaluation of physical activity interventions that have built on young people's ideas
• Arts and Prisons: there appears to have been little evaluation of arts projects and programmes in UK prisons/rehabilitation centres over recent years, and how they may contribute to a reduction in re-offending, learning life skills such as literacy, and increased employability; longitudinal research should be carried out in this area
• Arts and Health: there is a need for a more formal outcome evaluation of the role of arts in health, with many projects being too small-scale for rigorous analysis; there is also a need, where appropriate, for the social and economic impact of the benefits of arts interventions in healthcare to be assessed, with financial savings to the National Health Service demonstrated by economically evaluating the beneficial impact to patients' health
• Creativity in Education: With recent developments in this field, longitudinal research requires to be carried out on the role of creativity in education, at both primary and secondary school level, and how it contributes to problem-solving skills, and cognitive and social development in school and beyond
• Employment: there has been little research or evaluation of projects or programmes designed to increase participation in the arts, culture or sport, and study how these impact on future employment; research has demonstrated that participation in arts activities at school can develop transferable skills for the workplace, but there is little evidence on the actual employment effect
• Art as itself: there is a need for research on the social impact of art itself. Research on the intrinsic nature of art and its capacity to provide meaning to different individuals and different cultures would assist art providers to better understand what art means to target audiences and perhaps encourage more participation/attendance
• Monitoring and Evaluation: Various evaluation toolkits are made available by certain organisations to assist those funded to evaluate individual projects; however, there is a clear gap in the availability of a consistent, common evaluation framework which can be used by all sectors to evaluate programmes and initiatives. There is therefore a need for the development of one (or more) of the existing evaluation toolkits, perhaps using the evaluation guidelines in the government's 'Magenta Book' ¹ as a starting point, and also taking account of HM Treasury's 'Green Book' for economic evaluation of programmes and events
• Combined Social/Economic Impact Evaluation: It would be useful to the government policy teams and to the organisations involved in the delivery and the commissioning of project or programme evaluation, if a) an economic appraisal took place prior to the start up of the programme; b) evaluation of the social outcomes and impact of the programme was carried out; and c) an economic evaluation such as a cost effectiveness or ideally cost benefits analysis was carried out to measure the economic benefits of the social outcomes
• Major Events: Conventional economic impact methodologies evaluating major events tend not to take into account the social impacts such as enhanced quality of life, sense of place, improved image of the host area and thus its ability to attract investment and employment as a result; they also often do not highlight the negative costs. A method that evaluates both the economic and social benefits and disbenefits should be found, examining the negative as well as the positive effects on the community
• Dissemination of information: sponsored NDPBs and agencies perceive a need for a 'central' point of information such as a website or publication, which would provide information on the level of attendance/participation in sport, arts and cultural activities, and the frequency and intensity of use of facilities across the different cultural and sport sectors in Scotland
• Partnership Working: the Review cites examples of successful projects where partnership working has taken place, and there are many examples of good practice for future use. There is much scope for more partnership working to take place between delivery organisations on research and evaluation of initiatives, and sharing findings. All organisations interviewed for this Review were of the view that there was scope for more co-operative working, which would be beneficial from a policy and cost point of view.

This Review has found that there does exist a wide body of evidence on the impact of culture, the arts and sport on individuals and communities, but that there are various gaps in research which, if filled, would contribute significantly to a robust evidence base for these policy areas. The Scottish Executive is currently considering the way ahead in light of the Review's findings, and the wider research community may also be interested to note the gaps in research and the potential for future collaboration in addressing these evidence requirements.

Footnote

1. The Magenta Book, www.policyhub.gov.uk/evalpolicy/magenta/guidance-notes.asp