Scottish Government

7 CONCLUSIONS AND RECOMMENDATIONS

Introduction

7.1 This chapter summarises the assessment of the delivery, impacts and value for money of the two projects. The chapter discusses the comparative effectiveness of the projects related to other initiatives and alternative models of intervention and sets out a number of recommendations.

Evaluation of delivery and impacts

7.2 Although both projects deliver messages about drugs and alcohol within a wider promotion of healthy eating and exercise, there are some differences between the two projects in terms of the aims and focus of their drugs education elements. Celtic Against Drugs is delivered to P5 pupils and is focused upon introductory messages about drugs and alcohol, although it is apparent that many pupils already have (often inaccurate) personal knowledge. Rangers Positive Choices is delivered to P7 pupils and is focused on providing drugs and alcohol messages and resistance skills to equip pupils for the transition to secondary school, thereby complementing the Choices for Life events. Celtic Against Drugs includes a multi-school event and parent's seminar and Rangers Positive Choices is more explicitly linked to a Community Health Care Partnership. Each project also uses a different model of delivery. Celtic Against Drugs is primarily delivered by Celtic FC coaches with inputs from delivery partners at culminating multi-school and parent's events. In contrast, three of the Rangers Positive Choices school-based sessions are delivered by external partners. In our assessment, the targeting of the respective age groups is appropriate and both delivery models are effective in that they combine messages on drugs and alcohol delivered by the football clubs' coaches with further expert inputs from the Glasgow Council on Alcohol and Strathclyde Police.

7.3 Despite these differences, it was striking that very similar perceptions about both projects were shared by teachers, delivery partners and our own observations. Both projects were positively viewed by pupils, teachers and delivery partners and were based on clear and robust rationales, appropriate training and effective financial and operational management. The professionalism of the project staff was noted by all research respondents and it was evident that the projects had engaged with boys and girls, children who did not like football or support the two teams and children with special needs. There was scope in both projects to improve evaluation, formalise partnership working, clarify the specific inputs of each delivery agent and enhance the provision of information to class teachers.

7.4 Both projects fit with the Road to Recovery and Changing Scotland's Relationship with Alcohol prioritisation of prevention (Scottish Government, 2008b: 8). The projects make a significant positive contribution to key health and wellbeing outcomes of Curriculum for Excellence (Scottish Government, 2008c), including developing children's understanding of the use and misuse of a variety of substances and enabling pupils to experience positive aspects of healthy living and activity for themselves. It is not possible, in a process evaluation, to determine other goals such as establishing a pattern of health and well being that will be sustained into adult life or whether the projects enabled participants to make informed personal choices with the aim of promoting healthy lifestyles.

7.5 Both projects also met a number of good practice principles as set out in the Road to Recovery (2008b: 17). In our assessment, both projects utilised appropriate teaching materials and activities and these were used effectively. The projects delivered culturally appropriate and relevant education targeted to needs and context, mixing football with real life and recognising the home and neighbourhood circumstances of pupils. Celtic Against Drugs also included engagement with parents. The projects were delivered by appropriately trained practitioners (and it should be noted that less than a third of school teaching staff have received such specialist training, Stead et al., 2007b). Although the projects linked well to other health education and promotion in the primary schools, we believe that they could be more proactively and effectively integrated.

7.6 Our study confirms previous research findings that external providers of drugs education achieved the most positive outcomes in terms of clarity of message, drugs awareness being enhanced and pupil engagement (Stead et al., 2007b). This is based partly on the novelty and expert status identified by Stead et al., but there also appeared to be an additional appeal arising from the use of football and the Old Firm clubs, and this appeal extended beyond their own supporters. Stead et al. found that external visitors were often the least interactive and did not employ the most effective teaching methods, but our study indicated that the football club coaches and their delivery partners used very interactive and effective teaching techniques.

7.7 Both projects met some of the key success criteria identified in previous research, including being:

• Very interactive;
• Multi-component;
• Appropriate to the age of the pupils;
• Relevant and socially and culturally specific to the targeted population;
• Based on a broad definition of drugs including medicines and legal substances;
• Focused upon normative and social influences, although we have insufficient evidence on the impact of this;
• Based upon a positive and holistic promotion of healthy lifestyles;
• Ensuring information and skills are translated into pupils' everyday lives, although again we do not have sufficient evidence to indicate whether pupils had learned and retained skills that they would subsequently utilise;
• Coordinated amongst partners, although there is a need for a clearer assessment of learning outcomes achieved by the projects.

7.8 The Rangers Positive Choices sessions and the Celtic Against Drugs Day at Celtic and parents seminar matched the research evidence that messages can be most effective if delivered in partnership with a range of agencies and integrating inputs from different sources (Scottish Government, 2008b: 18). Both projects also appeared to be meeting one of the other aims of the Scottish Government funding: providing diversionary activities and healthy alternatives to drugs and alcohol and anti-social behaviour. Both projects had provided direct opportunities for pupils to participate in out-of-school football activities and, certainly in the case of Celtic Against Drugs, had also informed pupils of other local sporting and exercise opportunities. The projects appeared to have had other impacts, including encouraging pupils to participate in physical education in schools and enabling teachers to address the issue of sectarianism. The Day at Celtic provided a direct opportunity for pupils to interact with children from other faiths, ethnicities and neighbourhoods although the impact of this is not known.

Comparative effectiveness

7.9 It is not possible to compare the effectiveness of the Celtic Against Drugs and Rangers Positive Choices projects directly with other preventative initiatives for young people as the longer term impacts of preventative programmes are unknown (Audit Scotland, 2009). Previous evaluations found that the key messages from the Know the Score media campaigns and the Choices for Life events for Primary 7 pupils were clearly understood ( TNS System Three, 2007; Phillips and Kinver, 2007; Menzies and Myant, 2006). However, the extent to which these messages increased factual knowledge varied. Celtic Against Drugs and Rangers Positive Choices also appeared to have delivered key messages and pupils' survey data and teachers' perceptions indicate that the pupils understood these key messages.

Comparing the projects to alternative delivery models

7.10 Both the Celtic Foundation and Rangers FC delivered a whole-class preventative model. This was different to previous programmes that had worked with pupils referred to the projects based on them being perceived to be at risk. There are three major differences between these two models. Firstly, the current model is geographically concentrated in particular areas of Glasgow rather than being city wide. Secondly, the group dynamics and intensity of interventions have been altered (i.e. most of the sessions do not bring pupils together from different schools and the size of pupil groups is considerably larger). Thirdly, the educational content of the sessions and activities are different. It may be argued that public funding is most effective when it is targeted at individuals identified as being at particular risk and provides more intensive supportive interventions to this group, which in turn is more likely to have a significant impact.

7.11 However, there a number of benefits to the current whole-class approach that justifies this model being retained. Audit Scotland (2009) has identified the relative lack of funding of preventative approaches across Scotland. A range of projects in Glasgow, including some delivered by Celtic and Rangers, already provide targeted support for at-risk young people, whereas there is a current gap in preventative projects. A focus on prevention rather than remedial intervention links to the strategic priorities of the Scottish Government's national drugs and alcohol strategies. The current model delivers preventative education to larger numbers of pupils at key stages (P5 and P7). This increases the education provided to children who may be at risk but who have not yet been identified as being so by agencies, and class teachers were not necessarily confident that schools could identify these children. The current model also provides opportunities for larger numbers of pupils to participate in sport and healthy exercise diversionary activities.

7.12 Although the projects are not individually targeted on at-risk pupils, their focus on primary schools in some of the most deprived areas of Glasgow results in messages being delivered to young people who are statistically more likely to be exposed to the risks of drugs and alcohol (in their home and neighbourhood environment) and less likely to be able to access sports activities. Previous evaluations of drug education campaigns have identified socio-economic status as a key determinant of how educational messages are perceived and interpreted ( TNS System Three, 2007: 20). Teachers believed that the most vulnerable children in their class had engaged with the projects.

7.13 There is a need for Glasgow City Council and its partners, including the two clubs, to consider how similar preventative messages may be delivered to primary schools in other areas of the city. The need for preventative messages at earlier ages is supported by previous research with 13-15 year olds ( TNS System Three, 2007). This research identified a lack of basic understanding about some drugs, reliance upon family and friends for (often inaccurate) information and a belief that national publicity campaigns such as Know the Score were not relevant to their own lives. Similarly, research with pupils moving from P7 to S1 (Menzies and Myant, 2006: 30) revealed pupils' misconceptions about situations involving drugs.

Value for money

7.14 The unit (per pupil) costs of both projects appear relatively modest and both projects have also generated significant additionality through in-kind contributions from the clubs and delivery partners. Given that the projects complement, rather than replace, current drugs education, they represent a significant increase in the amount of drugs education pupils receive, as previous research indicated an average of 3 and 4-5 hours provided annually to P5 and P7 pupils respectively (Stead et al., 2007b). The projects are also delivering significant diversionary programmes free of charge to pupils. At a basic level, the total combined grant to the two projects was only slightly more than the estimated annual economic and social cost of an individual problematic drug user (Casey et al., 2009), although the complexity of influences on drugs use and the lack of longer-term outcome data about the projects prevents any clear correlations being made here. Of course, other projects may be more effective, but it is simply not possible to establish this (Audit Scotland, 2009) and it should be noted that several research participants compared the two projects favourably to other external drugs education initiatives operating in schools. The fact that Strathclyde Police was reported to have recently reduced its own in-schools drugs education capacity suggests that that the projects may be plugging an important emerging gap in provision in the schools in which they operate.

Funding and partnership considerations

7.15 The Scottish Government's funding of the Celtic Foundation and Rangers FC may be justified on the grounds that Glasgow City has the highest numbers and prevalence of problem drug users in Scotland (Hay et al., 2009). Other football clubs in Scotland have either obtained funding for projects from a range of sources, including local authorities, public agencies and the private sector or funded projects themselves. It is particularly noteworthy that this includes some of the smaller clubs in the SFL. However, it is also evident that local authority funding sources are constrained and attempts at consortia funding arrangements are often complex (as in the case of Heart of Midlothian's Onside programme). Many SPL and SFL clubs are not in a financial position to provide substantial funding for such projects. The evidence from the English Premier League shows that even comparatively wealthy clubs rely on a national framework to maximise the effectiveness of educational projects. This suggests that the national Citizenship Through Football partnership will be very important, both in terms of facilitating Scottish football clubs' contribution to drugs and alcohol education and diversionary activities and evaluating existing projects and disseminating good practice. It is evident that football and football clubs do have a particular ability to engage some young people and act as effective providers of drugs, alcohol and healthy lifestyle messages and there is scope to build on this potential.

Recommendations

7.16 The recommendations arising from the research are:

• For both Celtic Against Drugs and Rangers Positive Choices, class teachers (as well as head teachers) should be provided in advance with more detailed information about the aims, content and intended learning outcomes of the projects in order to enable teachers to plan their own in-class activities accordingly.
• For both Celtic Against Drugs and Rangers Positive Choices, pupils and teachers' surveys should be complemented by an end of project assessment of each individual pupil's understanding of key information and messages and the specific new information that they have acquired as a result of the projects. These assessments will need to be developed in conjunction with key strategic and delivery partners. Pupils will also require further support to ensure that they fully understand, and are able to complete, evaluation exercises.
• Any future reports on the projects submitted by the Celtic Foundation and Rangers FC to the Scottish Government (and other funders and/or partners) should provide data disaggregated by the gender and ethnicity of pupils and school type (non-denominational / Roman Catholic / special) and quantify the numbers of pupils participating in the clubs' additional football or other sporting activities as a direct result of the projects.
• Rangers Positive Choices should include a parents evening and a Pathways to Participation-type event or session.
• The Scottish Government should work with the Citizenship Through Football Partnership to consider the findings of this report and to explore how football can contribute more widely to drug and alcohol education in the future, given the evident power of football to engage with many young people. Partners should work together to consider existing best practice in this area and disseminate this to clubs and others.
• Where, possible, this work should draw on evidence and good practice from other educational campaigns and initiatives that use sports personalities to inspire children and young people (such as the Sporting Champions programme). The most effective campaigns are both positive and seek to challenge peer-group influences by framing some risky behaviours as social conformity and appealing to young people's autonomy and self-direction (see Stead et al., 2009 for some international examples).