PART THREE CONCLUSIONS AND RECOMMENDATIONS
CHAPTER TEN CONCLUSIONS
The main aims of this evaluation were to assess progress towards the development of a sustainable infrastructure nationally and locally to support the implementation of Choose Life objectives, to review progress towards the key milestones identified in the Choose Life strategy and set the findings in national and international context, to examine whether and how Choose Life is stimulating effective forms of practice nationally and locally, and to make detailed and staged recommendations to guide the next phase of Choose Life. In this chapter we draw together the main findings relating to the first three aims of the evaluation under the following headings: progress, focus and targeting, innovative development and the use of evidence and sustainability. In the following, and final, chapter we make our recommendations for the next phase of Choose Life.
10.2 Progress
10.2.1 Scotland in context
A growing recognition of suicide as a major public health issue has fuelled interest in and commitment to national strategic approaches to suicide prevention, to promote coordinated and multi-sectoral interventions towards long term goals (Anderson and Jenkins, 2005).
Choose Life has much in common with other national strategies to address suicide which tend to draw on a common set of international guidelines and a growing body of evidence on risk factors and causal pathways leading to suicide and suicidal behaviour. As yet, however, there is little indication that national suicide prevention strategies have a positive impact on trends in suicide (De Leo and Evans, 2004). This may reflect the challenges associated not only with achieving change in behaviour where causal pathways are complex and multi-factorial and where change is likely to occur (if at all) over protracted time scales, but also in evaluating the effectiveness of multiple and multi-level interventions that aim to tackle a broad range of upstream and downstream determinants. It is vital, therefore, to gain a better understanding of the means by which strategies are translated into effective interventions and actions.
Although there are many similarities between Choose Life and other national suicide prevention strategies, Scotland's approach is distinctive in several respects. Choose Life sets out a clear approach and plan for implementation, which includes dedicated national capacity to support and coordinate implementation, underpinned by an earmarked national and local budgetary allocation and guided by the findings of an early formative evaluation.
One of the distinguishing features of Choose Life is that is placed within a national public mental health programme, that is part of wider Scottish Executive policy commitments to improve population health, promote social justice and tackle inequalities. This allows suicide prevention work to be undertaken within a wider framework of policy objectives and initiatives that share the overarching goals of population mental health improvement.
The approach taken to Choose Life implementation follows the model of transformational change espoused by the National Programme (Kotter, 1996): building coalitions and alliances, developing a joint vision for action and using resources and initiatives as catalysts to facilitate system change and promote mainstreaming.
10.2.2 Infrastructures for implementation
The National Implementation Support Team has played a pivotal role in working towards the mainstreaming of suicide prevention activity with wider Executive policy agendas. Despite a lengthy process to establish the team, NIST has made demonstrable progress and built momentum in relation to all its key functions (awareness raising/campaigning; working with the media; development and dissemination of information and knowledge; and supporting local implementation), while also recognising the need to be increasingly strategic. Challenges ahead for NIST include: building clinical involvement and engagement at national and local level; and facilitating local capacity building in key areas of identified weakness, e.g. monitoring and evaluation. In view of the early experience in the first stages of NIST, it would be valuable to have agreed objectives for NIST for the next phase which recognise the changes in implementation support requirements and the external policy and organisational landscape.
The findings from the evaluation of phase one suggest that Community Planning Partnerships have been the best available mechanism to take forward local planning, coordination and implementation of Choose Life objectives, in view of the importance that was attached in the Choose Life strategy and action plan and in subsequent guidance to local, cross-sectoral ownership of, and grass roots engagement in, suicide prevention activities.
Our findings demonstrate that considerable progress has been made towards the establishment of the kind of infrastructure which the Scottish Executive and partners in the development of Choose Life considered to be important to achieve these goals.
Choose Life has provided a powerful, unprecedented rallying call to bring partners together in order to expedite cross-sectoral planning and action to tackle suicide prevention as part of a mental health improvement agenda. The challenge now will be to maintain the momentum and translate this into sustained action, to track implementation across sectors and, where possible, to achieve measurable impacts.
Progress has been made in encouraging the adoption of suicide prevention objectives in a range of local policies and service plans and Choose Life partnerships have generally sought proactively to achieve this, by building links with key partners, seeking engagement with key decision makers locally and hooking into other relevant policy priorities. This has proved to be a gradual process that requires time and concerted effort. It cannot be said that, as yet, Choose Life had been mainstreamed, although it is making progress in that direction.
However, the variability in the maturity of local Community Planning Partnerships has had a critical influence on Choose Life progress at local level. Key factors that are likely to aid progress include: facilitative leadership style, strong strategic focus on jointly agreed priorities for action, early attention to mainstreaming (see below) as a desired intermediate outcome, capacity to draw on a range of intelligence to inform decisions, and feedback mechanisms to review, learn and adjust.
Local experiences in phase one of Choose Life implementation have highlighted that Community Planning Partnerships have the potential to be effective vehicles to build engagement among partners in the community and voluntary sector and to build Choose Life objectives into the fabric of other policy structures and priorities. However, at this stage, for a number of reasons it is difficult to be able to make judgements about outcomes and impacts, in part because implementation of Choose Life is not yet fully developed and, more fundamentally, because of the difficulties with outcome measurement and causal attribution in respect of complex interventions.
In tackling complex issues and seeking to engender systemic change, it has been noted that it is the interactions between and among partners that are critical for success (Berwick, 1997).
This was borne out strongly in the evaluation on a number of dimensions:
- Interactions within local areas, where local coordinators played a key role in fostering and supporting relationships with strategic players, service providers and local communities
- Interactions among local areas, which had the potential to facilitate exchange of learning and resources
- National: local interactions. It was clear that the maturity of links and connections at national level had implications for partnership engagement and development at local level. Endorsement of Choose Life objectives by the relevant policy community and support from key national bodies helped prime local activity and provide levers for change.
- National: national interactions, bringing together areas of expertise, working to promote awareness and to advocate for change. To date, there has been an under-utilisation of national organisations and bodies. They should be encouraged to contribute their proven expertise in particular topic areas. Their engagement in key stakeholder groups at national level would undoubtedly benefit local partnership development.
In looking ahead, it is important to consider the focus of activity required at national level for the future stages of implementation, to make use of resources of all national players, recognising what it is that NIST is uniquely placed to do and what contributions can be made by other agencies. The evaluation suggests that progress towards Choose Life objectives is based on effective activity at national level in the following arenas:
- Policy advocacy within the Scottish Executive and with other relevant national bodies
- Raising awareness and influencing those who shape opinions
- Promoting engagement and facilitating dialogue
- Acting as a catalyst for coordination across boundaries
- Performance management to track and oversee progress
- Building capacity, in particular to use and generate evidence.
There are indications that these activities are instrumental in facilitating local progress. In addition the effective engagement of key national stakeholder groups has been found to benefit local partnership development by proving a mandate to support the participation of key sectors. There is a need to build on the effective and extensive developments of links and alliances achieved in phase one, to ensure that this is carried forward in ways that enhance capacity to attain the objectives of phase two, both locally and nationally. The findings from the evaluation provide an opportunity for NIST and the National Programme to take stock of relationships established within and beyond the Scottish Executive and to identify key next steps.
The evaluation found that various models of local coordination had been developed and these had often been subject to refinement as local work progressed. A dedicated coordination post tended to be preferred, for reasons of clarity of communication, capacity to be proactive and forge effective strategic and operational links, and to develop expertise. However, the evaluation has not been able to provide conclusive evidence that this model is more effective than alternatives. It is worth noting that the financial data indicate that only 10% of all funds allocated for Choose Life were spent on staff posts which were exclusively dedicated to coordination and development activity. More commonly coordination was combined with delivery of specific interventions or activities.
10.2.3 Phases and stages
It is important to recognise that phase one was focused on development. It brought particular challenges and requirements and was a key opportunity to gain interest and attention, test out ideas and approaches and establish capacity for action.
The infrastructures developed in phase one were shaped by the key tasks identified at that stage, especially to build capacity and influence mainstream policy and practice. Initial partnerships grew up among those with immediate interest in, and commitment to, Choose Life goals. There is a need to reflect on the infrastructure and partnerships in place and to consider what is required to ensure the achievement of goals in the next stages of implementation, in particular how to widen engagement as part of the drive to influence mainstream policy and activity. Choose Life partnerships now need to engage in learning from phase one and adjust their status and composition accordingly. There is evidence that this is already happening in some local areas.
Getting and keeping partners involved proved to be time consuming and demanding in phase one. Challenges lie ahead in balancing the involvement of an increasing range of players with the delivery of an achievable programme of work. Major players have to be able to articulate and demonstrate how actions are contributing to desired goals and outcomes within a coherent framework.
10.3 Focus and targeting
10.3.1 Use of resources
It is difficult to gain a clear picture of the balance of resource allocation and the range of activity supported due to:
- Gaps in data which create problems in mapping resource allocation
- Problems with categorisation of activity, so that an accurate breakdown is difficult to obtain
- Underestimation of additional/complementary investment in Choose Life
- Absence of data on activities relevant to Choose Life but funded through other routes, including as part of mainstream service delivery.
Nevertheless, in the first phase of Choose LifeCPPs attracted substantial additional investment at local level (£1.6m), and there has been a substantial level of in-kind contribution which is grossly under-recognised. On the other hand, not all areas have been equally successful in raising additional monetary funding.
A high degree of variability is evident among local areas in terms in the way resources are allocated to:
- Three key functions of coordination, training and support for voluntary and community sector
- The seven priority groups
- Specific activities and interventions.
A range of factors has influenced the determination of local priorities for investment of Choose Life monies: local consultation, lobbying of groups advocating for a particular issue or set of interests, needs assessment data, building on previous initiatives, local concerns including recent incidents of suicide, and other national policy priorities.
It is important therefore to bear in mind the expectations set for the early implementation of Choose Life. The availability of resources for local partnerships to allocate to agreed priorities appears to have had instrumental value at this stage of development, as a means to stimulate involvement and bring people together. It will remain very difficult to demonstrate the specific outcomes achieved by this tactic but qualitative evidence suggests that the value should not be underestimated. However, if strategic planners (local and national) want to achieve demonstrable outcomes and impacts in relation to stated objectives, there may be case for adopting a different or complementary approach which requires greater specificity of inputs, outputs and outcomes, and which operates on a larger scale.
The evaluation documented considerable commonality in many of the initiatives and projects developed in local areas, in particular in terms of training and public awareness raising. There are grounds to conclude that there is a degree of unnecessary duplication of effort and some aspects of local work might be better undertaken on a collaborative basis, ensuring that best use is made of common methodologies and effective tools and resources. Steps towards building collaborative models of development are already in evidence (for example, some neighbouring areas were making arrangements to pool coordinating capacity; and the work to support local evaluation has given nominated local coordinators a lead role).
The tentative conclusions relating to potential cost-effectiveness of delivering Choose Life should be noted. If Choose Life proves to be an effective approach to suicide prevention, it is likely to be a good use of relative modest resources, in view of the costs associated with suicide.
10.3.2. Equity
The degree of variability gives rise to questions about why some groups tend consistently to feature more prominently (in particular children and young people) and others tend to be given a low priority (e.g. prisoners and homeless people).
There are also striking variations in the types of interventions and approaches used to address similar issues or target similar groups. This may suggest a need to pool information and experience and review what is known from research and practice experience.
Overall, the evidence would suggest that the emphasis to date has been on gaining local engagement with Choose Life and on supporting local initiatives that facilitate such engagement. Funded interventions often aim to tackle several objectives suggesting a lack of specificity and focus that has been identified in reviews of other mental health improvement interventions. This is consistent with findings from a recent review of mental health improvement activity ( SDC, 2004). Evaluation can provide a useful lens through which projects and partnerships can clarify objective setting (Patton 2005).
The broad range of priorities set in the Choose Life strategy allowed local areas a high degree of latitude to determine their local focus. It may be that, in future stages of implementation, more attention needs to be directed towards considerations of equity on at least two counts: to take account of what is known about relative importance of particular risk and protective factors in determining suicidal behaviour; and to ensure that interventions are targeting inequalities and focusing on how to reach those for whom support is currently least accessible.
Seeking to make resources and responses more accessible and acceptable to certain groups who tend to be deemed 'hard to reach' currently will have implications for the types of interventions offered and methods and mechanisms of delivery, as well as for the partners who need to be involved.
10.3.3 Self-harm
Choose Life has stimulated a considerable amount of activity relating to self-harm, but there is evidence of widespread differences across local areas in definitions of what constitutes 'high risk' suicidal behaviour and in the range of activities which have been developed to address the problem. While this diversity is in keeping with Choose Life's dual emphasis on a broad public health approach combined with targeted interventions aimed at high risk groups, the findings suggest the need for further consideration by NIST about how to integrate action on self-harm into the wider suicide prevention strategy.
10.4 Innovative development and the use of evidence
The Choose Life action plan and supporting guidance contained an expectation that local areas would use a proportion of their funding allocation to support innovation in the voluntary and community sectors and that this would include self-help initiatives.
Innovation was seen in two main ways: in terms of interventions, as a means to introduce new ways of working and widen the range of available options; and in instrumental terms, as a means of promoting partnership development and local 'ownership' and engendering learning and wider system change.
The evaluation is unable to assess the effectiveness of innovative developments initiated in phase one. Documented examples and experiences of introducing innovation do, however, exemplify some of the conceptual and practical difficulties involved in the context of Choose Life implementation:
- Short-term funding cycles and concerns about long-term responsibilities for the resourcing of developments
- Fragility of the infrastructure within the non-statutory sectors which can militate against commitment to new ventures
- Imbalances in the status and resources of statutory and non-statutory sectors and cultural differences in values and beliefs
- The tension between risk taking and using tried and tested vehicles for/ methods of delivery.
In particular, there were some indications that 'instrumental' innovation required certain preconditions to flourish:
- Capacity for reflexive practice and evaluation to allow innovative developments to act as a test bed for wider learning and implementation
- Established partnerships to ensure that learning from innovations could be readily imported into mainstream working.
We have been encouraged by the evidence that many local Choose Life partnerships were engaging in a process of reflection and review at the end of phase one with the revision of local action plans. This suggests the importance of using key staging posts in the implementation of policy initiatives working towards long-term goals to prompt review. In other recent initiatives, a readiness to reflect and learn and to build in data gathering and analysis was found to enhance capacity to use innovation to engender wider system change (McCollam et al, 2006).
There was limited progress at local level in generating evidence of impact. Multiple sources of information and types of evidence, including research, were used to inform local planning and activity. However, research was rarely used systematically. There remains an absence of robust, definitive evidence of effective practice that is framed in ways which are useful and useable in local contexts to inform planning and priority setting. Local areas were more likely to be familiar with the evidence to support a particular project or intervention than to have a broad understanding of the evidence base. Local capacity to identify, interpret and apply this information in the local context was not consistently present. There are also tensions between a 'rational' model of evidence-based decision making and the more consultative approach espoused in community planning.
Taken together, these factors signal the importance of proactive signposting, framing and interpretation of research evidence in ways that are relevant and accessible to practice communities. At the same time, it is important to acknowledge gaps in the available evidence base and work with planners and practitioners to explore opportunities to develop theoretically robust models to guide decisions on priorities and project design.
10.5 Sustainability
The evaluation found evidence of some progress towards sustainability at the three levels identified: delivery of interventions, assimilation of good practice and incorporation of objectives and priorities into other policy streams and initiatives.
The main emphasis in local Choose Life implementation has however tended to be focused on sustainability at project level, although it has been argued that mainstreaming of effective ways of working and influencing policy are more important (MacKenzie et al, 2005). These more abstract notions of sustainability are contingent on champions and culture carriers to advocate in other arenas and make connections with other agendas. This is inevitably complex long-term work. Where policy diffusion and implementation have been accomplished in other arenas (for example, in relation to equalities issues), it has required considerable investment of time and resources, supported by persistent campaigning and favourable legislative and policy frameworks.
Training activities accounted for a significant proportion of Choose Life investment and local and national informants attached considerable weight to the anticipated benefits to be derived by building capacity and skills at community level and among key staff groups. It will be therefore be important in phase two to track and assess the impact of training in view of what is known about the difficulties of ensuring that those equipped with skills and confidence are enabled to make best use of these.