CHAPTER ONE Introduction
1.1 This report is the final evaluation report on the two-year Intensive Support and Monitoring Services ( ISMS) evaluation that DTZ was commissioned to undertake by the then Scottish Executive.
1.2 ISMS is a new type of disposal within the Children's Hearings System that involves issuing a young person with a Movement Restriction Condition ( MRC), by means of an electronic tag, for a set period of time to be accompanied by intensive support during their assessment for ISMS, while on the MRC and in a post- MRC phase for the same length of time that the young person was on the MRC. ISMS is currently being delivered in seven local authority ( LA) areas in Scotland (the phase 1 local authorities): Dundee, East Dunbartonshire, Edinburgh, Glasgow, Highland, Moray and West Dunbartonshire.
1.3 In this three-year period, a considerable amount of information and data was collected, which is reflected in the final report. However, the Executive Summary acts as a very useful overview of our main findings and conclusions.
1.4 The report covers the period from October 2004 (when the phase 1 LAs were told by the Scottish Executive that they were to receive funding to implement ISMS) to the end of April 2007, which is two years on from the official starting date for ISMS in April 2005.
1.5 Given the length of this report and the amount of detail which it contains, the key arguments and conclusions are highlighted in bold type.
Research specification
1.6 In its Research Specification, the Scottish Executive Education Department stated that it wished to commission an evaluation of the use of ISMS within the Children's Hearings System. This research was to evaluate the operation, the financial costs and long-term effectiveness of ISMS. The principal aims of the research were twofold
- To evaluate the effectiveness of ISMS in reducing the offending behaviour of persistent young offenders, in responding to the needs of young people who are at risk of absconding, and/or who are likely to injure themselves or others, and improving the longer term outcomes for young people, for example returning to full-time education.
- To assess the cost-effectiveness of ISMS in delivering successful outcomes especially for the children and young people, compared with alternative measures.
Research method
1.7 We designed and employed a number of different research instruments to address the requirements of the research, which we have had to adapt as these requirements have shifted over the course of the research, which is to be expected in any complex evaluation of a major new initiative. Figure 1.1 below outlines the various stages of the research over the two-year period. The stages are then explained in more detail.
Figure 1.1 Research method outline
1.8 A policy and research review was conducted based on review of the relevant legislation, best practice elsewhere, the academic literature and other sources in order to help to understand the origins of ISMS and lessons that have been or could be learned from elsewhere.
1.9 Interviews with other researchers working in the field was undertaken to map all of the other relevant research activity and to attempt to reach collaborative working arrangements to share knowledge and avoid unnecessary duplication.
1.10 Interviews with the phase 1 LA areas on the implementation and operation of ISMS in their area, has been an ongoing feature of the evaluation over the two years. . They have considered issues such as the area's motivation to participate, how they structured themselves to deliver ISMS, the assessment process, partnership working, service provision, costs and resources, their monitoring and evaluation arrangements, and details on outcome measures. These interviews led to the construction of profiles for the LAs that were revised during the course of the research and give the LAs the opportunity to explain why they chose to implement and operate ISMS in the way in which they did.
1.11 Feedback to the Scottish Executive and the Research Advisory Group ( RAG). There were also regular meetings with the Scottish Executive and the wider Research Advisory Group (which included representatives from the Scottish Executive, the LAs, other researchers and SCRA). These meetings discussed the progress of the research and were a source of advice and guidance for the researchers. We also liaised with the LAs and the Scottish Executive in two national conferences on ISMS and one on secure accommodation that were held during the course of the research.
1.12 Interviews with main external providers on ISMS. These providers are Includem, NCH, Reliance Monitoring Services and Serco. This was to gain a perspective of the roll-out of ISMS from the view of these key partners. These interviews were also repeated on a number of occasions over the period of the research.
1.13 Mapping and baselining. The early stages of the research led to a mapping and baselining report that mapped all the ISMS and research activity to provide the Scottish Executive and the RAG with a full picture of the initial stages of the implementation of ISMS. It also confirmed the collaborative working arrangements that were to be put in place with other research teams as well as providing initial (baseline) data on ISMS performance.
1.14 Survey of wider stakeholders. This was a postal survey of a number of individuals within a range of stakeholder groups, including social workers, teachers, addictions workers, etc., to understand their views and opinions of ISMS, and their involvement in the programme.
1.15 Survey of Panel Members within the Children's Hearings System. This was another postal survey to understand the views and opinions of Panel Members, and their involvement with ISMS.
1.16 Case studies of individual ISMS and Intensive Support only cases. This involved collecting a range of data on ten ISMS cases throughout Scotland and ten intensive support only cases to act as a broad comparison group, and analysis of this data with a particular focus on outcomes.
1.17 Other outcome data. This involved analysis of self-report data held by Includem and from interview transcripts held by Glasgow City Council from their own internal evaluation of ISMS.
1.19 An analysis of the cost of implementing and operating ISMS in each of the phase 1 LAs, based on returns that each of the LAs made to the Scottish Executive.
1.20 A final report that pulls together all of the data and attempts to triangulate the evidence to derive the main findings and conclusions of the research.
1.21 This method has been successful in helping the research team to build a full picture of the available evidence on the implementation and operation of ISMS to date.
Report outline
1.22 We have decided to report by research instrument rather than thematically. This is because the study method was designed so that information on all of the different aspects about ISMS implementation would be captured. This was a complex task and a number of research instruments were used for specific purposes, as highlighted above. The results from each of these instruments are outlined in each of the report sections, with the final section attempting to tie together and triangulate these varied findings into an extensive and referenced conclusions section that has robust and valid findings.
- Section 2 provides details on the policy and research review.
- Section 3 outlines the profile of ISMS implementation and operation in each LA
- Section 4 profiles each of the main external providers
- Section 5 presents results from the survey of wider stakeholders
- Section 6 presents results from the survey of Panel Members
- Section 7 analyses the results from the case study research
- Section 8 analyses the results from the other outcome data sources
- Section 9 produces the analysis of the costs of the programme
- Section 10 produces the main findings and conclusions from the research.
1.23 Care is taken throughout this report to maintain the confidentiality of people who we spoke to and the files and records of the young people who we considered.
Acknowledgements
1.24 This study has very much been a collaborative effort involving many other consultants and researchers who are working for DTZ as sub-consultants, or who are working on their own evaluations in this area and with whom we have established collaborative working partnerships to avoid duplication of efforts and unduly burdening study participants.
1.25 Firstly, we would like to thank Fiona Dill and her colleagues within the Scottish Government who have managed us on this project and were always readily available with advice and guidance.
1.26 We would like to thank Lorna Shaw and Elaine MacKinnon at Research Resource, who undertook the surveys of Panel Members and wider stakeholders as sub-consultants to us. Our other sub-consultants, Dr. Mark Penman (freelance researcher) and Dr. Kay Tisdall (University of Edinburgh), we would like to thank for their advice and work on the case studies and literature review.
1.27 We would like to thank Furzana Khan, Professor Malcolm Hill and Germana Vitrano, who work in association with the Glasgow Centre for Child and Society based at the Universities of Glasgow and Strathclyde, for their advice and guidance in developing the interview schedules and for their assistance with the young people, carers and Includem aspects of the consultation. We would like to thank Nina Vaswani, Research Officer (Youth Justice) in Glasgow Social Work Services, for her help and advice during the project and for making available the results from our own evaluation of ISMS in Glasgow. We would also like to thank Dr. Gillian Henderson and Alyson Evans from the Scottish Children's Reporters Administration ( SCRA) for their help and advice and for making available to us their research on the use of MRCs in the Children's Hearing System.
1.28 In addition, we would like to thank Tim Chapman and Dr. Manzoor Khazi for making available material from the evaluations that they are conducting for the Moray Youth Justice Team, and Dr. Morag MacNeil, Dr. Bob Stradling from the UHI Millennium Institute and Alison Clark from Highland Council with whom we collaborated in undertaking the evaluation in Highland.
1.29 We would like to thank all the local authority ISMS teams and the other organisations that have assisted us during our research ( SCRA, Includem, NCH, Reliance Monitoring Services and Serco).
1.30 Lastly, the continued advice and support of the ISMS Research Advisory Group is very much appreciated.