CHAPTER TWO: METHODS
2.1 A range of qualitative and quantitative methods were drawn upon in the evaluation of the Youth Court pilots. They are described briefly in this chapter.
PROCESS EVALUATION
Analysis of marking decisions by Procurators Fiscal
2.2 The Youth Court Procurators Fiscal provided information about marking decisions in respect of relevant young people reported to them by the police. The information provided included the age and sex of the young person, whether there were any co-accused, whether other incidents had been rolled up with this one, whether the case was discussed with the social work department or the Reporter, whether the case met the persistency criteria or contextual criteria, the case outcome, the route of the referral and, for those prosecuted in the Youth Court, the date of charge that triggered the referral. An additional space was provided for Procurators Fiscal to record any other observations about the case. This information was provided on 1,668 youth prosecution cases marked by the Procurators Fiscal in Hamilton between June 2003 and October 2004 and 2,236 Cases marked by the Airdrie Procurators Fiscal between June 2004 and February 2006. It enabled some comparison to be made between Procurator Fiscal marking outcomes and the characteristics of reported cases.
Analysis of cases prosecuted in the Youth Courts
2.3 Information on cases prosecuted in the pilot Youth Courts was collected by the Youth Court Co-ordinator. The database provided information about the characteristics of young people and the progress of cases through the Youth Court. The daily court sheets completed by the court clerk were the main source of information. Supplemental information came from other agencies involved in the court process. An anonymised version of the co-ordinator's database was provided to the research team. This provided details of 611 cases prosecuted in to the Hamilton Youth Court from June 2003 to December 2004 and 543 cases prosecuted in the Airdrie Youth Court between June 2004 and December 2005. Young people could feature in more than one case. For example, where an individual was prosecuted for 2 separate cases on the same day, these were treated as separate cases if this was how the court regarded them. Young people with multiple cases may have had their them merged (with the cases being heard on the same day) as they progressed through court.
Analysis of data provided by the Scottish Children's Reporters Association
2.4 In order to gather information on the previous involvement of young people prosecuted in the Youth Court with the Children's Hearings System, the Scottish Children's Reporter Administration ( SCRA) provided information held on their national Referral Administration Database ( RAD). This covered whether, since RAD was rolled out in 2002, the young person had been referred to the Children's Reporter (Reporter), the details of such referrals and Reporters' and Hearings' decisions, including whether to make supervision requirements. In order to maintain the anonymity of young people, the Youth Court co-ordinator forwarded the name, date of birth, SCRO number 7 and date of first appearance in the Youth Court directly to SCRA. This enabled SCRA to provide the researchers with individually anonymised data from RAD. It was not possible to link this information to other records collected. The analysis focused upon all referrals recorded on RAD while taking some account of the variable length of time for which information was available in individual cases. Because information was not available prior to 2002, the data are likely, if anything to underestimate the extent of historic involvement with the Children's Hearing System, especially in Hamilton where the relevant data were accessed in February 2005. A similar process was undertaken for Airdrie Youth Court cases in February 2006.
Analysis of social work data
2.5 To gain an insight into the situation of young people at time of sentencing in the Youth Court, Social Enquiry Reports ( SERs) written by social workers in one of the local authorities serving the Hamilton Youth Court were reviewed. Including supplementary SERs (produced for reviews and deferred sentences), this local authority had produced at least 402 reports for the Youth Court up to the end of December 2004, an average of 21 reports per month. The aim was to study the young person's first SER written for the court. However, in a few cases only later reports were made available. Details of the information extracted from SERs are provided in the Appendix. It should be recognised that SERs are written in a specific context (to inform a Sheriff at sentence) and thus the information presented here is indicative rather than a definitive record of the young people's situations.
2.6 It was also intended that information be gathered on the characteristics of young people convicted in the Airdrie Youth Court and the services provided to those made subject to probation orders and structured deferred sentences. While some basic demographic data (age, sex) was available from the Youth Court database, it was envisaged that information held by the social work department would provide for a richer profile of Youth Court cases. Requirements of the Data Protection Act meant that it was not possible to access social work files directly. Instead, it was agreed that relevant data could be provided anonymously from the Social Work Information System ( SWIS) in an electronic format. However it transpired that the information required was not in electronic format and that extracting it manually would be a very time-consuming exercise that was not feasible within the resource and time constraints on the research.
2.7 Alternative approaches were taken to secure additional information about the services provided to young people who were sentenced in the Youth Court. First, the groupwork project to which most young people given supervisory disposals in the Youth Court were referred (Community Alternatives) provided anonymised information about the services made available to young people. Second, one of the Youth Court social workers provided anonymised information about the characteristics of and services provided to a sample of 39 young people who had been sentenced in the Youth Court between July and December 2005. Third, information on the background of all young people given orders (probation orders, community service orders and restriction of liberty orders) by the Airdrie Youth Court was provided by the Youth Justice Social Work Co-ordinator from North Lanarkshire Council. These data referred to Social Enquiry Reports completed prior to sentencing throughout the pilot period of the Airdrie Youth Court and related to 118 orders imposed on 90 young people. Although these approaches cannot be said to provide a comprehensive or representative overview of Youth Court cases and the interventions undertaken with them, the data thus generated are included to provide a flavour of aspects of the circumstances of the young people sentenced in the Youth Court.
Interviews with professionals
2.8 During the first 6 months of operation of the Youth Courts, interviews were conducted with a range of professionals associated with its operation. A further round of interviews was conducted after the pilot Youth Courts had been operational for around 18 months. For the most part, the same respondents participated in both rounds of interviews. Where this did not occur, it was as a result of personnel changes in the agencies concerned. The purpose of these interviews was to elicit views about the operation and effectiveness of the Youth Court and its associated processes. In total 32 and 41 professional interviews were conducted in Hamilton during the first 6 months and after 18 months respectively. The comparable number of professional interviews in Airdrie were 31 and 32. A summary of those interviewed is presented in the Appendix. Subject to the agreement of respondents, all of the interviews were tape recorded and transcribed.
Interviews with young people
2.9 Through their social workers young people who had appeared in the Youth Courts were asked to consent to participate in a research interview. The purpose of these interviews was to elicit young people's experiences of and views about the Youth Courts and the services they received through them. Interviews with young people, arranged through the relevant social worker, took place either in the social work offices or, more rarely, the young person's home. Nearly all the interviews were conducted in private, with the social worker present during only one. In total 27 young people in Hamilton consented to being interviewed and 19 did not. It proved possible to interview 23 young people (18 male, 5 female), 21 of whom were on probation from the Youth Court and 2 of whom were engaged with social work as part of a structured deferred sentence. Overall 9 young people were interviewed at the start of their order (within the first few months), 9 in the middle part of their orders and 8 towards the end of the order (with 3 interviewed both at the beginning and the end).
2.10 Five interviews with young people who were made subject to supervision from Airdrie Youth Court took place between October and December 2005. It had not been intended that interviews with young people would form a substantial part of the Airdrie Youth Court evaluation. Social work staff reported a significant refusal rate amongst the young people who were approached to participate in the research. Three other young people consented to be interviewed but employment and others commitments served as an obstacle to participation.
2.11 Whilst it would have been preferable to have had more control over the composition of the sample, recruitment of interviewees proved challenging and a pragmatic approach had to be adopted, with interviews arranged through the supervising social worker. This restricted the sample to those who were currently engaged with the social work department (that is, those who had breached their Orders were not included) and their views cannot be considered representative of all young people made subject to supervisory orders through the Youth Courts.
Observation of the Youth Courts in operation
2.12 Detailed observation of Hamilton Youth Court was made between November 2003 and July 2004. In total the business of 31 days and 200 case stages was recorded by 4 of the research team. This represents approximately one sixth (17%) of all Youth Court business during that period. Five Sheriffs sat in the Youth Court during the observations. More limited observation was undertaken in Airdrie where 2 of the Youth Court Sheriffs were observed over 7 days between September 2004 and October 2005 involving 145 separate case stages. In both courts observation covered the full range of court business from first callings through intermediate diets and trials to sentence, reviews and breaches.
2.13 A pro forma observation schedule was used to record the court sessions observed. It included details of those present, the duration of each session, the content and nature of the interactions between the various parties (Sheriffs, offenders, Procurators Fiscal, defence agents and social workers) and the proportion of time in which the bench and the offender were engaged directly in a dialogue. Information recorded was processed in 2 ways: firstly quantitative information on the proceedings was entered in SPSS for analysis; secondly qualitative descriptions of what occurred during the Youth Court were written and subsequently coded for analysis.
Analysis of the progress of Airdrie Sheriff Summary cases
2.14 One of the aims of the Youth Court is to enable the fast-tracking of cases to and through the court. An exercise was therefore undertaken to compare the timescales of Youth Court cases with cases going through the Airdrie Sheriff Summary Court. Following discussions with the Airdrie Sheriff Court clerks, the Scottish Court Service were approached to obtain information about the timeframes of case processing, stages at which guilty pleas were entered etc. in relation to adult cases calling at Airdrie Sheriff Summary Court during the pilot period. The Scottish Court Service were unable to resource this task to completion so it was therefore decided to gather this information from the court sheets and associated hard copy documents held at the court itself. Due to the resource intensive and time consuming nature of such a task, rather than examine all cases, a decision was made to sample between 100 and 150 cases brought before the citation court during the pilot period. New citation cases call in Airdrie Sheriff Summary Citation Court for the first time every second Tuesday. Sampling the cases calling at all of the diets falling within a particular month (sufficiently historical to allow the completion of most cases) during the pilot period was seen as the preferable method of gathering this information. The month of March 2005 was selected, yielding 3 diets and a total of 140 cases relating to 148 individuals 8. This constituted a sufficiently sized sample to compare usefully with the data from the Airdrie Youth Court.
Analysis of documentary material
2.15 Relevant documentary material was scrutinised to obtain insights into the operation of the Youth Court and to identify operational issues arising during the period of implementation and early operation. This included the Youth Court Information and Reference Document, minutes of the Implementation Group Meetings (the multi-agency Youth Court Advisory Fora) and regular statistical summaries complied by the Youth Court Co-ordinator. A member of the research team was invited to attend the regular multi-agency Implementation Group meetings for the Hamilton Sheriff Youth Court. This offered an opportunity to identify emerging issues and how they were addressed.
OUTCOME EVALUATION
Survey of community attitudes to youth crime
2.16 One of the objectives of the Youth Court is 'to enhance community safety by reducing the harm caused to victims of crime, and providing respite to those communities which are experiencing high levels of crime'. To evaluate this, a baseline and follow-up study of members of the public living within the Hamilton Youth Court jurisdiction was undertaken. The overarching aim was to measure any changes in public perceptions of crime and confidence in the judicial system. To do so, the study measured changes in: perceived patterns of offending locally; fear of crime; actual experience of crime; perceptions of the criminal justice system; and awareness of the Youth Court and its effectiveness.
2.17 The fieldwork for the baseline study was undertaken between mid-September and early November 2003 and the fieldwork for the follow-up survey was undertaken approximately 16 months later, between mid-January and mid-February 2005. A total of 1069 individuals were interviewed, 541 in the baseline and 528 in the follow-up. Further details of the methodology are presented in the report of the survey which is presented as an Appendix to this report.
Analysis of sentencing patterns following the introduction of the Youth Courts
2.18 To assess whether the introduction of the pilot Youth Courts had an impact on sentencing patterns of the target age group, the Scottish Executive's Justice Statistics Unit provided details of sentencing in Hamilton Sheriff Court in 2002, 2003 and 2004 and in Airdrie Youth Court in 2003, 2004 and 2005. In the analysis 2002 was treated as the baseline year in Hamilton and 2003 as the baseline year in Airdrie since in these years all young people had their summary proceedings heard in the normal adult summary court. 2003 and 2004 were transitional years in Hamilton and Airdrie respectively with the introduction of the Youth Court pilots taking place mid-way through the year. This analysis focused on the impact of the Youth Court on summary level sentencing of 15-17 year olds with a view to identifying whether the absolute and proportionate use of different disposals had changed following the introduction of the Youth Court.
Analysis of changes in recorded crime
2.19 To assess whether the operation of the Youth Courts had impacted upon recorded crime levels in the area covered by the, recorded crime figures for the 2002 and 2004 calendar years were requested from Strathclyde Police for the operational areas serving Hamilton and Ayr Sheriff Courts and from Central Scotland Police for the region covered by Falkirk Sheriff Court. Comparable data were obtained for the police areas covering Airdrie Sheriff Court and the comparator courts for 2003 and 2005. The comparison courts were selected on the basis of their local authorities having a similar socio-economic profile to North and South Lanarkshire, the local authorities serving the Airdrie and Hamilton Sheriff Youth Courts. The analysis focused upon the full year prior to the introduction of the Youth Courts and the first full year of the Youth Courts' operation.
2.20 Recorded crime statistics are dependent on the level of crime reported to, and subsequently recorded by, the police. Evidence from the 2003 Scottish Crime Survey suggests that 49 per cent of crimes in 2002 were reported to the police (McVie et al., 2004). Historically reported crimes were only subsequently recorded as crimes if the police deemed there to be evidence of a crime having taken place. However, under the new Scottish Crime Recording Standard introduced for the 2004/05 financial year the recording of crime became victim led, meaning that reported incidents were more likely to be recorded as crimes ( HMIC, 2003). Obviously these changes in police recording made direct comparison of data for before and after this figures problematic as like with like were not being compared. The greatest impact of the new standard was predicted to be on minor crimes such as vandalism; the type of offence commonly dealt with in a summary court. However it should also be recognised that even if there had not been a change in recording standards any changes in recorded crime could be caused by a variety of factors independent of the impact of any changing court process.
Monitoring the progress of young people under supervision
Individual case discussion with social workers
2.21 To obtain an indication of the progress made by the Hamilton Youth Court cases they supervised, almost all of the social workers in South Lanarkshire's Youth Justice Team were interviewed about their individual clients. This method allowed a large number of cases (45) to be covered whilst minimising the impact on individual workers. It supplanted earlier attempts to encourage social workers to complete written questionnaires on individual young people, which had met with limited success 9. Young people sentenced by the court to a period of probation or deferred sentence with social work involvement (a structured deferred sentence) were included.
Completion of questionnaires by supervising social workers
2.22 Social workers involved in the supervision of young people given Probation Orders or Structured Deferred Sentences by the Airdrie Youth Court were invited to complete questionnaires regarding the focus of intervention and the young person's progress approximately 6 months after the making of the Order. As had previously occurred in Hamilton 10, attempts to encourage social workers to complete the questionnaires met with limited success. Twenty questionnaires in total were completed (16 relating to probationers and 4 to young people subject to structured deferred sentences), mostly by the dedicated Youth Court social workers, and they cannot therefore be viewed as a definitive record of all work that took place. As with the social worker interview in Hamilton, however, they provide some indication of the focus and perceived effectiveness of the interventions with which young people were engaged.
Analysis of reconviction data
2.23 To examine whether the Youth Court had an impact upon recidivism among those appearing before it, aggregate rates of reconviction were assessed for young people sentenced in the Youth Courts, in the Sheriff Summary Courts at Hamilton and Airdrie and in 2 other comparator courts: Ayr Sheriff Court and Falkirk Sheriff Court. As with the recorded crime data, the comparison courts were selected on the basis of their local authorities having a similar socio-economic profile to North and South Lanarkshire, the local authorities serving the Airdrie and Hamilton Sheriff Youth Courts. The Scottish Executive Justice Statistics Unit provided information on convictions among those sentenced in these courts (June 2003 to May 2005 in Hamilton and June 2004 to May 2005 in Airdrie. The data provided was for convictions up to the end of 2005. Because of the short time frame of the study, these data were provisional and incomplete. It was possible only to obtain details of reconvictions (excluding pseudo reconvictions 11) within 6 or, at most, 12 months of sentencing in the Youth Court. A two-year follow-up period following adjudication is generally accepted as being required to provide a more accurate picture of the impact of different disposals upon recidivism.