Safer and Stronger: Summary Justice Reform: System Performance, Monitoring and Evaluation

Listen

Monitoring and Evaluating the Impact of the Reforms to Summary Justice

28. The Scottish Government, in close collaboration with our partner criminal justice organisations, has devised and implemented a comprehensive monitoring and evaluation framework to ensure that the impact of the reforms to summary justice can be measured. The Framework comprises three closely related strands of work:

  • The commissioning of a suite of seven evaluations of specific aspects of the reforms;
  • The collection of high level key performance information; and
  • The collection of lower level, more detailed contextual monitoring data.

Evaluation

29. A comprehensive four-year programme of work has been put in place to evaluate each of the key aspects of the reforms to summary justice. Each of these aspects of the reforms will be evaluated at multiple levels of:

  • Their own specific policy objectives;
  • The contribution they make to the realisation of the intended outcomes of SJR; and
  • The contribution they make to the overarching objectives of SJR.

30. Both the intended outcomes of SJR and the overarching objectives are taken to be those identified in the System Model paper. The overarching objectives are specified as being the intention to create a summary justice system that is Fair, Effective, Efficient and Quick and simple in delivery (c.f page 4 above). The intended outcomes of SJR are taken to be:

  • The removal of a significant number of appropriate cases from the court through a greater use of non-court options or alternatives to prosecution including adult warnings, fixed penalty notices and procurator fiscal Direct Measures.
  • For cases that do come to court, those cases will come to court more quickly.
  • Improved case handling, namely:
    • Early, effective preparation of cases;
    • More effective court hearings;
    • Cases will be dealt with at the earliest possible stage in proceedings.
  • Appropriate allocation of cases to forum, including sufficient use of better trained lay justices.
  • To make a contribution to reducing re-offending by dealing with cases at the earliest possible stage in proceedings.
  • To reduce inconvenience for victims and witnesses.

Those aspects of the reforms that will be subject to detailed evaluation are:

  • Direct Measures
  • Summary Criminal Legal Assistance and Disclosure
  • Fines Enforcement
  • Bail and Undertakings
  • Lay Justice
  • Victims, Witnesses and Public Perceptions; and
  • Summary Justice System Performance.

31. For the most part the evaluations are being externally commissioned. The projects are being widely advertised to attract expressions of interest, which are then sifted for invitation to tender. Selective competitive tendering exercises are being undertaken to ensure the appointment of experienced independent researchers who are able to offer value for money.

32. It is necessary to allow the reforms time to bed in and for robust data to become available before the evaluations can begin. As such, for some of the evaluations (and in particular the system wide evaluation) a formative approach is being adopted. 10

33. The focus of each evaluation and the planned timetable is as follows:

  • Direct Measures - this study will focus on the use of both police and the extended range of procurator fiscal direct measures and their impact on the overall summary criminal caseload. The evaluation commenced in April 2009 and will report in November 2010.
  • Summary Criminal Legal Aid and Disclosure - the legal aid element of this evaluation will focus on the impact of the reforms to the structure of summary criminal legal assistance on accused and defence agent practice and the trajectory of cases. The disclosure of evidence aspects will look at how the disclosure provisions are working in practice and assess their impact on the progress of cases. This study will commence in July 2009 and report in August 2011.
  • Fines Enforcement - This study will look at the way in which the payment of all types of financial penalties (both direct measures and court imposed fines) is enforced and assess the impact of the introduction of Fines Enforcement Officers on the processes for enforcing payment. The study will commence in the summer of 2009, adopting a formative approach, and will report around May 2011.
  • Bail and Undertakings - The bail element of this study will evaluate the extent to which the reforms to bail are operating as intended and their impact on the behaviour of accused on bail and their attitudes towards bail. The undertakings aspect will look at the extent to which the undertakings provisions are operating as intended and their impact on the progress of cases. It is anticipated that this study will commence at the start of 2010 and will report in the autumn of 2011.
  • Lay Justice - This study will focus on the impact of the reforms to the way in which lay justices are recruited and trained, the type and volume of cases they preside over and perceptions of JP courts. It will also consider issues of access to justice. The study will commence in mid 2010 and will report towards the end of 2011.
  • Victims, Witnesses and Public Perceptions - This study will explore the experiences of victims and witnesses and the views of victims, witnesses and the general public on each of the key elements of the reforms (with the exception of public views on the changes to legal aid and disclosure). It will include victims whose cases resulted in a direct measure as well as those whose cases were prosecuted in court. It will also include civilian, police and expert witnesses. This study will commence in September 2009 and report in August 2011.
  • Evaluation of Summary Justice System Performance - this study will aim to evaluate the impact of the reform package as a whole on summary justice system performance and will attempt to assess the extent to which each of the key aspects of the reforms have contributed to the desired outcomes and overarching objectives of the reforms. Full scoping of this project, including the timetable, has yet to be undertaken.

34. Together, this suite of evaluations represents a complex and closely inter-linked programme of work. A variety of existing data sources will require to be accessed, including the routine KPI and monitoring data contained in the MI System (see below) and the collection and analysis of ad hoc data sets from the criminal justice organisations' operational IT systems. A variety of analytical methods will also require to be utilised including interviews with criminal justice system practitioners and those with experience of crime (whether as victims, witnesses or accused); surveys; focus groups and court observation. A key challenge facing the research teams will be in attributing observed outcomes to specific aspects of the reforms since each of the outcomes might be influenced by a number of the reforms or other factors. A sum in the region of £600,000 has been committed by the Scottish Government to fund the independent evaluation of the specific aspects of the reforms.

Key Performance Indicators

35. The National and Local Criminal Justice Boards have access to a web-based Management Information ( MI) System containing a range of Key Performance Indicators ( KPIs) on the criminal justice system. The MI System contains 34 KPIs focusing on the four key stages of the criminal justice system of 'caution and charge to report', 'case marking', 'court' and 'disposal'. (See the following section on Post- SJR System Performance for KPI data covering the first year since implementation of the reforms.) Data are supplied by the key criminal justice organisations (the eight Scottish Police Forces (via the Association of Chief Police Officers in Scotland ( ACPOS)), COPFS, SCS, SLAB and SPS) and are uploaded onto the MI System on a monthly basis. The secure system is available to criminal justice practitioners within each Local Criminal Justice Board ( LCJB) area and provides a single source of cross-agency system performance data. This enables the LCJBs to see snapshots of system-wide performance and also to identify particular points in the criminal justice process where the process appears not to be functioning as desired. A number of the KPIs in the system relate to agreed national targets, such as the Scotland Performs target for dealing with summary criminal cases within 26 weeks 11 and the target for submission of SPRs by the Police to COPFS. 12 Other data in the system allows close monitoring of the reforms contained in the Criminal Proceedings etc. (Reform) (Scotland) Act 2007, such as the use of police direct measures, the use of the new procurator fiscal direct measures; and the combined effect of the reforms on the point at which cases are concluded. The MI System contains close to real time data (i.e. at the end of any given calendar month data pertaining to the previous month will be available on the system) and contains information dating back to April 2006.

Monitoring Indicators

36. In addition to the KPIs the MI System contains data on a set of detailed SJR Monitoring Indicators. This information provides context to the KPIs and focuses on the detail of the reforms such as the nature of offences for which police Fixed Penalty Notices for Anti-social Behaviour are issued, the level of fiscal fines issued, the number of pleading, intermediate and trial diets held in the JP and sheriff summary courts and the number of fines requiring enforcement action. The SJR monitoring data is being added to the MI System on a phased basis as work is completed by the key criminal justice organisations (again, these are ACPOS, COPFS, SCS, SLAB and SPS) to extract the data from their operational IT systems. In addition, data for a number of the monitoring indicators are being extracted by Scottish Government Statisticians from data provided on a routine basis by COPFS on cases opened and cases closed by COPFS each month. The Monitoring System contains a total of 89 SJR related indicators and, at present, information dating back to April 2008 (when the reforms were implemented) is available for half of the indicators. Data for the majority of indicators will be available by the end of September 2009.

37. Taken together the KPI data in the MI System and the SJR Monitoring workbook are intended to enable the National and Local Criminal Justice Boards to analyse and understand system performance. The data can be mined at multiple levels down to individual court level, enabling local boards to explore specific aspects of performance in detail, identify good practice and examine the reasons for less than optimum performance. Local Boards can also explore how performance in their own area relates to performance in other parts of Scotland.

38. As well as being available to the National and Local Criminal Justice Boards, the data in both the MI System and the SJR Monitoring workbook will be made available to the researchers undertaking the independent evaluations of the various aspects of the reforms. Summary data will also be put into the public domain in a variety of formats.

39. The extensive work underway to monitor and evaluate the impact of the reforms to summary justice is being undertaken within a formal project management structure. The structure adopted is set out in Annex A.

Page updated: Friday, July 10, 2009