CHAPTER FIVE: Review
Chapter 4 established the importance of the right individuals having ownership and being held accountable for established priorities. The need therefore arises to ensure that there are structured processes in place to monitor and review performance against these priorities. This is a particularly crucial area if organisations are to have confidence that they are achieving their priorities, identifying areas for improvement and sharing good practice. Elements of team and individual performance appraisal have already been looked at, but the focus now turns to performance at an organisational level.
Performance review meetings are considered to be a key aspect of a performance management system. The purpose of the review meeting is to focus on organisational priorities and assess performance to date. This allows areas of under performance or good performance to be identified. It is in this way that performance monitoring becomes performance management. Decisions that are taken at these meetings move the organisation from simply monitoring its performance to managing it by using the information provided to make informed decisions about future actions.
For the meetings to be effective they must be well informed and relevant. Having said that, there is clearly a need to ensure that performance against priorities is being monitored in a manner which does not involve a heavily bureaucratic process and that only indicators which will inform on progress of priorities are being monitored.
A necessary ingredient of an organisation which demonstrates a strong performance management culture is a structured review process, ranging from the strategic to the operational level.
Matters arising from such meetings are progressed and thereafter reviewed at subsequent meetings. Much of the literature in relation to performance management highlights the need for 'relentless follow up' of actions to ensure that performance improves or evidence is documented as to why improved performance is not being achieved. Where performance against a declared priority is not being achieved, it may be that there are other factors which now exist that make the achievement of the said priority unrealistic. However, through a robust review structure, the organisation can evidence its decision making process and also has an opportunity to confirm or amend its priorities as necessary.
5.1 Link to NIM
Within police forces, a natural link exists between performance meetings and tasking and co-ordinating groups convened in accordance with the NIM, with an overlap of information used by each. HMIC found that the majority of forces have, or intend to have, separate force level tasking and co-ordinating group and performance meetings. The general view being expressed is that some performance areas are beyond the remit of tasking and co-ordinating, the focus of which should remain firmly operational.
Naturally, actions from the performance meeting which relate to operational matters should be given a high priority by the tasking and co-ordinating group. However, the imperative to pursue a force or national performance target should not be allowed to take precedence automatically over a local issue identified through the NIM process. If operational priorities are properly aligned to the NIM control strategy, the potential for such conflict is reduced.
5.2 Review Structures - Force
HMIC noted during inspection that different forces use slightly different review processes and there is no one structure better than the others. What matters most in deciding on a structure is what suits the organisation concerned. Factors which might usefully be considered include organisational size, geographical spread of resources, spans of responsibility and perceived need in terms of frequency and desired attendance.
Within a policing context, 'Compstat' is perhaps the most familiar example of a review process. That said, a considerable mythology now surrounds the process since it was first introduced in New York, in 1994, which detracts from what, at a basic level, is simply a data driven, problem solving approach combined with rigorous challenge. Without offering a view on its overall efficacy, HMIC considers that a brief description of the key processes nonetheless adds value to the issue under current discussion.
'Compstat', as operated by the New York Police Department, is widely seen as the seminal case study of local, real-time performance management leading to performance improvement. That said, its success in reducing crime has also been attributed to other issues including an increase in police numbers, implementation of the 'Broken Windows' theory and a rise in economic growth.
'Compstat' involves four key steps in a weekly cycle:
- At a local precinct level, data is captured and problem solving conducted.
- Central analysis and quality assurance then takes place via a 25 person team.
- This paves the way for the weekly accountability meeting, during which precinct commanders are challenged by a high level management panel, on their efforts to reduce crime. Analysis of crime patterns is pursued in detail with an emphasis on ensuring that robust crime reduction plans are in place.
- Senior representatives of specialist units and other support agencies are also present and held to account against their commitments to crime reduction initiatives.
Among the myths surrounding the 'Compstat' process, much weight is given to the role that IT plays. IT was not a pre-requisite for success, indeed manual data entry was the norm at the outset and most precinct level analysis was done by hand. As the system has matured, greater use has been made of IT based systems. The critical success factor was the collection of timely, robust data.
Equally, peer comparison may not necessarily be the critical incentive that it is often perceived to be. In practice, the key challenge for precinct commanders is to make year-on-year improvements against historic results.
In UK policing terms, 'Compstat' has erroneously become synonymous with a confrontational review meeting, rather than as a whole process ensuring accountability. However HMIC notes that, where a similar review meeting process has been adopted in Scotland and observed elsewhere in the UK, organisations have shown a tendency to develop their meetings in a more inclusive and supportive style, which nevertheless remain challenging and deliver strong accountability.
The scheduling of review meetings must also take into consideration whether sufficient time has elapsed for any actions arising to have been resourced and supported for progress to be made, while also recognising the potential loss of effectiveness if there are lengthy gaps between meetings. In Scotland, existing performance review meetings are generally being held on a monthly basis, either at area command and/or at force level, to prevent the latter occurring. As well as monitoring performance since the previous meeting, consideration is also given to emerging trends over a longer time frame. HMIC considers that performance reviews over a longer period risk losing their ability to address under-performance effectively.
HMIC noted a common theme across forces and common police services was the involvement of chief officers and directors in the performance review process. This illustrates leadership and commitment and helps to ensure that the process is focused on the attainment of the goals of the organisation. The review processes involved are tending to evolve over time as forces determine which method best suits them.
All forces have recognised the need to have processes in place through which they can review performance, particularly in respect of operational command areas. The process of reviewing the performance of support departments or business areas against organisational priorities is less well established. The development of service plans, as previously mentioned in Chapter 4, will assist in establishing suitable review processes. It should be noted that programme boards, or similar, perform a review function for strategic priorities, generally along project management lines.
The examples highlighted below illustrate the different types of review processes currently being used.
Lothian and Borders Police recently introduced a Performance Activity Management ( PAM) meeting. The meeting is currently held on a quarterly basis and is chaired by the Chief Constable and is attended by other members of the force executive, all area commanders and heads of relevant support department. Performance is reviewed against performance measures which have been identified either as priority areas for improvement or as high risk areas for the force. The force performance improvement unit produces a report for this meeting, outlining performance at force and command area level. Monthly performance reports are also produced to feed divisional performance meetings.
The PAM meeting is seen to be evolving into a discussion forum for improving force performance based on informed dialogue with individual area commanders and departmental heads, in turn allowing the meeting to concentrate on anomalies and best practice rather than 'business as usual'. While designed to be a challenging environment, the style of the meeting is not aggressively confrontational.
Fife Constabulary holds its Force Operations Group ( FOG) meeting, which is chaired by the Assistant Chief Constable, on a monthly basis. The role of the FOG is to ensure the achievement of force goals within the territorial and functional command areas of the force by considering data on progress against targets set in the policing plan and other relevant performance information, and identifying areas of underachievement.
In addition, the Assistant Chief Constable ensures accountability by visiting command areas and operational headquarters departments on a quarterly basis to focus on performance issues with the management teams. Also, the Deputy Chief Constable attends biannual meetings with departmental management teams (headquarters) to focus on how departments support operational performance within the command areas and how they perform against pre-set objectives. The Deputy Chief Constable then holds meetings with the command areas to assess the support given by headquarters functions.
Tayside Police similarly has a monthly Operational Commander's Meeting ( OCM) chaired by an assistant chief constable who also reviews individual divisions on a quarterly basis.
Dumfries and Galloway Constabulary conducts a daily video conference between commanders and the force executive at which ongoing performance issues can be addressed. Performance is a standing item at the monthly force executive policy group meeting and the Chief Superintendent also conducts a quarterly review of command area performance.
West Midlands Police holds a central performance meeting for its operational command units every 2 months.
Each assistant chief constable in the force has been allocated responsibility for the performance of a number of operational command units, ensuring full involvement of the force executive. On months when there is no central meeting, each assistant chief constable will hold a performance meeting with their cluster of operational command units, looking at cross border issues as well as individual unit performance. These meetings are more focused than the central force meeting.
It should be noted that, where a central performance meeting takes place, this can often involve a large number of senior managers. It is therefore vital that there is a clear agenda for the meeting, that matters discussed are relevant and that the business is conducted efficiently.
HMIC was pleased to note that forces actively pursue outstanding actions arising from previous meetings, using structured records of outstanding actions and of the individuals with responsibility for progressing the actions.
5.3 Review Structures - Command Area
It is equally important to review performance at a command area or departmental level. This provides managers at all levels with a focus on performance. As illustrated previously some command areas and departments utilise team and individual statistics to monitor performance.
At command area level HMIC has noted that the majority of forces do not hold separate performance meetings. Instead issues of performance are discussed at management team meetings and are fed into tasking and co-ordinating meetings. The majority of forces see tasking and co-ordinating as the key driver for command area performance, linked to individual performance meetings with team leaders. However, where there is no separate command area performance meeting, the previously stated benefits of separating performance from tasking and co-ordinating to ensure all aspects of performance are considered, should not be overlooked.
HMIC considers that the style of performance review used at this level is a matter for individual forces. What remains the key issue, however, is that command areas must regularly monitor and review their performance to ensure that they are achieving their desired outcomes, as well as identifying and addressing any areas for improvement.
5.4 Review Structures - Support Functions
With regard to the performance of support departments, HMIC found strong formalised structures for reviewing performance within the common police services, with individual departments having well established links to desired organisational outcomes. It was also found that the majority of forces had recognised the need to review the performance of their support departments in a more structured manner. Forces also acknowledged the need to develop the linkages between departmental and organisational priorities. This is demonstrated in the intention of Tayside Police to establish a Policy and Performance Committee.
Within the Scottish Criminal Record Office ( SCRO) the Director holds a monthly management meeting where each of the individual bureau heads are present and are held to account for bureau performance. Service area performance reports capture a range of performance information relevant to each bureau and set against agreed targets. The content of these reports is a constantly evolving process, as key indicators are identified and methods of capture improve.
A similar structure of management meetings is also used within the Scottish Police College and the SDEA.
5.5 Processes
The information provided to performance review meetings should serve to highlight areas within the organisation/force/department that are performing well and those which are not. When choosing indicators and measures of performance it is stressed that organisations should seek to keep these to a limited number and should focus on key areas of business.
By identifying and critically examining the key processes required to achieve the desired outcomes, an organisation can achieve performance improvements. Review of the processes involved could arise as an action from a performance review. Examining organisational processes as a means of achieving sustainable improvements can often lead to training implications for teams and individuals.
Areas for improvement can often be identified by staff working within particular departments, either as a result of something going wrong or simply, through an individual's experience in that role, identifying a better way to do business. This is every bit as valid as more formal methods.
The Scottish Executive Guidance to Chief Constables on Implementing Best Value in the Scottish Police Service 20 advocates that police authorities should agree a systematic approach to service reviews designed to achieve real improvements in resource use, focused on priority areas or areas of concern.
During inspection, HMIC noted evidence that Best Value reviews and schedules remain an important feature within forces' continuous improvement effort. The means by which the areas of work are identified and the manner in which reviews are managed, differ considerably. The level to which police authorities are engaged in the process is less clear, although most forces indicate in their responses to HMIC, that a dedicated Best Value Sub Committee is a feature of the authority structure.
Within Strathclyde Police, selected individuals at divisions and headquarters departments have been designated to carry out process mapping, supported by staff from Corporate Planning and Development. With the introduction of a new business continuity planning initiative, the Standard Operating Procedure specifies that it is incumbent on local business continuity teams to identify their processes in order to assess risk and prioritise management actions and business continuity plans.
The recent change in the role of the Deputy Divisional Commander within Strathclyde Police to ensure a stronger focus on performance management, brings with it a role in managing the performance of local processes. To this end, in February 2005 a workshop was held for Deputy Divisional Commanders to develop their understanding of process management and their role therein. This was closely followed by a process mapping workshop for divisional and departmental quality assurance staff.
In addition, the use of internal inspection by forces and common police services has often resulted in improved processes and procedures leading to performance improvement. The EFQM self assessment tool is still widely used in the Scottish police service, its use highlighted by the Scottish Police College obtaining an award from the European Foundation for Quality Management for Excellence in Europe in 2003. The balanced scorecard approach is also being developed by a number of forces, either as a means of reviewing a particular department or business area or, as in the case of the SDEA, SCRO and Grampian Police, as an integral part of their business planning cycle. Both methods allow the organisation or command areas and departments to identify processes or areas which would benefit from review.
HMIC considers the review of processes to be an important aspect of performance management. Ongoing examples of work in this area in Grampian Police and Lothian and Borders Police are worth noting.
Grampian Police and Lothian and Borders Police have both undertaken similar but separate projects to review criminal justice procedures, with a view to recommending and implementing changes that improve speed, efficiency and effectiveness of their local criminal justice systems.
The need to examine and improve upon the various processes embedded within a complex criminal justice system was recognised in the review of summary justice undertaken by a committee chaired by Sheriff Principal McInnes and the report by the then Crown Agent, Andrew Normand, on the integration of elements within the criminal justice system. Grampian Police and Lothian and Borders Police have engaged with their local Criminal Justice Boards, and partners in the projects and include Crown Office, the Procurator Fiscal Service and the Scottish Courts Service.
The inter-agency project teams employ a proven methodology originally developed in manufacturing but successfully adopted within the service sector. The fundamental principles of this approach are outlined in this abbreviated list:
CHECK
Understand demand.
Obtain clarity of purpose of the system from the customer's point of view.
Understand work flow i.e. identify value work and waste.
Identify the conditions that help or hinder performance.
PLAN
Identify value steps from the customer's point of view.
Redesign process flows.
Tackle the main system conditions e.g. new structure, new measures, new job roles.
DO
Experiment with demand using the redesigned process flows.
Act on system to remove system conditions.
Evaluate and return to 'check'.
This is a holistic approach which can provide an end-to-end view of the current capability of a system to meet its stated purpose. The approach should identify weaknesses at each stage of the criminal justice process and suggest possible solutions through to a re-design stage.
HMIC was sighted on the progress of both forces' projects. HMIC also saw the results of the process analysis which had included the gathering of end-to-end data from all agencies concerned. This allowed the project teams to understand demand, variation and system capacity with a view to designing out weaknesses.
Although at the early stages of practical implementation, HMIC commends this approach to tackling complex multi-agency issues and looks forward to seeing positive developments in criminal justice system processes.
Recommendation 8
HMIC recommends that forces and common police services ensure that they have structured processes which ensure that performance of the whole organisation, including support departments, is reviewed and that actions arising therefrom are appropriately pursued.
As well as reviewing areas of apparent poor performance, the examination of why performance is apparently good needs to be reviewed. Examination of a highly performing department or area command may lead to the identification of good practice that could be applied across the organisation or indeed the country.
ACPOS has recently introduced a Business Benefits Database which will be populated on an ongoing basis with good practice that has led to efficiency savings for the submitting force. A number of forces are attempting a similar approach at a force level.
All forces also feed into a national suggestion scheme which is currently facilitated by Strathclyde Police. In practice, facilitators for each force suggestion scheme submit what they consider to be notable suggestions to Strathclyde Police who, in turn, collate them into a single document which is then circulated to each facilitator. HMIC noted that contributions from some forces to this system were sporadic and that therefore it may not be realising its potential. Further, the process of introducing national suggestions into forces does not necessarily ensure that these suggestions are being given due consideration by the senior managers in every force. HMIC considers that there may be value in forces considering their contribution to the national suggestion scheme to ensure the sharing of information and potential good practice is maximised.
Internally, some forces are highlighting good operational practices or tactics on their intranets, in order to allow other area commands to examine their action plans and results achieved. At the conclusion of a particular action plan or operation, pressures upon forces to move on and tackle the next issue are great. However, the future savings and potential for the organisation to learn from a structured review should not be underestimated.
5.6 Best Value
All forces are committed to Best Value and this is highlighted by ACPOS through its Best Value Sub Committee and its annual report on Best Value. HMIC notes that the themes for Best Value reviews are often decided by chief officers following consultation with police authority members and, on occasion, police authority members are directly involved in the review process. Resultant efficiency benefits from force reviews are reported to police authorities to illustrate their achievement in securing Best Value. They are also submitted to the ACPOS Best Value Sub Committee and collated into an annual report on Best Value. It is also worthy of note that Audit Scotland is currently developing a Best Value inspection for specific use with police authorities. This promises to be a substantial development in the scrutiny of performance in the Scottish police service.
5.7 Benchmarking
The general concept of benchmarking is one of seeking improvement through the discovery of the methods used to achieve superior performance elsewhere. One of the common themes in benchmarking is the identification of others from whom one might learn either within an organisation or from outside. The choice of a benchmarking partner is therefore crucial to the process.
The Audit Scotland guide for councils on Best Value 21 states:
"No league tables or scores: The emphasis on local context recognises the difficulty of making comparisons between councils. In addition councils are large and complex organisations, with a range of strengths and weaknesses. Therefore, there will not be any use of simplistic league tables or overall labels for each council. Instead, reports should provide a robust assessment of each council's strengths and weaknesses, focusing on the need for specific improvements."
It should be noted that the statutory guidance on Best Value under the Local Government in Scotland Act 2003 22 states that an authority should demonstrate sound governance at a strategic, financial and operational level by:
"effective performance management systems, which include the use of external comparison, through which performance issues can be identified, monitored and addressed."
The value that can be derived from benchmarking cannot, therefore, reasonably be taken from broad organisational comparison but from a more focused comparison of individual areas of performance. For example, some forces feel that benchmarking has been beneficial in certain areas where there is strong commonality, such as in property or fleet management.
The use of benchmarking in police circles is probably best known in the 'most similar families' methodology used in England and Wales. The Policing Performance Assessment Framework ( PPAF) has been developed by the Home Office in conjunction with ACPO, the Association of Police Authorities ( APA) and HMIC in England and Wales and is intended to provide a mechanism to make rigorous and fair assessments of performance within policing.
The framework has been designed to assess performance across the breadth of policing responsibilities and attempts to include the range of activities that the police are asked to undertake. In order to achieve a balanced assessment, policing responsibilities have been divided into six outcome areas or domains, which recognise the importance of priorities set locally and nationally.
Comparison via the 'most similar' forces, basic command unit (England and Wales) or command area (Scotland) is in itself a contentious issue generating much debate from practitioners as to the validity of such comparisons. This is explored in Chapter 6.
The value of external benchmarking has, however, been questioned by some academics on the basis that it encourages imitation rather than innovation. Instead it is advocated that the organisation's own people are used to review working practices, which engenders a better understanding of what does or does not work and leads to longer term improvements.
HMIC considers that there can be value in both internal and external comparison, although any data or information used must be put in context and consideration given to that context before any conclusions are drawn. In addition, broad organisational comparisons should be avoided and focused comparisons sought regarding individual departments or processes as a means to improve performance. Failure to do so can lead to the development of league tables which are presented without context.