Scottish Government

MAIN FINDINGS

• The Pathfinder projects found it difficult to create new models of basic service delivery. Some successful new models have emerged which have brought tangible benefits to their communities. Generally however, rather than testing out new models of service delivery, the Pathfinders have, in response to the needs identified by the community, provided new, additional services. These services were generally of positive benefit to their neighbourhoods.
• There were various reasons for this - most notably the community-based structure of the Pathfinders. The Pathfinders were very successful at engaging with the community and identifying its priorities, but found it more difficult to achieve the high level influence that was needed if these quite radical changes in basic service delivery were to be achieved. The emphasis on community consultation also created pressures for additional services, as many of the issues highlighted by the community appeared to call for new or additional services.
• The approaches suggested in the prospectus to improve service delivery were tested by the Pathfinders at least to some degree, although not all were fully developed. For example, although several Pathfinders focused on neighbourhood management issues, none brought the management of core local services under a unified management structure accountable to the local community and none employed a local services manager or neighbourhood manager as such, with one partial exception.
• Although the fit with the overall programme objectives was weak in some areas, the Pathfinders funded services which had a significant positive impact in their localities, and fitted well with the social justice targets. Several pathfinders used innovative and effective approaches to engaging with the community and in establishing community priorities and a number of the Pathfinder projects were both innovative and outstandingly successful in improving service delivery.

DTZ Pieda Consulting was commissioned by the Scottish Executive to carry out a final evaluation of the Working for Communities (WfC) Programme. The aim of the evaluation was to assess the extent to which the WfC Programme as a whole met its objectives, to assess how individual Pathfinders operated in relation to the Programme aims, to examine how Programme funds were spent, and to make recommendations for how future programmes are designed and managed.

ABOUT THE WFC PROGRAMME

WfC was a relatively small-scale three year pilot programme. The overall objective of the Programme was to provide funding to test out innovative new models of delivering services to local communities. Following a competitive bidding process 13 Pathfinders were supported, diverse in location and focus - taking in both urban as well as rural areas across Scotland - and addressing specific groups such as young people as well as whole communities and their diverse needs.

The Scottish Executive allocated 5.25 million to the thirteen pathfinders. Individual annual Pathfinder budgets varied from a minimum of 184k for the Stornoway and Barra Pathfinder to 666k for the Auchenback Pathfinder, with an average expenditure of 386k.

The prospectus for the WfC programme indicated the types of projects it might support - unified neighbourhood management services, multidisciplinary teams to provide services to families and children, one-stop shops, pooled budgets, new structures to deliver services, and service level agreements, and new ways of testing community opinion of local services.

The aim was to support innovation, particularly innovation which emerges at a local level as a result of a local analysis of local problems and needs, projects which challenge orthodox models of service delivery and to test out innovative approaches to delivering 'joined up' services and to create flexible, responsive services for local people.

The expectation was that local people would be involved in setting up new ways of delivering the basic local services. A number of other criteria were set including replicability elsewhere. The prospectus also contained references to a number of emerging policy areas which it was hoped the WfC Pathfinder might pilot and develop further: neighbourhood management, community participation, and a wider role for housing associations.

KEY FINDINGS

Despite the overall objective of the Programme and references in the prospectus to improving 'basic' or 'core' local services, while almost all of the projects provided new services in their communities, very few of them tested new models of delivery, and fewer still related to the delivery of basic or core services.

With a few exceptions, the Pathfinder projects found it difficult to create new models of basic service delivery, although they sought to influence the delivery of basic services to their neighbourhood. Generally however, rather than testing out new models of service delivery, the Pathfinders have, in response to the needs identified by the community, provided new additional services. These services generally brought positive benefits and fitted well with the social justice targets, but were less likely to be sustainable post-Pathfinder.

There were various reasons for this, of which the most important was the structure of the Pathfinders, which were community based and generally staffed by co-ordinators with a community development background. The co-ordinators were very successful at engaging with the community and identifying its priorities, but found it more difficult to achieve the high level influence that was needed if these quite radical changes in basic service delivery were to be achieved. Other pressures also impact on how local authorities and other partners deliver services and in many cases service providers appear to have been reluctant or unable to fully embrace the Pathfinder approach to service delivery.

The emphasis on community consultation also created pressures for additional services, as many of the issues highlighted by the community appeared to call for new or additional services.

Several pathfinders used innovative and effective approaches to engaging with the community and in establishing community priorities and a number of the Pathfinder projects were both innovative and outstandingly successful. For example:

Youth Tracker in Greater Easterhouse provided a seamless support and referral service for vulnerable young people aged 15 to 18 leaving school (school non-attenders, early school leavers, young people in social work day care and school leavers with unknown destinations). Professionals from over a dozen agencies were involved in setting up the service. The model influenced the development of the Glasgow Inclusiveness project - a Glasgow wide project with similar objectives.

The Community Health Shop in Easterhouse was a very successful community-managed initiative which addressed a particularly poor level of service provision. The major success of this initiative was the holistic range of services which were provided by health and other service providers. The CHS was well used and highly rated by users and service providers alike.

The Inverclyde Pathfinder made services more responsive to the needs of users, and introduced a new approach to service delivery. The approach adopted was not to fill gaps in existing service provision but to identify ways that existing services could be better co-ordinated. The production of the Inverclyde Youth Strategy linked with more collaborative working across the professions led to improvements in the delivery of services to young people, including supported accommodation, the careers service and a number of other services.

The Media Unit in the Great Northern Partnership was an innovative way of engaging the public in a way that influenced service providers. Based in a local community centre, the Unit worked with small groups and individuals, helping them to research and articulate local concerns. Video was used to gather local views, as well as capturing evidence of local problems and conditions. The Unit then worked with the community on editing to produce a professional video. The videos were an effective way of highlighting the concerns of the community to service providers.

Most of the suggestions for approaches set out in the programme prospectus were tested by the Pathfinders at least to some degree, although not all were fully developed. The most notable exceptions were unified local neighbourhood management and pooled budgets. Although several of the Pathfinders focused on neighbourhood management issues, introducing new services such as estate wardens/caretakers and various mechanisms to make local services more responsive to local needs, none brought the management of core local services devolved to a neighbourhood level under a unified management structure accountable to the local community. It was perhaps surprising that none of the projects employed a local services manager, identified in the prospectus as being one of the potential uses of the funding. None of the Pathfinders employed a neighbourhood manager as such, although one of the projects employed a worker who played a limited neighbourhood manager role.

Pooled budgets were not explored at all, and although there were a number of successful 'one-stop shop' projects, (both virtual and actual) they mainly involved a limited range of services, not the broad range of services envisaged in the prospectus. For example, the Easterhouse Health Shop which provides a very wide range of health related services, together with other supporting services (benefits, training, advice etc.) but not the full range of local basic services including cleansing, environmental services and housing. The main role of the virtual one-stop shops was information provision, although the main benefit of the Lochaber project was the access to IT facilities it gave isolated communities.

RECOMMENDATIONS FOR DESIGN AND MANAGEMENT OF FUTURE PROGRAMMES

Although it achieved much that was worthwhile, the programme drifted from what had been proposed in the prospectus. There is a need for improved communication between the Scottish Executive and participating authorities on the intended aims and the implementation of programmes in line with their aims. In particular, there is a need to establish the views of key partners on the realism of the overall goals of the programme and their willingness/ability to embrace them.

The programme was very successful in setting up mechanisms to establish what the community considered to be the needs of their area, and engaging with the community but in some areas, it was clear that, even with support, community members were being overloaded with demands and turnover was high. The need to make services responsive to community needs should be approached in a way which does not make excessive ongoing demands on a limited number of community activists - but rather seeks to engage with local communities at key points in the design and monitoring of the programmes. In future, local community planning will be the mechanism used to establish local priorities and the framework and guidance for local community planning should be realistic in its demands on community activists.

Some of the Pathfinders were very multi-faceted - they were pursuing action on a wide range of fronts, which made it difficult to assess the effectiveness of the individual approaches. In future, pilot programmes should be given more encouragement to pursue more focused approaches where appropriate. This would also make it easier to evaluate the effectiveness of particular types of approach.

Although the bidding process was not unduly demanding of applicants, it lacked clarity. There appeared to be little to distinguish the successful from the unsuccessful bidders and the successful bids were selected to achieve a good spread of projects and approaches. Unsuccessful bidders did not know why their bid had not succeeded. A bidding process may be used where the aim is to select the best or most innovative projects, or indeed where there is a need to attract a range of projects. However, the selection criteria should be made explicit and unsuccessful bidders should be told why they were not successful

The programme was restricted to existing regeneration areas. Some bidders felt they would have been more able to be innovative in an area that did not have any previous regeneration experience or 'baggage'. Many of the Pathfinders were in SIP areas and there may have been a degree of 'innovation fatigue'. Where innovation is sought, it is desirable to allow bidders sufficient freedom to choose areas which allow them to demonstrate it.

At the same time, an excessive emphasis on innovation may leave other deserving and successful approaches unsupported. More careful consideration should be given in programme design to the degree of emphasis to be put on innovation.

Pathfinders did not have enough time to consult with the community, establish appropriate structures, refine their approach, and appoint a co-ordinator in the first year of the programme. Most WfC co-ordinators were not appointed until in January 2000. A longer lead-in time is essential in a programme of this type - a 'year zero' - to allow improved planning, appropriate appointments to be made and buy-in secured from the community and other partners and maximisation of the time available to implement the programme

The co-ordinator was crucial to the success of the Pathfinder. The combination of community development and strategic influencing skills needed for the co-ordinator post was hard to find at the level of remuneration offered, and not all co-ordinators had the full range of skills needed. Consideration should be given in the design of programmes to the level and types of skills likely to be needed to implement the programme and guidance given about appropriate skills mix and salary levels. If there is reason to believe that it will be difficulty to obtain the mix and range of skills needed, the programme should be adjusted.

The flexibility afforded to Pathfinders in carrying forward unallocated finances from one year to the next was welcome but the administration of carrying spend forward was seen as quite bureaucratic. We would argue for maximum flexibility in financial management.

The monitoring and evaluation of any future process driven programme should be tackled differently. Changes in relationships and working practices are not easily captured by a structured pro-forma and more flexibility is required. Because they tried to fit all Pathfinders, the pro-formas were excessively long and complex, and the reporting frequency was excessive. A bespoke approach to the monitoring and evaluation of each Pathfinder might have been more user-friendly.

It is important that the amount of monitoring information captured and disseminated should be tailored to the capacity of the Programme's sponsors to absorb it. There is a need for shorter simpler monitoring and evaluation frameworks, and for greater emphasis upon the completion of monitoring forms. However, there may be a need to accept that the monitoring of a programme which focuses on change in process and allows for a wide diversity of individual projects will always be difficult to accommodate within a standardised approach.

The project researcher played a valuable role in 'digesting' the monitoring information and in providing an overview of the initiatives - the decision to appoint a researcher compensated to a large extent for the difficulties and limitations associated with the monitoring framework. We suggest that the Executive should continue to include a dedicated researcher within programme management for pilot programmes, particularly where programme management resources are limited.

It was felt that there was insufficient awareness of the programme in some of the spending departments (such as health and justice) within the Scottish Executive. There is a need for improved internal dissemination beyond the Regeneration Division to spending departments within the Executive.

Staff continuity at both Pathfinder and Executive level resulted in a loss of momentum. The objective should be greater continuity of programme management within the Scottish Executive particularly in the initial design to implementation period. Longer handover periods should also be allowed for.

ABOUT THE STUDY

The study was undertaken between April 2002 and August 2002 by DTZ Pieda Consulting. It included a desk review of documentation (the original bid, monitoring and annual reports, business plans; financial returns, and where these existed, final evaluations), face to face consultations with all the main stakeholders in the Pathfinder projects (co-ordinators, partners, the community and the Scottish Executive) and an analysis of programme expenditure.