Scottish Government

Main Findings

• The overwhelming majority of respondents (94%) had worked in partnership before, with 44% of partners stating that such work was, in their experience, always worthwhile.
• Overall respondents were satisfied with the partnerships' composition, coverage and the clarity of its aims, both before and after the pilot period. An exception to this was an increase from an initial 47% to a subsequent 74% of respondents stating that the relevant organisations were represented in the partnership.
• Overall, partners dedicated less time to the partnership than they initially expected. However, a large proportion (40%) of partners went into the process unsure of how much time they expected to dedicate to the partnership.
• Although the majority of respondents initially expected improvement in planning and co-ordination (81%) and greater innovation (50%), only 49% and 26%, respectively, stated that these improvements had taken place by the end of the pilot phase. Some indicated that it was too early to tell whether these would be achieved.
• 86% of partners anticipated greater joint working as a result of the partnership process. This was almost matched by 80% of partners stating that the pilot process had indeed resulted in such an improvement.
• There was a positive response about the degree to which the pilot partnerships always allowed partners to have their views taken into account, although respondents from statutory agencies were more likely to hold this view (100%) than voluntary organisations (64%).
• The formal structure of the partnerships proved to be adequate in terms of allowing partners to participate in the partnership process, with all of the respondents stating that this 'had' (59%) or 'had mostly' (41%) been the case.
• The overwhelming majority of partners (94%) found the level of help and guidance provided by the Scottish Executive and the Scottish Legal Aid Board to have been 'just right'.
• Almost half (49%) of partners stated that the needs assessment research exercise had been essential for the production of a strategic plan for services in their area, a further third (33%) stated that it had been useful but not essential, and the remaining 18% stated that it had been not useful.
• When asked whether the partnership approach would bring added value to the planning of services in their locality, 79% of partners agreed that it would.

Introduction

A Working Group was established in October 2000 to examine and review the provision of legal advice and information in Scotland and to generate recommendations on the development of a community legal service. In response to these recommendations 5 pilot partnerships were established with the aim of improving the planning and co-ordination of legal services in their area. The first partnership was established in Glasgow West, with subsequent partnerships set up in May 2003 in Fife, Edinburgh and Argyll & Bute. A thematic partnership on access to legal advice for disabled people was also established at that time with a national remit.

In order to facilitate the planning and co-ordination of service provision, the area-based partnerships were provided with the results of a major research exercise that aimed to assess the need for legal advice in their area. A series of separate publications, reporting the results of this needs assessment research can be viewed at www.scotland.gov.uk/socialresearch

It was also decided that the views of partnership participants on the structure, appropriateness and potential contribution of the pilot partnerships should be assessed. This was achieved by conducting a relatively small-scale evaluation that covered the 4 partnerships set up in May 2003; Fife, Edinburgh, Argyll & Bute and the national thematic partnership on access to legal advice for disabled people. The first partnership in Glasgow West had been established too long to participate in this before and after evaluation.

Methods

Three key methods were used in this evaluation:

1. Questionnaire surveys were conducted to capture participants' views on the partnership process, and to highlight how these views evolved over the 10-month process. An initial questionnaire was issued at the outset of the process in July 2003 and a follow-up questionnaire was issued towards the end of the process in April 2004.

2. Non-participant observation of 3 of the partnerships was undertaken to observe group dynamics, participation, points of conflict, decision making and production of outputs.

3. A documentary analysis of the partnership outputs, including minutes of the meetings and the draft strategic plans for the provision of legal advice and assistance in the various areas, helped to establish whether the original tasks were completed successfully.

This paper focuses on the results of the questionnaire surveys. The 'before' questionnaires explored partners' previous experiences of partnership working and their expectations about the pilot partnership. Some questions were repeated in the 'after' questionnaire, to allow a comparison of partners' initial views and their views at the end of the process. Additional questions were also included in the 'after' questionnaire, informed by the non-participant observation process, to investigate views about additional aspects of the partnership process such as:

• whether their views were taken into account
• the formal structure of the partnership
• the level of guidance that they were provided with
• the usefulness of the needs assessment exercise
• whether the partnership approach would bring added value to planning in their locality

The sample sizes for the before and after questionnaires were the same, at 36 responses (58% and 59% response rate, respectively).

Previous experiences of partnership working

The results of the questionnaires showed that the overwhelming majority of respondents (94%) had worked in partnership before, with 44% of partners stating that such work was, in their experience, always worthwhile. Half of the respondents felt that this was only sometimes the case.

Composition, coverage and clarity of aims

The total number of partners within the partnerships remained largely constant, with 62 partners at the beginning of the process, and 61 at the end. There were, however, within each pilot partnership, 'core' partners who regularly attended the meetings and maintained correspondence throughout the process. In total, these 'core' partners numbered 42. Each partnership had at least 10 'core' partners.

As detailed in Table 1 below, composition of the partnerships remained largely consistent in terms of the proportion of partners from voluntary or statutory agencies.

Table 1: Composition of the partnerships

Both initial and subsequent responses were positive in relation to the partnerships' composition, coverage and clarity of aims (see Table 2). An exception to this was an increase from an initial 47% to a subsequent 74% of respondents stating that the relevant organisations were represented in the partnership.

Table 2: Views about clarity, composition & coverage

Expected and actual workload

There was a difference between the amount of time that partners expected to dedicate to the partnership per week and the time that they reported having dedicated to it. However, there were also a large number of partners who began the process not knowing how much time they expected to dedicate to the partnership. Overall, 71% of respondents stated that the amount of time they had dedicated to the partnership had been as expected.

Table 3 shows the number of hours that partners expected to dedicate to the partnership per week (excluding the partnership meetings) and the number of hours that they reported having dedicated to the partnership. A significant number of respondents (40%) did not know at the outset how much time they would dedicate to the partnership.

Table 3: Expected & actual time dedicated

These findings indicate that the partners who initially did not know how much time they expected to dedicate to the partnership are likely to have dedicated up to one hour to the partnership a week, excluding meetings. Overall, partners seem to have dedicated less time to the partnership than they expected to.

Expected and actual achievements

There was a considerable difference between the expected and actual achievements within the partnerships. Table 4 shows partners' expectations of the improvements as a result of the partnership process; against the percentage of partners who felt that those expectations had been fulfilled.

Table 4: Perceived expected & actual achievements

However, these results alone do not provide sufficient insight into the discussion on expected and actual achievements. Some partners (19%) in the 'after' questionnaires added that although the achievements had not yet been attained, they might be in the future. It was widely recognised by the partners that the pilot phase had been too short to achieve all these goals, and that some of them were medium to long-term.

From a list of 7 possible options, partners were asked to identify 3 key achievements that they expected from the pilot partnership process. The 'after' questionnaire revisited this question, allowing for observation of any change in perceived key achievements over the partnership process. Results show a slight shift in focus over the partnership process, from specific achievements, such as the development of referral systems between partners, towards more general priorities, such as sharing information on local need and supply. This indicates recognition of the need for agreement on a base-line of issues and definitions between partners before more detailed and specific tasks can be addressed.

Experience of participation in the partnerships

Partners were positive about the degree to which the pilot partnership process allowed them to participate and have their views taken into account, as well as by the level of support provided.

All the respondents from statutory organisations (100%) and 64% of respondents from voluntary organisations stated that their views were always taken into account. The remaining 36% of respondents from voluntary organisations stated that their views had been taken into account only 'sometimes'.

The questionnaire also asked about the formal structure of the pilot partnerships, such as the location and timing of the meetings. Over half (59%) of the partners stated that it had allowed them to participate as they had wished, with the remaining respondents indicating that this had mostly been the case.

The facilitators to the pilot partnerships (Scottish Executive and Scottish Legal Aid Board) provided a high level of support throughout the process, including: arranging meetings; recording minutes; helping draft the strategic plans for legal advice; and providing assistance in the various areas. This level of help and guidance was rated as being 'just right' by 94% of the partners. However, it was more than was anticipated at the outset by the Scottish Executive and the Scottish Legal Aid Board.

Utility of needs assessment work

A large-scale research exercise was carried out at the beginning of the pilot partnership phase, which aimed to assess the need for legal advice in each of the partnership areas. It was anticipated that this piece of research would inform the pilot partnerships in the planning and co-ordination of legal services in each of the area based partnerships.

Nearly half of the partners (49%) rated the needs assessment exercise as essential in allowing them to produce a strategic plan for services in their area, whilst a third (33%) described it as having been useful but not essential. Just under a fifth (18%) of the partners stated that the needs assessment exercise had not been useful in their development of a strategic plan.

Views on the future value of partnership working

When asked whether they thought that the partnership approach would bring added value to planning in their locality, 79% agreed it would, and only 6% disagreed; a further 15% responded 'don't know'.

Conclusions

The evaluation set out to ascertain the views of participants on the appropriateness and potential contribution of the pilot partnerships to the planning of legal services in their area. The questionnaire results show that the majority of partners believe there is potential in the partnership approach for planning in their locality.

The results also indicate, however, that in order for the partnership approach to improve planning, co-ordination and innovation of legal services, it is likely that base-line issues such as common goals need to be addressed first. The questionnaire results indicate that the pilot phase has not been established long enough for this to take place. Following the pilot phase, the 3 area-based partnerships have each moved into a next phase of activity, grounding future work within the framework of other local planning initiatives. This has been more difficult for the thematic partnership due to the absence of such a framework.

There was some support for the needs assessment exercise that was undertaken as part of the partnership process. It is more likely to be seen by partnership members as useful or essential, than as neither.

Partners who had expectations of the time the partnership would involve, found it less time consuming in practice. Partners also found that the support and facilitation from the Scottish Executive and Scottish Legal Aid Board were of the right level for the process. However, that level was of a greater degree than had been initially envisaged by the facilitators. This may raise questions for any future work of this nature about how to achieve the best balance of involvement from partners and facilitators.