Scottish Government

CHAPTER TWO: METHODOLOGY

2.1 This evaluation is based on an extensive range of quantitative and qualitative data. The evaluation was conducted in 3 parts: a baseline evaluation, which established the context of services for women in Glasgow and obtained statistical data against which the impact of 218 could be measured; a process evaluation, which examined the implementation and development of the service and described the process through which 218 was established and the policies and procedures that were developed; and an outcome evaluation, which examined the operation and impact of 218. This report presents the evaluation as a whole.

Methods

Analysis of Documentary Material

2.2 The researchers examined all available minutes, consisting of minutes and informal notes from meetings of the 218 Direction and Focus Team ( DAFT). The DAFT team consists of the Service Manager, the 2 part-time and 3 full-time team leaders, an administrative team leader, and the ( NHS-employed) health team leader. This enabled identification of the aims and objectives of the project and how these might have changed over time as well as identifying any factors that have had a bearing upon the model adopted and possible changes in criteria, procedures and services over time. The DAFT team holds weekly meetings at 218.

2.3 The research team also intended to analyse the minutes of the Monitoring Group for 218. This Group was expected to follow on from the work of the Commissioning Group involved in the establishment and development of 218. The Commissioning Group ceased its work at the beginning of 2004, and the Monitoring Group was intended to include original members of the Commissioning Group. For a variety of reasons, including the retirement of one member and long-term sick leave of another, the Monitoring Group had yet to commence its oversight of 218 while the evaluation was ongoing. Accordingly no minutes were available for examination.

Collection of information from Project Records and Information Systems

2.4 Collection of information from project records and information systems was obtained in order to describe the women accessing 218; to determine whether the service was being accessed by those for whom it was intended; and to begin identifying women whose experiences during and after contact with 218 could be tracked both in the short and longer term.

2.5 Fifty-six women signed consent forms for the research, agreeing to participate in at least some part of it. Due to delays in the establishment of a computerised database, a provisional paper-based system was implemented to collect essential data by hand. The formal paper-based system of data collection began on 1 September 2004, though statistics on referrals and programme completion had been collected from the outset. Where women did not provide active consent for their files to be accessed by the research team, relevant statistical information was obtained in an anonymised form.

Focus Groups

2.6 Focus group discussions were conducted with a number of women who made use of the different elements of 218. The intention was to conduct 4 focus groups consisting of 4-6 women each. These were to comprise of women who had made use of the detoxification and supported accommodation facilities (one group each) and 2 groups of women who had accessed the Day Centre services. The purpose of the focus groups was to elicit women's views about the extent to which the services offered met their needs and to identify any perceived gaps in provision. The focus groups also sought advice in identifying discussion topics for the interviews to be conducted with women in the third phase of the study.

2.7 Staff at 218 found that group sessions of 2 or 3 women were most productive for their programme. This meant that only small numbers of women were generally scheduled to attend each session. In order to minimise disruption to the service, the researchers aimed to coordinate focus groups within the time frame of the regular group programme at 218. This meant that the numbers of women attending the focus groups also tended to be small. The residential facilities for detoxification and supported accommodation had been merged into a single format residential programme, so the researchers did not need to create distinct focus groups for residential participants. They did however, divide participants in the day programme into groups depending on the stage of programme they were in, namely SAFE (the initial stage of community-based support), CONNECTIONS (the next stage of progression for more stable clients), and LOSS (the final phase for women who had progressed through the other programmes and were ready to address underlying issues in more depth).

2.8 In total, the researchers conducted 3 groups with a total of 8 women in the day programme ⁶ and 2 groups with 4 women in the residential programme, affording a total of 12 participants. Women were selected on the basis of availability; those who had attended group sessions on the day that the focus groups were organised were asked if they would be willing to participate. The researchers also took part in a series of informal discussions with service users, primarily in the residential unit, and sat in on 2 sessions of the day programme (one for the LOSS group and one 'Free for All' in which anyone could take part).

Interviews with Members of the Monitoring Group

2.9 Individual interviews were planned with representatives of the various agencies represented on the Monitoring Group for 218. The purpose of these interviews was to explore the rationale for the model adopted from the perspectives of different agencies and the extent to which the model, as operated, corresponds to what was originally envisaged. Views were to be sought on the appropriateness of the referral criteria and the mechanisms for accessing and referring to the service and on the ability of the services to meet women's needs (including the effectiveness of mechanisms for linking women into services and supports after they leave 218). As mentioned above, however, the Commissioning Group for 218 had been dissolved, and the Monitoring Group had yet to be established while the evaluation was ongoing.

Interviews with 218 Staff

2.10 Interviews were carried out with a cross-section of staff at 218. The purpose of these interviews was to explore with staff their views about the operation of the Centre in its early phase, including the appropriateness of the referral routes and criteria; the ability of the services to meet women's needs; and the effectiveness of the longer-term supports available to women when they left. Interviews also focused upon the training and other support provided to staff, and on the effectiveness of multi-agency work within 218 and with respect to 'in-reach' and other external services.

2.11 The second round of interviews sought the views of workers about how well the project was operating and the extent to which its objectives were being achieved. This included how well they believed 218 was succeeding in addressing issues related to substance misuse in conjunction with offending behaviour and how effective it was in linking women into employment, education, training and other mainstream services and supports. The interviews also sought to identify perceived areas of good practice and gaps in service provision and to explore how effective project staff considered the multi-agency approach to have been.

2.12 Twenty-four interviews with staff employed by 218 were conducted in total. Participants were selected to represent a cross-section of staff employed by Turning Point and included the following posts:

• Chief Executive for Turning Point Scotland
• 218 Project managers
• Team Leader - Residential
• Team Leader - Day Programme
• 2 Project Workers - Residential
• 2 Project Workers - Day Programme
• Support Worker - Residential
• Support Worker - Day Programme
• Assessment Worker
• Team Nurse

Interviews with Key Stakeholders

2.13 Interviews were conducted with representatives of a range of organisations that have an interest in the operation of 218. These included agencies involved in referring cases, in supervising women who attend the service, and in providing services on an 'in-reach' basis or after women have moved on. The interviews explored issues such as the criteria and routes of referral to 218, perspectives on the services provided, and the perceived effectiveness of the services in meeting women's needs and linking them into longer-term supports (including drug services and services related to employment, education and training). As with the staff interviews, the purpose of the second round of interviews was to seek the views of stakeholders about how well the project has operated and the extent to which its objectives have been achieved. Interviews also identified perceived areas of good practice and gaps in service provision and how effective the multi-agency has been. Respondents included:

2.14 In-reach services

• Health Team Leader
• GP contracted for work at 218 (2004 and 2005);
• Psychologist
• Occupational Therapist
• Community Psychiatric Nurse (2004 and 2005)
• District Nurse (2004 and 2005)

2.15 Accommodation services

• Homeless Social Inclusion Partnership (2004 and 2005)
• Talbot Association (2004 and 2005)
• Simon Community (2004 and 2005)
• Homeless Addiction Team;
• Govanhill Women's Project (2004 and 2 workers in 2005)

2.16 Social work and addiction services

• Maryhill SWD Addictions Team (2004 and 2005)
• Easterhouse Community Addictions Team
• Parkhead SWD Addictions Team (2004 and 2005)
• Possil SWD Addictions Team (2004 and 2005)

2.17 Specialist services for women

• Base 75 ⁷ (2004 and 2005)
• Routes Out Intervention Team ⁸ (project manager, project worker - 2004 and 2005)
• Routes Out Social Inclusion Partnership Chair and members (2004-2005)

2.18 Criminal justice

• 8 Criminal justice social workers (including Bail officers, court officers and Drug Court Supervision and Treatment Team members in2004 and 2005 and Commissioning and Monitoring Managers in 2005 only)
• 3 Police Representatives (Community Safety) (2004 and 2005)
• 3 Police street liaison team (officers)
• Addiction Officer at Cornton Vale (2004 and 2005)
• Justice of the Peace (2004 and 2005)
• 2 Sheriffs (2004 and 2005)
• 5 Stipendiary Magistrates (2005)
• 2 Solicitors (including one PDSO) (2004 and 2005)
• Solicitor
• 2 Procurators Fiscal (2004 and 2005)
• District Court Clerk (2004 and 2005)

2.19 The methods for this series of interviews varied. Most were recorded and transcribed. For stakeholders with less knowledge and experience of 218 notes were taken by hand. In a small number of cases, interviews were conducted by telephone for the convenience of the person interviewed. To avoid identification of individual respondents, all comments and quotations have been anonymised.

Interviews with clients

2.20 The final stage of the research included interviews with women in an 'intensive' cohort towards the end of their engagement with 218. In addition, attempts were made to contact for interview those in the sample who left the project early, including women who were in Cornton Vale. It was anticipated that there would be difficulties in contacting this group of women. The interviews sought to identify the extent to which the women perceived the services 218 provides as meeting their needs and as impacting upon their offending and associated problems. We explored perceptions of the arrangements that had been put in place for follow-on support (including employment, education and training and access to drug services) and sought to identify perceived gaps in provision while attending 218 and in the longer term. Brief follow-up interviews were also conducted with a smaller sub-sample of women approximately 3 months after moving on from 218.

2.21 Interviews with the intensive sample were originally planned with 50 participants in the day programme and 35 participants in the residential programme, based on an estimate of the annual throughput of clients. However, many women were referred to 218 several times over the year, so the actual number of women in real terms was much smaller. We therefore based our final figure for interviews on the 143 women who engaged ⁹ with the service during the study period. In total, 51 women were interviewed as part of the intensive cohort, 32 of whom were about to move on from the residential unit, though many had taken part in both residential and day services. At least 9 of these women had taken part in a programme at 218 previously and had left the programme completely, usually in an unplanned manner. Fifteen of these women took part in follow-up interviews at least 3 months after leaving 218.