Scottish Government

3. Experiences of Advice Organisations

3.1 Introduction

We conducted interviews with a range of advice organisations to learn of their experiences. We wanted to compare the views of those that had undergone audit and that were "known to be self-assessing but (to date) had not taken a decision to seek accreditation".

As we describe on page two of this report we encountered a number of issues with the sampling framework. We finally consulted with 14 organisations that had undergone audit and 21 that had not. Crucially we found that organisations that had not been through the audit process (and were not preparing for audit) were not self-assessing their performance against the Standards in a systematic way. Only organisations that have been audited or intending to undertake a future audit were systematically working to the Standards.

The interviews focused on:

• the background to taking up the National Standards;
• the process and experiences of adopting the National Standards;
• the background to the decision whether or not to undergo audit;
• funding and experience of the audit process; or, what alternative mechanisms have been adopted to validate adoption of the National Standards;
• impacts on staff, clients and partners;
• views on the value of accreditation in adopting the Standards; and
• lessons learned to date.

Following the interviews we selected 12 organisations as case study examples. Of these, nine organisations had been through the audit process (one had given up accreditation and one was not yet fully accredited) and three had shown interest in the Standards by attending some training but had not taken a decision to undergo audit. The case studies explored in greater detail the contribution adopting the National Standards had made to improving the quality of services, and any added value accreditation had offered compared with self-assessment.

The views reported are based on the perceptions of those interviewed. We have not sought to correct issues of fact. And in some cases, procedures may have changed or been amended since they were last involved in issues relating to the National Standards. The important issue - from the point of view of this evaluation - is that these are the perceptions that are held by those within the organisations selected.

3.2 Views of organisations that have been through audit

We spoke to organisations across Scotland that had been through the audit process. These ranged from small information and advice providers employing five or six staff members to larger support services with over a hundred staff and volunteers.

Generally, consultees were positive about the accreditation framework and benefits to their organisation, although there were some issues raised about the audit process.

3.2.1 Adopting the National Standards

Consultees were asked about: their decision to adopt the National Standards in their working practices (before any decision to undergo audit); the process required to adopt the Standards; and their experiences of the process.

3.2.2 Deciding to adopt the National Standards

Consultees expressed different reasons for adopting the National Standards although most were clear that it was about ensuring a high standard and providing quality information and advice. Some were clear that they wanted to improve the service that they are providing by increasing their knowledge base and ensuring a more consistent approach to the provision of information and advice. One organisation said that the Standards provide a measurement of how well they are doing - and meant that they would have goals to work towards to improve the service.

Some organisations emphasised the importance of external validation and demonstrating to others the quality of the information and advice being provided.

A number of the organisations that have adopted the National Standards (and have been through audit) said that this was a continuation from being involved in the development of the Standards. This meant that there was support for adopting the National Standards from senior levels within the organisation.

Some consultees cited external factors as influencing the decision to adopt the Standards. One provider said that they had adopted the Standards as a condition of grant (HomePoint) and decided to continue after the period of grant had ended as they felt they were beneficial. Others said that they had adopted the National Standards because they are working under local authority strategies on homelessness and housing which have been developed on the basis of the National Standards.

3.2.3 Process undertaken and experiences

Typically, organisations have started by collating existing policies and procedures in written form (and in some cases writing up policies / procedures for the first time), looking at these against the Standards and establishing areas for improvement and training. Having undertaken self-assessment and developed an action plan, organisations worked on outstanding issues before deciding whether to apply for audit.

One support service which is part of a local authority set up a working group to oversee the process. Briefings were set up with staff to inform them about the accreditation process. In many cases, the process of looking at the Standards was divided between staff members. One organisation divided up the Standards between team members and they were asked to evidence what the organisation was doing to meet the Standard - or where they were falling short.

Some consultees felt that the process of adopting the Standards was time consuming and one described it as "onerous".

The time taken to adopt the Standards varied across organisations from six months to five years and was influenced by the quality standards that the organisation was already working to and the resources (especially staff time) that were available. Resourcing was a particular issue when it came to audit.

Several organisations said that although the process is demanding it is worthwhile.

'This was a long process but one that was worthwhile. It was through addressing the issues that we identified in the self assessment that the organisation was able to work to the Standards.'
(Housing advice provider)

Organisations had different views on the amount of administrative time that was required in adopting the Standards. Some found it difficult to find time and the heavy reliance on recording activities (e.g. interview times) was challenging and labour intensive. Others said that this was less of an issue as they were able to build the work into existing workplans. For some the process was not as difficult as they had anticipated at the outset.

Consultees were particularly positive about the input from the HomePoint team and it was felt that the training seminars were "essential to understanding and achieving the Standards". The self-assessment form was also considered to have been particularly helpful by a range of consultees - and there were positive comments about the quality of support from HomePoint staff at this stage and throughout the process.

There were some concerns expressed about how easily the Standards could be interpreted. One organisation felt that the Standards were difficult to interpret and that they didn't always apply to the activities that they were undertaking. A different organisation expressed concern that other agencies were interpreting the Standards in different ways from them - they felt that this was leading to confusion and raised questions about the Standards being properly met across the country.

One organisation providing a range of support projects (each providing information and advice to some degree) initially tried to adopt the National Standards across too many diverse activities. They decided to focus the work of one project with the intention to roll out the Standards across the organisation at a later date.

3.2.4 Audit and Accreditation

Consultees were asked about the decision to apply for audit, the process of audit and the impact that accreditation has had on their organisation.

3.2.5 Background to decision to undergo audit

Many of the organisations that we spoke to underwent audit because they wanted the badge of accreditation. They strongly believe that accreditation demonstrates to others that you are providing a good quality of services. Some felt that accreditation is a reassurance to clients that they will receive good quality advice.

Several consultees decided that they would apply for audit as they felt it would be beneficial for future funding and a good selling point when competing for contracting opportunities. However, this was not a consideration for all consultees. Some had applied for audit at a stage where the accreditation for the National Standards was "relatively unknown" and several stated that the relationship between accreditation and future funding opportunities was, and remains, unclear.

A number of organisations told us that they were already working to many of the National Standards and sought accreditation as they wanted recognition for the quality of service that they were providing. Others said that they had committed significant time and resources to working to the Standards (and adopting new procedures where necessary) and felt that it wouldn't have been worthwhile to put in the effort and not then seek accreditation. One support service based at a local authority applied for audit as they felt it was important to get external validation - the homelessness service were asking partners to work to the Standards and they felt this would be difficult if they didn't take the lead with accreditation.

Several organisations sought accreditation as they felt it would have a positive impact on staff morale and would create a sense of organisational pride.

It was also felt that accreditation can have an important impact on external relationships. For some, audit / accreditation was a way of benchmarking against other organisations and an opportunity to increase the profile of the service. For smaller organisations providing services which are perceived to be less mainstream, accreditation was felt to be a strong "vindication" of approaches - and had the potential to strengthen their position in discussions with partner agencies.

Some of the organisations that we spoke to had undergone audit as part of the pilot process. For some there was an added financial incentive as there would be no charge for the audit but the main reason for these organisations to be involved was the opportunity to be part of something new and get "ahead of the game".

3.2.6 Funding the audit

For most of the organisations that had chosen to go through the audit process funding was not a significant issue. Some organisations had had the audit part funded by CAS, while others had met the cost from their own budgets and not found the price prohibitive.

However, there were some concerns about cost raised by a few consultees, particularly where there had been no charge to the organisation for the pilot audit and uncertainty about meeting the costs of re-accreditation. In one case there was significant concern about the costs associated with the 18 month interim validation - the organisation felt that they could not justify paying the fee and did not continue with accreditation.

3.2.7 Experiences of audit process

The majority of organisations that had been through audit felt that the process had been challenging but worthwhile. However, there was a clear view that the audit process was labour intensive and required a lot of paperwork although this appeared to be less of an issue for organisations applying for Type I accreditation.

A number of consultees expressed concern that the audit was conducted in a one-size-fits-all way that was not appropriate to their organisation or way of working. The way services were delivered did not always match the assumptions of the auditors. This was particularly the case where information and advice was delivered in an ad hoc way as part of wider support to case clients or where a number of different advisors were responsible for dealing with one case.

Some consultees also expressed concern that the audit was significantly about the perceptions of the auditors. There were also some comments about the expertise and experience of the auditors. In particular, some council staff felt that the audit team was not sufficiently familiar with the local authority environment.

A number of organisations that we spoke to felt that the audit was too picky and there was a perception that some changes that were asked for were bureaucratic rather than resulting in benefits to clients. A few consultees said that the audit process was daunting for staff members and volunteers. It was suggested that the audit could have been a more supportive experience, particularly where organisations make use of volunteer advisors.

One organisation had applied for Type II accreditation across all areas of housing law and was not accredited. They felt that they had been too ambitious and would have benefited from more detailed advice on what to apply for.

3.2.8 Impact of audit / accreditation

Staff

Several organisations that have been accredited told us that it has been a significant boost for the confidence and morale of staff and volunteers.

It was broadly felt that the work undertaken to prepare for audit has had the biggest impact on staff. One organisation told us that the process of preparing for and going through audit has brought about a change in culture at the organisation - staff are more inclined to find out what help and support clients need rather than waiting to be asked.

Several organisations told us that a key change has been the way staff feel about information resources and training.

'Accreditation has changed the way we think about ensuring we are up to date with housing information and advice. Staff are much more enthusiastic about training too - because they can see the value in it more.'
(Housing advice provider)

Clients

Most of the organisations we spoke to felt that audit / accreditation had improved the service provided to clients although evidence for this was primarily anecdotal. One of the organisations had undertaken a feedback exercise which suggested that client satisfaction with the service had improved since accreditation.

Organisations were clear that the systems and procedures that they had in place after audit were more robust and it was generally felt that there was a more consistent provision of information and advice as a result. In particular, a number of organisations told us that they had adopted better recording of information from clients - and more robust procedures for signposting of clients to other agencies.

Partners

Most of the organisations that we spoke to felt that audit and accreditation for the National Standards had not had a significant impact on relationships with partner agencies. Some smaller organisations felt that accreditation had given their service greater credibility and respect, but none of the consultees said that it has been an influence in terms of funding.

Some organisations told us that partners had shown interest in the National Standards as a result of their accreditation and they were hopeful that they would influence others to adopt the Standards.

A key issue was the level of interest in accreditation from local authorities. Consultees had varied experiences with some councils more interested and knowledgeable than others.

3.2.9 Added value of accreditation

Nearly all of the organisations we spoke to that had been through audit felt that there was added value in achieving accreditation. Consultees felt that the badge of accreditation was a valuable recognition of quality.

Many of the accredited organisations felt that it was going to put them in a strong position in future applications for funding - although generally the consultees were unclear how big an impact accreditation would have on their chance of winning funding or securing contracts.

Consultees felt that there was a clear impact on staff morale and greater confidence within organisations. Preparation for audit was considered an important process for identifying areas that need improvement. Achieving accreditation was considered valuable as it sets the standard for the organisation and the desire to be reaccredited encourages continuous improvement.

However, there was some concern that accreditation is still not widely understood. It was suggested that partners (and critically funders) are not clear about what the different levels of accreditation mean and how this translates into the standards of service that can be expected. One of the organisations that we spoke to felt that there had not been any added value as a result of accreditation. This organisation had not undertaken any specific changes to their service as a result of the audit process - and felt that there had been limited interest in accreditation from funders and local partners.

3.2.10 Lessons learned

The consultees that had been through the audit process identified a number of key lessons from the process. These included:

• The audit process requires a lot of time, effort and commitment;
• It is important to make sure your organisation is ready for the audit - it can be difficult to change the way you do things once the audit process is underway;
• It is essential that staff see the importance of the Standards to the work that they do - and are involved in the process;
• There may be scope for the audit process to be streamlined - issues raised by auditors were not always felt to be relevant by providers;
• There can be issues of interpretation or language - on occasion this has led to a lack of clarity between organisations and auditors;
• It is important to integrate changes in practice into existing workplans - this helps to make the process less of a bureaucratic burden;
• Organisations need to be open minded and realistic about their strengths and weaknesses. They need to be willing to address areas that need improvement. Providers also need to be realistic about the number of areas of law and what type of accreditation they are aiming for.

3.3 Views of Organisations That Have Not Been Through Audit

Discussions were also held with organisations that have not been through the audit process. Most of these organisations had decided not to apply for audit although two said that they are likely to apply for audit in future - but have still to make a final decision on this.

Most of these organisations are not working to the Standards in a systematic way but are using them more generally as good practice guidance. In most cases staff have attended training to support compliance with the National Standards.

3.3.1 Adopting the National Standards

Consultees were asked about their decision to start working to the National Standards, the process undertaken and what this has meant for their organisation.

3.3.2 Deciding to adopt the National Standards

For most organisations taking up the Standards has meant staff attending training in housing law. Several said that taking up the Standards would benefit staff and the quality of service provided by the organisation.

The Registered Social Landlords ( RSLs) that we spoke to said that they had adopted the National Standards as they are included as part of their local authority's Local Housing Strategy. In some cases the impetus for adopting the Standards came from the development of a Common Housing Register ( CHR) - as part of this process, local housing providers were seeking to meet consistent standards and the National Standards (and training to support the Standards) fit with this.

For some organisations the impetus to adopt the Standards came as a result of discussions with other partner organisations - partners have been through the process and recommended it as being beneficial.

Some consultees told us that when they looked at the National Standards they realised that they were already meeting many of the competencies. These organisations are aiming to address any gaps in provision and would eventually like to be recognised for the quality of service they are currently providing.

3.3.3 Process undertaken and experiences

As stated, for most organisations that have not yet been through the audit process the main activity in relation to the National Standards has been staff attendance at training events in relation to housing law. Initially, staff attend a seminar and complete a training assessment form before undertaking further courses.

In some cases, HomePoint staff have visited the project to give advice on what needs to be done to meet the National Standards. A few of the organisations have completed the self-assessment checklist to identify gaps - this has been used to develop appropriate training programmes.

Two of the agencies that we spoke to had developed an action plan to help them work towards the Standards - these organisations are at an early stage and may progress to audit at a future date. One housing provider told us that they have developed an internal programme for working to the Standards. The process has been working relatively smoothly although some staff - particularly at area offices - have found it difficult to find the time to complete tasks. One organisation is currently undertaking an internal quality audit - as part of this process they are trying to match competencies with the National Standards.

A number of consultees said that many of the Standards were not difficult for them to meet - either they felt that they were meeting the Standard with existing practice or there was a body of existing knowledge within the organisation that made progress towards the Standard achievable. Standards were most challenging where they relate to policies or procedures that are new to the organisation.

It was stated that some of the Standards are vague and require further explanation - organisations gave the example of community profiling. Some of the organisations that have not been through audit felt that it would be difficult to quantify (and establish budgets for) the information that they give out. This was particularly the case where it is a continual (and often ad hoc) part of their service.

Some consultees felt that some of the Standards were overkill. It was stated that some of the competencies are implicit in their service and demonstrating them requires a new bureaucratic process to be introduced. Others stated that the training for the National Standards was demanding on time and required a lot of paperwork.

There were mixed views on whether the process of working towards the National Standards is resource intensive. While some have found it demanding on staff time, others said it was more manageable than they had initially expected.

3.3.4 Audit and Accreditation

The organisations were asked for their views about audit and, where applicable, were asked why they had decided not to undergo audit.

3.3.5 Background to decision not to undergo audit

Some of the organisations that we spoke to are at an early stage in the process and have not yet made a decision regarding accreditation - and two expect to apply for audit in the future. Those that have decided not to apply for audit raised concerns about the level of work involved, cost, the existence of other accreditation frameworks and the lack of incentives to become accredited.

Several organisations said that they were put off pursuing accreditation due to the perceived level of resources required.

A number of consultees (particularly those from smaller organisations) said that the cost of the audit was prohibitive and some felt that it was unfair that they were expected to meet the costs themselves.

Another disincentive was the limited take-up of the accreditation system to date. One housing provider said that since other local RSLs haven't been accredited it has not been a priority for them.

A number of organisations said that they are already part of accreditation systems and regulatory regimes that ensure a high quality of service. As such, there was no drive within the organisation to undergo another audit.

3.3.6 Mechanisms to validate adoption of Standards

The organisations that we spoke to do not have alternative mechanisms in place to monitor their compliance with the National Standards. Generally, organisations self-assess their performance against the Standards periodically. One organisation said that there is an issue of legitimacy with this approach as they are not being assessed independently.

3.3.7 Impact of audit / accreditation

Staff

Most of the consultees were positive about the impact that the National Standards have had on staff at their organisation. In particular the training for the National Standards has raised awareness among staff of the need for consistent and better quality information and advice provision.

A number of consultees said that the training sessions in housing law had helped to raise knowledge of current legislation. Working towards the Standards has helped to improve team working at some organisations - with more sharing of knowledge across staff.

It was stated that the Standards have given clarity on staff remit - and the point at which clients should be referred on.

Others were more sceptical about the impact of the Standards stating that they were already working to a high quality.

Clients

Among non-accredited organisations there were mixed views about the impact that the National Standards have had on the service provided to clients. While some felt that their service was improving, others said that the work they were undertaking was more about strengthening the organisation than direct service provision.

One organisation said that clients are likely to receive the required information and advice more quickly than before. However, generally, there was only anecdotal evidence of impacts on clients.

Partners

The consultees felt that there was limited awareness among partners of the work that they had been doing regarding the National Standards. Across the organisations consultees reported varying levels of interest from local authorities and other funders. Some participants felt that while the National Standards have a comparatively low profile at the moment they may become more of an issue for funders in future.

3.3.8 Lessons learned

The consultees that had not been through the audit process identified a number of key lessons from the process. These included:

• Working to the National Standards is useful for clarifying what you are doing well - and identifying what you need to do better.
• The Standards and the training provided demonstrate the need for organisations to evidence what they are doing to others.
• Costs and the perceived resource commitment are both disincentives to undergoing audit.
• There is limited drive to pursue accreditation where funders have shown limited interest - and where other standards frameworks are in place.

3.4 Summary

• Interviews were conducted with 37 advice agencies some of which had been through audit and others that had not. We subsequently undertook 12 case studies. These explored in greater detail the contribution adopting the National Standards had made to improving the quality of services, and any added value accreditation had brought.
• Organisations adopt the Standards as they want to improve their service and ensure a consistent approach.
• Although adopting the Standards is considered to be time consuming and resource intensive, it is felt to be a worthwhile process. Consultees were particularly positive about the training in housing law for the Standards and the support provided by HomePoint.
• Organisations decide to undergo audit as they value the external validation and want the badge to demonstrate the quality of their service. Some agencies decided to go for audit since they felt it would increase funding and contracting opportunities.
• Consultees felt that a one-size-fits-all approach to the audit may not be appropriate. There was also some concern that the audit was too dependent on the perceptions and expertise of the auditors - auditors may not be sufficiently familiar with the environment in which agencies are working.
• Accreditation raises confidence and morale among staff. Organisations felt that the service to clients had improved as a result of accreditation although evidence of this was patchy. There was less impact on relationships with partners although some felt that accreditation had resulted in greater credibility and respect.
• Nearly all of the organisations that we spoke to felt that accreditation brought added value and put their organisation in a stronger position, but there was some concern that accreditation is not widely understood.
• Most organisations that had been through audit were not concerned about costs although there was more concern about having to pay for an interim validation after 18 months and re-accreditation every three years.
• For organisations not working towards audit, taking up the Standards has primarily meant staff attending training sessions. These organisations did not have systematic processes for self-assessment against the Standards.
• Some organisations not working towards audit felt that some of the Standards need further explanation. There was concern that some indicators would be difficult to quantify and that adopting the Standards was a bureaucratic process.
• Organisations choosing not to be audited were mainly concerned about the level of work involved, the cost of audit (which were seen as prohibitive), and overlap with existing quality standards frameworks.
• However, they were positive that the Standards had raised awareness of the need for consistency and improved knowledge of current legislation. There were mixed views on whether the Standards had improved services to clients and concerns about limited awareness among partners and inconsistent approaches across local authorities.