DEVELOPMENT OF TOOLS TO MEASURE SERVICE USER AND CARER SATISFACTION WITH SINGLE SHARED ASSESSMENT
CHAPTER SEVEN VALIDATION ISSUES
7.1 Given the importance of validating the methods developed for assessing user and carer satisfaction with SSA and its outcomes, the Project Team held a specific meeting to discuss validation early on in the project.
7.2 The idea of a 'Toolkit' was identified at the meeting as the best way of conceptualising the Tools and Methodologies. It was agreed that the Toolkit would contain, for example,
good practice guidance
approaches - well indexed links to materials and resources which would help to support use with service users and carers with a wide range of language and communication requirements
commentary on the implications of a person centred approach
considerations (from best practice), covering specific issues such as
consent, when information should be collected from someone who knows the person well rather than the person themselves
questions to be addressed, including core or key questions
7.3 Other key conclusions reached at the meeting were:
Validation in this project was a path or cumulative process which was to be built into the framework of the research and development process (see Table 7.1 below on p 44 for the range of activities which contribute to the validation of the methods developed in the course of the project)
Validation would be carried out in different ways; linkages would be clear and explicit
The key task was to ensure that the research methods were measuring what they said they were measuring, i.e. service user and carer satisfaction with the single shared assessment - and not something else
The question format and sampling were 2 aspects of validation
Validation required sound, convincing and defensible reasons and explanations for process decisions and methods, e.g. about the selection of the pilot sample; the approaches used to collect information; how 'satisfaction' is defined and measured; and the methods used in the pilot
One of the underlying values of the approach of the project is that the most valid judge of satisfaction with the process of SSA is the person who has experienced it.
Table 7.1 Project Activities and their contribution to validation
Activity | Outcomes: Validation |
Literature search | |
Follow-up of survey responses | |
Practitioners Workshop 1 | Operational perspective built into the practice More detailed information from different areas provided Opportunities to test out ideas for questions, approaches and explanations
|
Development Groups | Consistency of approach Validity of service users and carers experience Thorough, careful and appropriate recording of meetings Opportunity to test out some approaches, materials
|
Practitioners Workshop 2 | Opportunity to further examine and test questions etc Reality/feasibility check Validity of practitioners' experience in different settings Practitioners provided further check on validity of questions
|
Pilot: preparation, implementation and analysis | Sample profile agreed with Project Steering Group Decisions made on considered, agreed, explicit basis Consistency built into pilot format Training and supervision/quality assurance for people who are carrying out the pilot Different ways used to check the validity of interpretation of responses including cognitive interviewing
|
7.4 The major difficulty in relation to the validation of the methods developed occurred during the pilot. These issues are discussed in the section on the pilot (see Chapter Eight below).