CHAPTER TWO: METHODS
2.1 This section outlines the research methods used in the study and highlights any limitations on the data resulting from these methods.
Stakeholder Consultation
2.2 A meeting was held on 15th July 2008, to discuss project set up and the initial draft of the questionnaire. The Scottish Government and the Scottish General Practitioners Committee ( SGPC) of the British Medical Association ( BMA) were the stakeholders involved in the meeting with Ipsos MORI. Initial proposals for the format of the questionnaire and covering letter were presented by Ipsos MORI at this meeting.
2.3 The issues discussed at the initial stakeholder meeting are outlined below. Following the stakeholder meeting 2 further proposals for the format of questionnaire were proposed by Ipsos MORI.
2.4 Following further feedback from stakeholders the final versions of the cover letter and questionnaire that went into field on 24th July 2008 for cognitive testing, were agreed. These incorporated the preferred wording of all stakeholders. The initial drafts of the cover letter ( Annex A) and questionnaire ( Annex B) that went into field addressed the following issues raised by stakeholders.
General Issues
Time period to be referenced
2.5 Stakeholders agreed that the time frame respondents should use would be 12 months (instead of 6 months). This would increase the number of sampled respondents who would be eligible (and therefore reduce the number of questionnaires that would have to be issued). As this is a reasonably long time period, it was agreed that throughout testing, the accuracy of respondents' recall of events would be monitored.
Respondents contacting their GP surgery on behalf of someone else registered at the same practice (e.g. a child, elderly person)
2.6 At the meeting it was concluded that the questionnaire could be answered by respondents contacting their GP surgery on behalf of someone else as long as they were registered at the same practice. This was to ensure that as many contacts as possible were included in responses and that surgeries that prioritise children for appointments are not disadvantaged.
2.7 To ensure it was clear to respondents that they could include this contact, it was agreed that this needed to be included in the instructions below Q1. Stakeholders did not feel it was necessary to know who a respondent had contacted their GP surgery on behalf of.
Respondents no longer registered with the GP surgery listed
2.8 There was discussion surrounding what should happen if a person receives a questionnaire after they have changed their surgery. It was not anticipated that this would be a major issue as patient turnover was only about 8% per annum. As data should be updated within 3 weeks, this meant only 0.46% of the sample will have moved practice and not yet had their records updated.
Issues specific to the questions testing PE7
The use of the term " GP or other healthcare professional"
2.9 Stakeholders felt that this wording should be the same as that used in the official indicators. It was agreed that the first draft of the questionnaire would include the term " GP, nurse or other healthcare professional" and interpretations of "healthcare professional," in particular, would be monitored.
The use of the word "appointment"
2.10 The stakeholders felt that the term "appointment" was usually associated with seeing a doctor face-to-face. The Scottish definition of 48 hour access, which underpins the PE7 indicator, includes telephone and email contact. This wording was therefore deemed unsuitable as it might be misleading. It was agreed that the questions relating to PE7 would be phrased to exclude the word "appointment".
"Speaking" to a GP or other healthcare professional
2.11 Stakeholders felt that referring only to "speaking" to a GP or other healthcare professional was misleading and it was more appropriate to use the words "see or speak".
Giving specific time periods as response options in the first question
2.12 At the meeting it was agreed that gaining information about exactly when a respondent made their last appointment (e.g. less than 3 months ago, between 3 months and 6 months ago, etc.) was not necessary. It was only necessary to find out if a respondent contacted their GP surgery in the last 12 months.
"on the same day or on the next 2 days"
2.13 Indicator PE7 states that patients should be able to obtain a consultation with an appropriate health care professional within two working days. Stakeholders decided they wanted to use the same wording that was used in the indicators for the questionnaire. However, there was also some concern that the meaning of the term "2 working days" may vary amongst respondents.
2.14 It agreed that "within 2 working days" would be included in the questionnaire; however instructions outlining the meaning of the phrase, with examples, would also be included.
Collecting information about why the PE7 indicator was not met
2.15 It was agreed that it was important to collect information about why the PE7 indicator was not met, to ensure data about exemptions was collected. For example, if they did not see or speak to a doctor or nurse within 2 working days because they only wanted to speak to their own doctor.
Issues specific to the questions testing PE8
2.16 It was agreed that the terms "appointment" and "doctor" could be used in question 4 and 5.
Background questions
2.17 It was agreed that the background questions, in the initial draft, should include questions on language usage, ethnic group, disability and health conditions.
Cognitive Interviewing
2.18 Once the questionnaire and cover letter were designed, cognitive question testing was conducted on the proposed wording and instructions. This was to inform further development to ensure that the final letter and questions would be easily understood by a wide range of respondents.
2.19 Cognitive interviewing is a widely used approach to critically evaluate survey materials. It allows an investigation into the way target audiences understand, mentally process and respond to survey materials. For example, when a questionnaire is designed it is possible that the author may intend one interpretation of a question but find that respondents presented with the question adopt an alternate understanding. If cognitive interviewing is used successfully in testing questions, survey materials can then be modified to enhance clarity 3.
2.20 There are several different techniques that can be used in a cognitive interview. A technique called "retrospective probing" was deemed to be most appropriate for this study. Retrospective probing involves the interviewer presenting a question to be answered, the respondent answering it and the interviewer following up by probing for specific information relevant to the question or to the specific answer given (e.g. What does this question mean in your own words?). Respondents read the cover letter, completed the questionnaire and then we used retrospective probing.
The Sample
2.21 Overall, 40 cognitive interviews were conducted with a broad range of respondents. The sample was not intended to be statistically representative of the Scottish population, but was designed to include people from the main groups that we anticipated might:
- have trouble working out the meaning or intention of the questions
- have difficulty selecting the accurate response option
- have an objection to the inclusion of a question or the phrasing of the question/response option.
2.22 For the questions to be fully tested, it was important to ensure a range of respondents were included in the sample. We interviewed 40 respondents with the following characteristics:
| Number of interviews |
|---|
Frequency of accessing GP services |
|---|
People who have made an appointment with a primary healthcare professional in the last two months | 24 |
|---|
People who have made an appointment with a primary healthcare professional within six months but over two months | 5 |
|---|
People who have made an appointment with a primary healthcare professional in the last year but not in the last 6 months | 4 |
|---|
People who have not made an appointment with a healthcare professional in the last year | 7 |
|---|
People who have access to a GP practice with "open access" | 9 |
|---|
People who have most recently made an appointment for someone other than themselves | 6 |
|---|
English as a first language | 25 |
|---|
English as a second language |
|---|
Asian | 5 |
|---|
Polish | 4 |
|---|
African | 5 |
|---|
Turkish | 1 |
|---|
Socio-Economic Group |
|---|
AB | 7 |
|---|
C1 | 9 |
|---|
C2 | 13 |
|---|
DE | 11 |
|---|
Literacy Levels |
|---|
English as a first language and educated to degree level or above | 13 |
|---|
People attending an adult literacy class | 4 |
|---|
2.23 There was also a mix of men and women included in the sample and respondents were from a spread of age groups. Interviews were primarily conducted in Edinburgh or Glasgow. However, it was recognised that it was important to interview some respondents who live in rural areas as they are likely to have different access issues to those respondents living in a city. In total, 7 respondents were recruited from rural areas: 4 in the Borders and 3 in Perthshire.
Respondent Selection
2.24 Most of the sample was recruited from the general public using on-street recruitment (22 interviews). When conducting on-street recruitment, our recruiters used recruitment questionnaires designed to ascertain whether a potential respondent met certain sample characteristics. An interview time would then be scheduled which was convenient to the respondent.
2.25 The other interviews were recruited using a variety of approaches.
Three of the lower literacy interviews were recruited with the help of the Edinburgh University Settlement Community Learning Centre.
- Fifteen of the interviews were with English as a second language respondents:
- Three of these interviews were recruited on-street
- Seven were recruited through community groups
- Three had taken part in previous research where they had indicated their willingness to take part in future research (these respondents had only taken part in one previous research study in the past year)
- Two were recruited by snowballing 4 from respondents who had taken part in previous research (and agreed to be re-contacted).
Fieldwork
2.26 During the cognitive interviews respondents were presented with an envelope containing the questionnaire and cover letter. We asked the respondents to open the letter and do exactly as they would if they were at home.
2.27 We then used a combination of interviewer observation (for example, noting points at which the respondent looked puzzled or confused, where they hesitated, where they seemed to be taking care or where they seemed to skim over or ignore instructions or response categories) and retrospective verbal probing (asking a series of questions after the respondent has finished completing the survey).
2.28 During the course of testing, small changes were made to the questions for subsequent testing. Illustrations of the different versions, along with the rationale for the changes, are included in the relevant sections of the report.
2.29 Interviews were conducted using a topic guide 5 designed by Ipsos MORI in partnership with the Scottish Government ( Annex E). All interviews were conducted in respondents' homes. On average the interviews lasted around 45 minutes. The discussions were digitally recorded and transcribed. Respondents were given £20, in cash, to cover any expenses and to acknowledge that they had given up time to take part. All interviews were conducted in English.