Scottish Government

3. Response rate analysis

3.1 Introduction

The response rate for the survey is the number of forms returned as a percentage of the number of people in the sample. In working out the survey sample size, a response rate of 44% was assumed for each GP practice, except for practices serving deprived areas where 30% was assumed. More information on the sample size calculation is provided in the methodology section.

A number of questionnaires were returned from people who had not tried to use their GP services in the last year, or had not answered all questions required. Although their returns are included in the overall response rate, they may not be valid for calculation of results for 48 hour access and / or advance access. In this chapter, we first look at overall response rates, and then look at the "valid" response rates for each of these types of access and how they relate to the accuracy of GP practice results.

3.2 Overall response rates for NHS Board areas

543,246 survey forms were mailed and 269,186 were returned across Scotland, giving a response rate of 50% for this survey. This was higher than the expected overall response rate of 42% (based on the assumptions described above). Of the forms that were not returned, 15,000 (5%) were undelivered by Royal Mail on the first mailing, with a further 11,385 undelivered to us after the first reminder.

Table 7: Response rates by NHS Board

The highest response rate for an NHS Board was for Orkney (66%) and the lowest response rate was for Greater Glasgow and Clyde (43%). Greater Glasgow & Clyde and Lothian had the highest rates of undelivered mail - most likely due to higher levels of mobility in the city populations causing more addresses to be out of date on the patient registers.

Response rates in all boards were higher than those assumed in the sample size calculation. Expected response rates ranged from 39% in Greater Glasgow & Clyde to 44% in the majority of board areas.

3.3 Response rates for GP practices

The information available on the Scottish Government website shows the details of response rates for each GP practice. The minimum for a practice was 12%, with the maximum being 75%. Chart 17 shows the spread of response rates for practices. The majority of practices (86%) had response rates above 40%, with more than half having rates above 50%.

All but 5 of the practices defined as "deprived" (>50% of patients live in Scotland's 15% most deprived areas) had response rates above their expected 30%. Of the "non-deprived" practices, 162 had response rates below 44%. Of these, half had rates of over 40%. Looking at the practices with response rates lower than 40%, most have relatively high levels of patients living in areas of multiple deprivation or else are located in inner cities or areas with high student populations.

Chart 17: Distribution of practice response rates

3.4 Response Rate by Patient Characteristics

Gender

The response rate was higher for females (55%) than for males (43%). It is notable that 10,376 forms (3.7%) sent to males on the initial mailing were returned by Royal Mail as 'undeliverable' compared with only 4,590 (1.7%) for females.

Age

The response rate increased with patient age. A response rate of 74% was achieved for those aged 60 years and above, compared with only 24% for the 16-29 year age group.

Chart 18: Response rate by age group

Urban Rural Classification

The average response rate was 46% for patients residing in an areas classified as urban compared with 59% for patient residing in an areas classified as rural. Chart 19 below shows the more detailed classification.

Chart 19: Response rate by urban rural classification

Deprivation

As survey recipients' area of residence becomes more deprived, response rate decreases. For people living in deprived areas, the overall response rate was 37% compared with 52% in other areas.

3.5 Valid Response Rate

Although the overall survey response rate was 50%, a number of responses were not useable within the agreed coding as shown in Annex B. For example, the respondents may not have tried to contact the practice, to obtain 48 hour access or to book an appointment in advance, or they may have missed out a question or ticked more than one option. Therefore the valid response rate is less than the overall response rate.

On average for 48 hour access, responses were valid for 198,420 (37%) of the original sample, whereas for advance access responses were valid for 122,393 (23%) of the original sample mailed. The calculation of sample size described in the methodology section allowed for a valid response rate of 34% (31% in Greater Glasgow and Clyde and 35% in other boards).

It is evident that while the valid response rates for calculating 48 hour access results are around or above the level assumed for sample size calculation as described in chapter 4, they are lower for advance access. This is because fewer patients have tried to book an advance appointment with their GP - just under half of those responding. While this may have been anticipated, as previous surveys in England have shown similar levels, it was not considered practical to increase the sample size by at least a further 320,000 to try to account for this. Therefore some reduction in accuracy around advance access results may have been inevitable.

Table 8: Valid response rates by NHS Board - 48 Hour Access

Table 9: Valid response rates by NHS Board - Advance Access

Gender

The percentage of returns that were valid for inclusion in the results calculation was higher for females than for males for both types of access. Females were more likely to have contacted their GP practice in the last year.

Table 10: Valid responses by gender - 48 Hour Access

Table 11: Valid responses by gender - Advance Access

Age

The youngest and oldest age groups were less likely to have valid responses on each type of access than the other groups. Valid response rates for 16-29 year olds were low, both because they had low response rates to the survey, and those who did respond had a lower chance of having contacted the GP surgery.

Table 12: Valid responses by age group - 48 Hour Access

Table 13: Valid responses by age group - Advance Access

Urban Rural Classification

For 48 hour access, the percentage of forms returned that were valid was similar in all types of area, so the valid response rate shows a similar pattern to overall response rate. However, the percentage valid for advance access was lower in the remote rural areas as fewer people had tried to book in advance in these areas.

Table 14: Valid responses by urban rural classification - 48 Hour Access

Table 15: Valid responses by urban rural classification - Advance Access

Deprivation

For respondents living in deprived areas, there was a slightly higher proportion of valid responses on 48 hour access than for those in other areas. Therefore while the valid response rate was still lower in areas of deprivation than in others, the gap was smaller than for the overall response rate.

Table 16: Valid responses by deprivation - 48 Hour Access

Table 17: Valid responses by deprivation - Advance Access

3.6 Accuracy of results for GP practices on 48 hour and advance access

The reported results for the percentages of patients in GP practices obtaining 48 hour and advance access have been estimated from a random sample of patients. As with any sample, if we had asked a different group of patients, we could have ended up with a different result. How different this result might have been depends on (a) the number of people asked (we can be more sure with higher numbers) and (b) the variation in reported experience from that practice (if nearly everyone we asked had obtained access, we could be more confident that this would be the experience of most other people in the practice than if only half had obtained access).

Confidence intervals provide a way of quantifying this sampling uncertainty. A 95% confidence interval gives a range that we can be 95% sure contains the "true" result i.e. the results we would have obtained had we asked the entire practice. 95% confidence intervals have been published on the Scottish Government website for the estimates of the percentages of patients in each GP practice obtaining 48 hour and advance access.

If, for example, the result for a practice on 48 hour access is 80% and the confidence interval is 5%, this means we are 95% sure the result should be between 75% and 85%.

Table 18 summarises the sizes of the 95% confidence intervals ( CI) that have been achieved from the survey for Scottish GP practices. (These have been rounded to the nearest %.)

Table 18: Size of confidence intervals for GP practices' 48 hour and advance access results

It can be seen from table 18 that despite the lower valid response rates for advance access, 71% of practices have 95% confidence intervals within +/- 7%, and 96% within +/-10%. For 48 hour access 98% of practices have 95% confidence intervals within +/-7%.