5 Data collection and analysis
5.1 Survey design
The questionnaire and covering letter were designed and tested by IPSOS-Mori, with reference to a steering group which included members of the Scottish General Practitioners Committee, ISD Scotland, NHS Boards, Practitioner Services Division and Scottish Government. The questions had to be designed to fit with GP contractual obligations around access.
The full report on the development and testing of the survey can be found here: http://www.scotland.gov.uk/Publications/2009/05/18142136/0
Advice was sought from the Lothian ethics committee, who advised that because this survey was about service evaluation, full ethics approval was not required. However, permission was received from the CHI Advisory Group to sample from the central CHI register of patients registered with a GP.
5.2 Survey distribution and returns
The survey work was carried out by Scottish Government approved contractors Advanced Data Services Ltd ( http://www.ads.uk.com/). The names and addresses of the people who had been randomly selected to receive the questionnaire were transferred to them by secure FTP. The forms were initially sent out in October 2008 to everyone included in the random sample, excluding people who were known to have died since the register extract was taken in August. Forms were posted back to a Freepost address, and logged by the data contractors.
Following the initial mailing, 26% of the questionnaires were returned by 7 th November. A reminder was sent on the 24 th of November to those who had not responded. This increased the response rate to 45%. It was decided to go ahead with a second reminder in January 2009 in order to maximise the response rate, and this brought the response rate up to the final 50%. At each stage, checks were made against the updated CHI registers to make sure people who were known to have died were not sent reminders. People whose mail had been returned as undelivered were also taken off the list for reminders each time.
All data entry and verification was carried out by ADS, to their quality standards that guarantee 99.98% accuracy in data capture.
5.3 Survey accessibility
Helpline
A helpline was available to assist patients to complete the survey and answer any queries. The help line was charged at local rate and was available from 1pm-7pm Monday to Friday and 1pm-5pm on a Saturday in the periods following each survey mailing. Telephone completion was available for people who had a disability or were unable to return a paper form.
Translations
Translated forms were available on request in the following community languages: Arabic, Bengali, Cantonese, Hindi, Polish, Punjabi, Urdu.
Information
All GP surgeries were provided with an information pack including leaflets to inform staff about the survey, and posters so that patients who attended would know about the survey. Background information including Q&A were provided on the Scottish Government website.
5.4 Data analysis
Data collected were securely transferred to the Scottish Government Health Analytical Services Division, where statisticians analysed the information using the statistical software SAS. Data were checked for duplicates, and information on age and gender were extracted from the CHI number. Patient postcodes were coded to the Scottish Government urban rural classification (see Annex F) and datazones to enable analysis by the SG Scottish Index of Multiple Deprivation 2006 4. Unique patient identifiers were removed so patient responses cannot be identified.
Weighting
Estimates for Scotland and NHS Boards are presented weighted and un-weighted. Un-weighted estimates are simply a sum of numerator and denominator for each practice located within an NHS Board. Weighted results are calculated by weighting each GP Practice result (proportion of patients who obtained 48 hour /advance access) by the relative practice size. The weight is calculated as the practice patient list size (of patients eligible for survey) as a proportion of the entire NHS Board patient list size (of patients eligible for survey). Weighting the results in this way provides results more representative of the population at NHS Board level.
Confidence Intervals
Confidence intervals for the results on both types of access have been published on the Scottish Government website for each GP Practice. Further analysis of the precision of the estimates based on the intervals rounded to the nearest percentage point is included in Section 3.6 of this document. The 95% Confidence Intervals for each practices were calculated using the following formula 5:
CI = 1.96 x SE
where SE=v(1 - n/N) x v(p x (1-p))/(n-1))
n=number of responses the practice result was based on
N= number of patients on the sampling frame for the practice
Example of calculation of confidence interval for a GP practice's result
For practice A, where the result (proportion of patients obtaining 48 hour access) is equal to 84%, with a denominator of 195, and the total number of patients in the practice eligible for the survey was 5,066.
Then p = 0.84, n = 195, N = 5066
SE = v(1 - n/N) x v(p x (1-p))/(n-1)
= v(1 - 195/5066) x v(0.84 x (1 - 0.84))/(195 -1)
= v(0.9615) x v((0.84 x 0.16)/194)
= 0.026
CI = 1.96 x SE
So, CI = 1.96 x 0.026 = 0.05
Expressed as a percentage (x100) is 5%
The 95% confidence interval for practice A, where the proportion is 84%, is 79% - 89%. We can be 95% confident that the result would be in this range if we were to survey the patients from practice A again.