5. Survey design factors and complex standard errors
Data collected in surveys are always an estimate of the true proportions in the population. The accuracy of these estimates - the sampling error - can be calculated for any estimate in the survey using information about the proportion of people giving the response and the number of people in the sample (or sub-sample). The sampling error can be expressed as a 'confidence interval', which can be added to and subtracted from the survey estimate to give a range within which it is fairly certain that the true value lies.
Since the SHS is not a simple random sample ( SRS) design, the confidence intervals need to take account of the impact of clustering and stratification. The SHS, therefore, has what is known as a 'complex standard error'. While for some variables the design of the sample improves the precision of the survey estimates compared with a simple random sample, the overall effect of the survey design is to reduce the precision of the estimates. The relationship between the complex standard error and the theoretical simple random sample standard error for a sample of the same size is summarised in the 'design factor'.
The Taylor Expansion Method was used to calculate the complex standard errors for a series of results in the study. This is a well-established technique for working through the effects of stratification and clustering. As can be seen from Table 5-1, these ranged from 1.08 to 1.76. The overall average is 1.17, but that should not be taken as a 'typical' value, given the distribution of values across different variables. However, it suggests that the original assumption of a design effect of 1.1-1.2 was reasonable and using a value of 1.2 as a 'rule of thumb' for adjusting the standard errors of the survey data would account for the design factors associated with most variables in the survey.
The 95% confidence intervals shown are based on complex standard errors.
Table 5 1: Design factors and confidence intervals for key variables in 2005 data
Characteristics | Estimate | 95% Confidence Intervals | SRS error for the same size of sample | SHS Complex Standard Error | Design Factor |
|---|
Lower | Upper |
|---|
Tenure |
|---|
Owner-occupied | 65.6 | 64.5 | 66.6 | 0.38 | 0.53 | 1.40 |
|---|
Social-rented Sector | 25.0 | 24.0 | 26.0 | 0.35 | 0.52 | 1.49 |
|---|
Privately rented | 7.4 | 6.9 | 7.9 | 0.21 | 0.25 | 1.21 |
|---|
Below bedroom standard | 2.7 | 2.4 | 3.0 | 0.13 | 0.14 | 1.08 |
|---|
Property type |
|---|
Detached house | 20.8 | 19.6 | 21.9 | 0.34 | 0.60 | 1.76 |
|---|
Semi-detached house | 22.5 | 21.6 | 23.5 | 0.34 | 0.49 | 1.44 |
|---|
Terraced house | 22.3 | 21.2 | 23.5 | 0.33 | 0.59 | 1.75 |
|---|
Flat/maisonette | 34.0 | 32.9 | 35.1 | 0.37 | 0.55 | 1.46 |
|---|
Economic status of working age adults |
|---|
Full time employee | 49.7 | 48.5 | 50.8 | 0.51 | 0.58 | 1.14 |
|---|
Part time employee | 13.5 | 12.7 | 14.3 | 0.35 | 0.40 | 1.14 |
|---|
Self-employed | 6.5 | 5.9 | 7.1 | 0.25 | 0.30 | 1.19 |
|---|
Unemployed | 4.2 | 3.8 | 4.7 | 0.21 | 0.24 | 1.11 |
|---|
HIH or partner has a bank/ building society account | 91.0 | 90.5 | 91.5 | 0.23 | 0.26 | 1.14 |
|---|
Marital status of all adults |
|---|
Married/cohabiting | 49.0 | 48.3 | 49.6 | 0.27 | 0.33 | 1.23 |
|---|
Separated/divorced | 5.9 | 5.7 | 6.2 | 0.13 | 0.14 | 1.14 |
|---|
Single/never married | 38.3 | 37.7 | 38.9 | 0.26 | 0.29 | 1.11 |
|---|
Widowed | 6.8 | 6.5 | 7.1 | 0.13 | 0.16 | 1.21 |
|---|
Access to the internet | 50.8 | 49.8 | 51.9 | 0.42 | 0.54 | 1.29 |
|---|
Travel to work in a car | 60.1 | 58.8 | 61.5 | 0.59 | 0.67 | 1.13 |
|---|
Require regular care or help | 12.1 | 11.5 | 12.6 | 0.26 | 0.30 | 1.13 |
|---|
Reporting long-standing illness, disability or health problem | 34.1 | 33.2 | 35.0 | 0.38 | 0.45 | 1.19 |
|---|
HIH = Highest income householder