Scottish Government

ANNEX 4 - TECHNICAL APPENDIX

SECTION ONE - INTRODUCTION

1.1 DTZ, in conjunction with Research Resource and Dudleston Harkins Social Research, has undertaken research into Parents' Access to and Demand for Childcare (Survey 2006) on behalf of the Scottish Executive Education Department, Young People and Social Care Group.

1.2 The purpose of this Technical Appendix is to present the methodology for the research in more detail than was possible in the full report. In particular it will:

• Set out the sampling methodology for the quantitative research;
• Set out the questionnaire used in the quantitative research;
• Set out the methodology for the qualitative research;
• Set out the topic guide used in the qualitative research.

SECTION TWO - METHODOLOGICAL NOTES ON QUANTITATIVE
RESEARCH

2.1. Introduction

2.1 This section of the Annex Report sets out the methodology employed in undertaking the quantitative research elements of the study including a comparison with the methodologies used in the previous surveys, the sampling methodology and the statistical confidence of the results. The questionnaire used for the quantitative interviews is included as Appendix A.

2.2. COMPARISON WITH PREVIOUS CHILDCARE SURVEYS

2.2 The methodology used in the 2006 study differs from the previous studies given that we agreed following the pilot that it was important to talk to parents
not using childcare in addition to those using childcare and subsequently ensured that they would be routed to the relevant questions in the questionnaire. In summary, the methodologies employed were as follows:

• In 2000, both parents who had used childcare in the previous year and those who had used it at some point previously were included (88% in past year or 1,175).
• In 2004, only parents who had used childcare in the past year were included (100% in past year or 1,003). A follow up survey was undertaken with 500 parents to follow up on some key issues regarding childcare.
• In 2006, we interviewed 1,304 parents with children in the relevant age category and 42% used childcare (42% or 547 using in a typical week, with a further 7% or 95 parents who have used childcare in the past but have made a change to not using in the last 6 months). No follow up survey was undertaken and the key issues followed up previously were covered in the main survey.

2.3 In terms of comparison, it was possible to select only those parents who report childcare usage in our survey and compare to the previous studies. The sample size of 547 (or 642 where appropriate) is large enough to allow robust comparison with previous years results.

2.4 The advantage of the methodology used in the current study is that it allowed a much more accurate assessment of true demand and drivers for childcare and a more robust process of aggregating up to the population as we interviewed a representative sample of parents, not just those who currently used childcare (or have done so in the recent past).

2.5 Furthermore, our methodology built intelligent crosschecks within the survey, which allowed us to identify the true extent of usage of informal childcare. For example, at Question B1, 58% of parents stated that they did not use childcare, however, at section H on childcare ideals and preferences, an additional 95 stated that they would prefer to use, and currently used, a family member or friend/neighbour to provide childcare. This brought the actual proportion of parents using childcare to 49%. These parents would have been missed through the previous survey method, where the interview would have been terminated at the equivalent of QB1 as the survey only focused on childcare users.

2.3. SAMPLE FRAME - SCOTTISH HOUSEHOLD SURVEY

2.6 The Scottish Household Survey ( SHS) dataset was used as the sample frame for the survey. This was chosen as the sample frame due to the lack of availability of any record of parents through Child Benefit records because ownership of the databases has moved from the Department for Work and Pensions ( DWP) to HM Customs and Revenue. Furthermore, the 2004 study found difficulty in finding parents of children aged 14 years and under within particular geographical areas. This was due to the 2004 survey using a sample population that comprised all households within specified geographical areas rather than households with children aged 14 and under. In addition, the sample had not agreed to participate in further research so attrition was an issue.

2.7 One of the specific aims and objectives of the SHS is to permit disaggregation of such information both geographically and in terms of population sub-groups (such as families with children), and to allow detailed follow-up surveys of sub-samples from the main survey sample, if required. As such, we believed that the SHS could usefully be used in order to identify the parents of children aged 14 and under and, of those, a sample could be drawn of those agreeing to participate in further research.

2.8 One of the greatest benefits of utilising the SHS as a sample frame is that it is up to date. The latest annual report was available in July/August 2006 (from interviews were conducted during 2005), which fitted in with the timing of the childcare survey, and it was likely that the sample would be up-to-date and accurate in its profile. Moreover, the data were robust, representative, households with children could be easily identified and potential respondents were likely to be willing to participate.

2.9 The principal concerns in using the SHS as a sampling frame were to ensure that the sample size was robust and the sample was representative. There was a risk of self-selection, because the survey targeted only those who had agreed to take part in further research, but self-selection is a potential problem in all survey work, and using the SHS simply allowed us to identify in advance those who would have been most likely to opt out of research and exclude them from the sample frame. It is likely that these individuals would have declined to take part anyway. In fact, using the SHS allowed us some opportunity to design a sample that was at least partly adjusted for the self-selection bias that is inherent in all exercises of this nature.

2.10 Use of the SHS also allowed us to identify those households who had children aged from birth to 14 (the target group) at the time of the SHS interview. In the intervening period some households will no longer be in the target group ( e.g. children will have become too old) and others will have entered it ( e.g. children will have been born). Those households leaving the target group were allowed for in the addition of 5% to the SHS sample frame to allow for "deadwood."

2.11 There is no straightforward way to identify those entering the target group, however, where a household had a child under one, we focused on this child in order to boost the sample of households with children in this age group. In the previous study, finding households who had children in the target age group was problematic and, again, using the SHS allowed us to identify these households much more easily (albeit with some limitations at the very oldest and youngest ends of the target group).

2.4. SAMPLE SIZE AND SELECTION

2.12 In order to achieve 1,300 interviews, it was necessary to over-sample from the SHS database in order to allow for deadwood, refusals to participate, households moving on, non-contact and so on. Experience dictates, that a response rate of 60% is achievable for face-to-face surveys of this type. As such, the decision was taken to draw a sample of 2,100 households in order to achieve 1,300 interviews.

2.13 In selecting the sample, it was not possible for the SHS team to provide full contact details aligned to survey results. As such, an anonymised data file was provided which allowed a review of the profile of those agreeing to participate in further research for households with children under 14 in terms of their geographic location both in terms of local authority area and utilising the Scottish Executive's six-fold urban rural classification. As such, a sample of 2,100 households was drawn from the database of participants utilising their unique ID number. This was undertaken on a stratified random sampling basis, with sampled households spread across local authority and urban-rural classification. The list of unique ID's sampled was then sent to the SHS survey team who then provided a list of the sampled addresses, again independent of SHS survey results back to the Childcare survey team. This represented our sample for the survey.

2.14 However, it became clear during the early stages of fieldwork that we were experiencing in the region of 40% attrition in the sample due to households moving on, households being un-contactable and, in some instances, refusal to participate. Further exploration of SHS data revealed that a third of 'small family' households and over 40% of lone parent households have lived in their current properties for less than 2 years meaning that a high proportion of the sample are likely to have moved on, potentially explaining the high level of attrition.

2.15 As such, the decision was taken to draw a further 800 sample, again, spread across local authority areas. This, however, was only allocated in local authority areas where interviewers were falling short of their quota allocation. As such, additional sample was allocated in 11 of the 32 sampled local authority areas. Overall, the total sample allocated to achieve the desired number of interviews was 2,487 addresses.

2.5. QUESTIONNAIRE DESIGN AND PILOT

2.16 The survey questionnaire was broadly based on the questionnaire used in 2004, however, significant amendments were made in line with the project brief and survey objectives. Questions for which the data were deemed to be least useful in 2004 were selected for deletion in 2006.

2.17 In turn, the 2004 questionnaire had been based on the one used in 2000, but with some changes to content and order to make the interview run more smoothly. Thus, some of the questions in the 2006 questionnaire can be tracked back to 2000 and/or 2004.

2.18 Once a draft questionnaire had been agreed, a pilot survey was conducted. Two interviewers conducted 20 interviews. The pilot ensured that:

• The questionnaire structure flowed easily, thereby maintaining the interest of the respondent over the duration of the interview, which was not considered to be onerous;
• The routing of questions was complete;
• A range of respondents understood the questions. It was recognised that the questions had to be coherent and meaningful to people of different levels of ability.

2.19 Following the pilot, a few minor changes were made to the questionnaire. These changes involved ensuring appropriate routing was in place (particularly for parents not using childcare); clarifying wording of questions; and making sure pre-coded responses covered the full range of possible responses. The final questionnaire agreed with the Scottish Executive and used in the interviews is shown in full in Appendix A.

2.6. FIELDWORK

2.20 Research Resource Ltd was responsible for the fieldwork element of the project. A team of 19 interviewers attended a briefing session, which was conducted by Research Resource. The briefing session involved full instructions in the conduct of the survey interview. Written instructions were given to all interviewers.

2.21 Interviewers were assigned a target number of interviews to achieve within a particular geographical area and provided with a sample list that held approximately 40% more addresses than the desired number of interviews. Their instructions were to make at least four calls at an address at different times of the day/days of the week before classifying the address as a non-response.

2.22 Given the geographical spread of sampled addresses, it was recommended that interviewers telephoned the sampled addresses, in advance of calling in person, to make an appointment to complete the interview.

2.23 Respondents were selected within households by identifying the individual with either main or shared responsibility for the child(ren) aged 14 years of age or under in the household. Thereafter, where more than one child aged 14 or younger resided in the household, one child was selected as the topic for discussion in the interview. Where a child less than one year old resided in the household, this child was selected (as the previous survey had problems in selecting this group of children). Where this was not the case, the child was selected at random using the 'next birthday' rule ( i.e. the child whose birthday fell next within the calendar year was the one selected for interview).

2.24 Each sampled address was sent an advance letter from the Scottish Executive explaining the purpose of the survey and requesting co-operation. As a result of this letter, 72 residents contacted Research Resource to 'opt out' of the survey. These addresses were removed from the lists given to interviewers and these households were not contacted further by Research Resource.

2.25 Each interviewer was also provided with a 'letter of authorisation' to show on the doorstep. Interviewers were also instructed to carry their Research Resource photo-identity card at all times and to display this to all potential respondents. Each interviewer also carried a stock of leaflets that explained more about market research, the survey process and why participation is important. A leaflet was left with every respondent.

2.26 Fieldwork began immediately after the briefing session on 2 November 2006, and the fieldwork was undertaken throughout November and December 2006, with final interviews undertaken in January 2007 to complete interviewing. In total, 1,304 interviews were completed.

Table 2.1 - Summary of Interview Outcomes

2.7. DATA ANALYSIS AND STATISTICAL SIGNIFICANCE

2.27 The results of the survey were entered into the statistical software programme SPSS. This software was used to generate the results and undertake cross-tabulations of variables as required. A chi square test has been used to test for correlation between the variables where we have tested particular hypothesised relationships. Chi Square is a non-parametric test of statistical significance for bivariate tabular analysis (also known as cross-tabulations). In the main report, we only report on relationships where the chi square test is significant to at least the 95% level. ¹

2.28 In general, the data presented in the report are rounded to the nearest percentage point and the bases used for each of the tables and figures are provided in detail. Due to issues relating to data confidentiality, it was not possible for us to be given access to detailed information on the background of the respondents in the SHS forming our overall sample. Therefore, it is not possible to weight the responses in our survey.

2.29 The ONS provides a helpful discussion of standard errors in sample design. ² In general, the size of standard errors is determined by a number of factors, including the sample size and the variability of the population from which the sample is drawn. The third important factor in determining the order of standard errors is the sample design. Standard errors calculated from simple random samples will, typically, differ from those calculated from more complicated sample designs, such as clustered or stratified samples.

2.30 The SHS is not a simple random sample ³ and uses a method of calculating confidence intervals to take account of the impact of clustering and stratification. As we have taken a stratified random sample from the SHS, we have a stratified random sample of parents with children under 14 from the SHS who agreed to participate in further research (the sample was stratified by local authority and urban-rural classification). Therefore, it is not possible for us to calculate the confidence intervals, as these sampling effects cannot be controlled for by calculating a design factor. However, it would not be unreasonable to suggest, as per the ONS guidance, that the factors by which we have stratified the sample would reduce the standard error.

2.31 In discussion with the Scottish Executive and the SHS team, it was agreed to use the SHS confidence limits as a guide to the absolute minimum confidence intervals for the results of this survey. These are shown in Table 2.2. It should be noted that these intervals only tell us which differences are definitely not statistically significant, and we do not have a measure that tells us which differences are statistically significant.

Table 2.2 - Confidence Intervals Calculated from the SHS

Source: Scottish Household Survey

2.32 Table 2.3 compares the distribution of the achieved sample by local authority with the distribution of all households in the SHS with children aged 14 and under and who had agreed to participate in further research. The achieved sample is broadly representative of the population from which it was drawn, with a slight over-representation in Highland and under-representation in Glasgow City.

Table 2.3 - Comparison of Achieved Sample of Households with Children Aged 14 and Under Who Had Agreed to Participate in Further Research with Population from SHS

Source: Scottish Household Survey and DTZ/Research Resource

SECTION THREE - METHODOLOGICAL NOTES ON QUALITATIVE
RESEARCH

3.1. INTRODUCTION

3.1 This section of the Annex Report sets out the methodology employed in undertaking the qualitative research elements of the study including the interview method and the sampling of sub-groups. The topic guide used for the qualitative interviews is included as Appendix B.

3.2. QUALITATIVE RESEARCH METHODS

3.2 The most common qualitative approaches are either focus groups or individual depth interviews. Each method has important advantages and disadvantages, depending on the type of respondents involved and the issues being explored:

• Depth interviews are often the best method for in-depth probing of personal opinions, beliefs and values. The close respondent-interviewer dynamic can be beneficial in making the respondent feel comfortable and unjudged, as there is no social pressure on respondents to conform to the group dynamic. It also allows the interviewer to spend more time with an individual to probe their responses thoroughly and fully understand motivations and attitudes. Additionally, depth interviews are very flexible and, therefore, can be conducted at a time and location that is suitable to respondents. This method is preferable where the topic for discussion if potentially sensitive.
• Focus groups consist of an interactive group discussion where the researcher encourages the free flow of ideas. The group dynamic is a useful forum for developing new streams of thought and covering an issue thoroughly. Respondents often challenge each other's ideas and raise unexpected issues for exploration in a way that is not possible in depth interviews. However, there is an element of social pressure meaning that respondents can behave and respond in a way that makes them socially desirable. Thus, although the group dynamics can be useful for generating ideas and exploring and elaborating issues, it also means that respondents often feel a group pressure to conform and this can contaminate results. Additionally, focus groups can occasionally lead to "group think," where a few dominant people bias the group. There can also be practical problems in organising focus groups, if respondents are geographically dispersed or have difficulties attending due to other commitments.

3.3 For this study, we had proposed to use a mixture of focus groups and individual depth interviews to ensure that, where appropriate, respondents feel comfortable to express freely their opinions, while also including a forum for parents' preferences and motivations for childcare to be discussed and challenged. We proposed conducting depth interviews with parents who may not respond well to focus group situations ( e.g. some young parents, single parents) or where it is impractical to organise focus groups ( i.e. parents living in rural areas) and conducting focus groups with other groups of parents.

3.4 After careful review of the options open for the qualitative survey, we decided that the methodology should change to depth interviews only rather than also including focus groups and depth interviews. Given the ongoing pressures on the timescale for the fieldwork, we felt this method allowed us to cover the number of respondents we were aiming for most effectively.

3.5 We conducted 60 depth interviews with parents from 14 target groups as discussed below, as well as including 'typical' parents e.g. two-earner households or households where one partner stays at home to provide childcare (thus interviewing around 4 respondents from each group). The qualitative elements are the best way of analysing childcare demand and access to some key sub-groups of particular interest who may not be picked up in sufficient numbers in the quantitative survey, e.g. black and minority ethnic ( BME) households.

3.3. SAMPLING OF SUB-GROUPS

3.6 We conducted qualitative research with the target groups of parents used in the last study, while introducing 4 additional groups:

Groups used in last study

• Lone parents
• Young parents
• Student parents
• Parents and families living in deprived areas and parents and families living on low incomes
• Ethnic minority parents ⁴
• Parents of older children
• Parents and families living in rural areas
• Parents of disabled children or children with special needs and parents of children with special educational needs ( SEN)

New groups

• 'Typical' parents
• Parents in receipt of Child Tax Credit and parents receiving childcare vouchers
• Parents who use free pre-school education
• Parents with children who are eligible but do not use pre-school education

3.7 As mentioned, in reports on the previous study, these groups are not mutually exclusive, as parents may fall into one or two categories ( e.g. young and single parents). This has been borne in mind when interpreting the results. Whilst we have tried to reflect sub-groups of particular interest in the qualitative research, this aspect of the study has not been entirely focused on these groups and also reflects mainstream childcare demand, clarifying and probing in depth responses of a range of parents including those without special needs or characteristics.

3.4. QUALITATIVE RESEARCH TOPIC GUIDE

3.8 The topic guide used in the depth interviews is shown in full in Appendix B.

APPENDIX A - QUANTITATIVE QUESTIONNAIRE

Project Number P107

Project Name - Parents' access to and demand for childcare survey 2006

PARENTS' DEMAND FOR AND ACCESS TO CHILDCARE 2006

SECTION B: CHILDCARE ARRANGEMENTS OVER THE PREVIOUS WEEK

SECTION C: CHANGES TO USUAL CHILDCARE ARRANGEMENTS

SECTION D: FACTORS INFLUENCING CHOICE OF CHILDCARE PROVIDER

SECTION E: COSTS OF CHILDCARE

SECTION F: GAPS IN CHILDCARE ARRANGEMENTS

SECTION G: CHILDCARE INFORMATION AND PERCEPTIONS

SECTION H: CHILDCARE PREFERNCES

SECTION I: DECISIONS TO WORK, STUDY OR STAY AT HOME

SECTION J: DEMOGRAPHICS

APPENDIX B - QUALITATIVE RESEARCH TOPIC GUIDE

PARENTS' ACCESS TO AND DEMAND FOR CHILDCARE SURVEY 2006: TOPIC GUIDE

Introduction

Dudleston Harkins Social Research Ltd along with DTZ and Research Resource (independent researchers) are undertaking this project on behalf of the Scottish Executive. We realise that you have already completed a survey on issues surrounding childcare, and would like to thank you for agreeing to speak to us again. The aim of our chat is to probe in a bit more depth and your views on your current childcare arrangements, the available childcare in your area and so on. We are therefore talking to a range of parents in different circumstances and from different areas to gain a broad range of opinions and experiences.

All information provided will be treated with confidence and your name will not be used in any report - we will simply say, 'people thought' or 'people said'. With your permission, we would like to tape record our chat - this is only for us to listen to later so we can talk to you properly now without having to take notes?

Is that okay? Any other questions about the research? If not, we can start if that is okay with you?

Current childcare arrangements

[Check number and ages of children.]

Firstly, could you provide me with details of your current childcare arrangements? PROBE

• Who is your main childcare provider?
• How many hours of childcare do you receive on an average week? PROBE: When does that occur? Ask for details of childcare provider and days, hours and times of childcare used
• Could you explain whether you use different childcare providers at different times/for different reasons?
• IF USE MIX OF FORMAL AND INFORMAL: Why is that?
• To what extent does your employment status have an impact on your childcare arrangements?
• IF HAVE MORE THAN ONE CHILD:
• Do you use different providers for your children? PROBE FOR DETAILS
• Do you experience any particular difficulties/issues in accessing and using childcare because you have more than one child?
• What would you say are the main benefits of using these childcare providers: PROBE: Note differences in formal and informal provision, e.g. cost, educational opportunities, convenience, social interaction for child, opportunity for bonding between family and child etc.
• IF ONLY INFORMAL CHILDCARE:
• Are there any reasons why you would not use formal childcare? PROBE: cost, access, availability, trust/safety issues?
• In what circumstances would you change from using informal to formal childcare provision? PROBE: family size changing, changes to employment status, change to cost of formal childcare and so on.

Particular issues:

• Does your child have any particular needs? PROBE FOR DETAILS
• Has this affected your childcare options? PROBE: Why is that?
• Are there many childcare providers in the local area that suit your child's needs?
• What particular issues (if any) have you faced due to your child's needs?
• Do you rely any more on informal provision due to your child's needs?
• Have you experienced any particular issues in general accessing childcare? PROBE: Language difficulties, rural location, times to fit with work/study patterns

Pre-school education

• Does your child receive pre-school education? READ OUT: By pre-school education I mean provision based on the 3-5 curriculum (not just any educational activity) in the two years before Primary 1, it might be provided in a primary school, nursery or family-centre not attached to a primary school
• Where does your child receive pre-school education?
• Did you experience any difficulties securing a place for your child in pre-school education?
• What are your views on pre-school education: PROBE: Helpful, creates difficulties due to the fact it only runs for 2_ hours per day etc.

Reasons for choosing provider

• Why do you use your current childcare providers? PROBE: Convenience, cost, trust, high quality, access, availability, location, social opportunities for child, recommendation, educational opportunities
• Have you had to make any compromises to your lifestyle to use these childcare arrangements? PROBE: employment, lack of time with partner, cost and so on
• Would you say that your current childcare arrangements are ideal? Why/Why not?
• IF NOT: What would your preferred childcare arrangements be?
• Why do you not currently have these arrangements? IF DUE TO COST: Is this based on your perception of the cost, or have you checked out the cost of this childcare in your local area?
• Have you looked into accessing your ideal arrangements?
• Do you feel you had a real choice in deciding to work/study or not. Why/Why not?
• Was accessing affordable childcare a factor affecting your decision?

Cost of childcare:

• On average, how much do you pay each week for childcare: PROBE: To what providers? Do you pay for informal childcare in kind?
• How much of your household monthly income is spent on childcare? PROBE: manageability?
• If you could afford it, would you use more childcare?
• Has the cost of childcare limited your childcare options?

Gaps in childcare:

• Are there ever times when you would like to use childcare but cannot access it? PROBE: When, Why?
• What problems do you experience as a result of this?
• Does this happen regularly?

Childcare information

• How aware would you say you are of childcare provision in your local area? Why?
• Where have you looked for information on childcare provision? PROBE: word of mouth, local authority, Childcare Information Services, local newspaper, internet and so on.
• What kind of information were you looking for?
• How easy was it to access this information?
• How useful did you find this information? PROBE: Up to date, informative, detailed
• What further type of information would you like to receive on childcare provision and via what method?

Opinions/perceptions of availability of childcare

• Did you have any difficulties accessing childcare providers in your area? PROBE: What problems? Why?
• Did you have to spend much time on a waiting list?
• Are you currently on any waiting lists for other childcare providers?
• Would you say there are enough childcare providers in your area?
• What is your opinion on available childcare provision in your area? PROBE: hours available, access, locality, quality, training, safety/trust issues
• Would there be any measures that could be taken to reassure you on the concerns about providers PROBE: training, qualifications, disclosure checks.

Any other issues

• Are there any other childcare related issues that you would like to raise or discuss that we have not covered?

THANK RESPONDENT AND CLOSE