Scottish Government

2 The Scottish School Leavers Surveys

2.1 The study is based on the Scottish School Leavers' Surveys ( SSLS).
The SSLS were nationally-representative, general purpose, postal-questionnaire surveys of young people in Scotland. They were initially commissioned by the Scottish Office Education Department and subsequently by the Scottish Government.

2.2 The surveys started in the 1970s as surveys of young people who had recently left school from S4, S5 or S6. From 1985 the SSLS was redesigned to include a new cohort series focusing on cohorts of young people after completion of their final year of compulsory schooling (in Scotland this is the S4 year-stage). The data are longitudinal, and include young people's outcomes and experiences of school to age 16, and their outcomes at age 18/19 including post-compulsory transitions and labour market, (with some cohorts surveyed again at 22/23).

2.3 The SSLS cohort surveys were carried out on a fairly regular basis from 1985 to 2005. Surveys from 1985-1991 were designed and conducted by the Centre for Educational Sociology ( CES), but since 1992 surveys were designed and conducted by the National Centre for Social Research (Natcen) and its Scottish branch Scotcen. Sample details were provided by schools from administrative data.

2.4 In recent years the design of the SSLS as a postal questionnaire survey has become problematic because of low response rates (an increasing problem for postal surveys more generally). Various options for its future were reviewed (Howieson et al 2008) prior to the decision being made to discontinue the SSLS. Consequently 2005 is the endpoint for the cohort data time series.

Table 2.1: Scottish School Leavers Survey: youth cohort time series

2.5 As part of the recent ESRC-funded Education and Youth Transitions ( EYT) project , a research team at CES brought together data from the SSLS and the England and Wales Youth Cohort study ( YCS) to create comparable cohort time-series datasets. The Scottish time series constructed for the EYT project (see Table 2.1) started with cohorts of young people who completed secondary schooling in 1984 and ending with Scottish cohorts who completed S4 in 1998. The first activity of the current sponsored research was to extend the Scottish cohort time series by adding data for the last SSLS cohort, who completed S4 in 2002, and were aged 18/19 in 2005.

Survey response, attrition and weighting

2.6 The reliability of findings from a survey such as SSLS is crucially dependent on the extent to which young people have been willing to respond to questionnaires. In the early years of SSLS, response rates were extremely high - 81% of the 1984 cohort responded to the first sweep of the survey. However, over time response rates have fallen markedly, and just 45% of the 2002 cohort responded to the first survey sweep (Table 2.2). Table 2.2 shows that respondents to the SSLS at age 18 averaged 50% of the initial target sample between 1984 and 1988, but were only 31% of the 2002 cohort.

Table 2.2: SSLS: Respondents as % of initial target sample

2.7 This low response to recent surveys presents particular problems for analysis because respondents may not be representative of the whole cohort. Bias in the findings may result because some groups of young people are more likely to respond to the survey than others. Young people with lower attainment are least likely to respond to surveys, and thus are likely to be under-represented in the data, and males are less likely to respond than females (Lynn et al 1994; Howieson et al 2008). Not only does low response introduce bias into the results, but this bias may change as attrition increases, thus reducing comparability across surveys.

2.8 In an attempt to compensate for non-response bias in the results, weighting variables were constructed by the administrators of each survey from known characteristics of the sample and population. The purpose of weighting is to make the characteristics of the "achieved sample" of respondents more like the characteristics of the "target sample". The effects of weighting the 2002 SSLS cohort are demonstrated by Table 2.3 which compares the distributions of un-weighted and weighted samples by sex and Standard grade attainment. The first row of Table 2.3 refers to young males who did not achieve any awards at Standard Grade. In the un-weighted sample, there were just 48 respondents in this category. However, each of these cases was allocated a relatively high weighting in the data - an average weight of 1.97 - which meant that each was counted as (almost) two cases in the analysis - and almost doubling the total numbers in this category to 94.7cases. Thus, while there are 2.1% of male respondents with no awards in the un-weighted sample, this percentage increased to 3.7% in the weighted analysis.

2.9 The opposite effect is shown by row 6 of Table 2.3, which refers to high-attaining males who achieved five or more awards at credit level. Although there were 1043 male respondents in this category in the un-weighted sample, they were each allocated a relatively low weighting - an average weight of 0.82 - which means that the total number of cases in this category is reduced to 854.2. Thus, while there are 46.3% of male respondents with 5+ credit awards in the un-weighted sample, this percentage is reduced to 33.2% in the weighted analysis.

2.10 The final column of Table 2.3 shows the extent to which each category was adjusted to compensate for low response rates. The highest weights were allocated to low-attaining young men who had the lowest response rates. The lowest weights were allocated to high attaining young women who had the highest response rates.

Table 2.3: Comparison of un-weighted and weighted sample of the 2002 SSLS cohort by sex and Standard Grade attainment

2.11 Social class is an important variable for analyses in this report, and Table 2.4 provides some information about how the distribution of social class is affected by weighting. The categories with the highest weighting were males whose parents were in working class (1.33) or unclassified (1.49) occupations - suggesting that response to the survey by these groups was relatively low. The lowest weights (0.76) were allocated to females with parents in managerial and professional occupations, who were most likely to respond to the survey. From Table 2.4 there appears to be a slight difference in the percentage social class distributions of males and females in the SSLS 2002 sample - 38.9% of males were managerial/professional compared with 36.1% of females. We have no reason to expect these differences in the population of school pupils, and they may have arisen from variations in the quality of answers to the survey or coding of these responses (see below). There is also a possibility that these small differences may be influenced by unmeasurable differences in response and weighting, but there is no information about the social class characteristics of the target sample for comparison..

Table 2.4: Comparison of un-weighted and weighted sample of the 2002 SSLS cohort by sex and social class

2.12 Weighting systems can have considerable effects on the results of analyses, as shown by the percentage distributions in Table 2.3 and 2.4, and the validity of analyses based on weighted data are dependent on appropriate data and methods being used to calculate weighting.

2.13 For the 1984-1990 SSLS cohorts the weighting calculations compared numbers of respondents in the achieved sample by sex, attainment and stage of leaving school with numbers in the S4 school population. Similar ² weighting systems were calculated for all cohorts by the survey organisation responsible for each survey. However, although weighting can take account of known characteristics such as sex, attainment and region, it cannot fully correct for low response that is associated with unknown characteristics such as feelings of alienation from the education system. Potential bias must be borne in mind when interpreting the findings.

2.14 In this report, findings displayed in tables and charts are based on weighted data - using the weighting systems developed for each survey. Statistical models are based on un-weighted data, so that standard errors are not skewed by the weighting systems - in the statistical models the effects of non-response are mitigated by including relevant characteristics of respondents as terms in the model.

Constructing the time series datasets

2.15 The aim in constructing the time-series dataset from SSLS was to derive a set of variables for each cohort that is comparable over time. However, the task was not straightforward because of changes in survey design and question wording (See Croxford et al 2007). For example, questions were asked in different ways, or used different response categories. For some variables there are gaps in the time series because questions were asked in some survey years and not others, or coding of responses was inconsistent over time.

2.16 There were also major problems with the design of particular surveys.
In particular, the design of the 1992 cohort is so problematic that the data have to be omitted from most analyses. In 1992 the SSLS was redesigned as an annual survey of school leavers (who might leave from S4, S5 or S6), instead of a cohort. The 1992 S4 cohort was then "reconstructed" by taking S4 leavers from the 1993 SSLS, S5 leavers from the 1994 SSLS and S6 leavers from the 1995 SSLS. The S5 winter leavers were not included. This design was so problematic that in 1996 the SSLS was again redesigned as a cohort survey. However, there were also errors in the design of the first survey sweep of the 1996 cohort: the sample was divided between "leavers" and "stayers" on the basis of whether schools expected them to leave school at the end of S4 or stay on for S5 - questions about destinations were not asked of those included in the "stayers" sample, and consequently this information is missing for approximately 20% of the cohort (mostly "winter leavers").

Qualifications

2.17 A further change in survey design from 1992 onwards was in the way that data on courses, attainment and qualifications were derived. Surveys of cohorts 1984-1990 included detailed questions about curriculum and examinations taken at school. From 1992 onwards these detailed questions were omitted, and variables summarising attainment were derived by the Scottish Examination Board (and later the Scottish Qualifications Authority ( SQA)) and linked to the survey data. This linkage became easier when a unique Scottish Candidate Number was assigned to Scottish students in the mid 1990s, but data are missing for some respondents in cohorts 1992-1998. The focus on national qualifications achieved also reduced the survey information on curriculum, and provides no information on non- SQA qualifications from 1992 onwards.

2.18 Changes over time in the Scottish qualification system have posed considerable challenges for constructing the time series. Our aim was to develop derived variables for the time-series that are relevant to historical data as well the current situation. Our approach has been to make some assumptions about equivalences based on advice from SQA. For example, the Ordinary Grade of the Scottish Certificate of Education ( SCE) was the main qualification at the time the 1984 cohort were in S4, but this was gradually replaced by Standard Grade. For the purpose of constructing time-series variables we assume that each grade awarded at Ordinary Grade is equivalent to the same grade at Standard Grade (Table 2.5). Variables constructed by this means include the total number of awards at the equivalent of Standard Grade credit, general or foundation, and also an overall point score.

Table 2.5: Equivalence of qualifications at age 16

2.19 Similarly, the task of extending the time series to include the 2002 cohort had to take account of changes in curriculum and assessment arising from the Higher Still reform, and removal of age/stage restrictions on entry to National Qualifications ( NQ). For post-16 qualifications we have assumed equivalences based on the UCAS tariff and Scottish Credit and Qualifications Framework ( SCQF). For example, we have assumed that the old Certificate of Sixth Year Studies and A-levels are equivalent to Advanced Higher (Table 2.6). We further assume that a pass at Intermediate 2 is equivalent to a credit level award at Standard Grade and a pass at Intermediate 1 is equivalent to a general level award. Variables constructed by this means include the total number of awards at the equivalent Higher or Advanced Higher, a UCAS tariff score and the highest SCQF level achieved.

Table 2.6: Equivalence of post-16 qualifications

Social Class

2.20 Measures of socio-economic status are derived from information provided by respondents to the surveys about their parents' occupations.
These responses may not be accurate, as in some cases young people do not know what their parents do for a living, and the majority cannot fully describe their parents' occupations in the detail required for accurate socio-economic classification.

2.21 Each of the SSLS includes detailed coding of parents' occupations using the Standard Occupational Classification ( SOC) for government statistics pertaining at the time of the survey. However, there have been three different SOC classifications in the course of the time series. In addition, the key social class classification has been changed from the Registrar General's classification to the National Statistics socio-economic classification ( NS- SEC) (Rose and O'Reilly 1998). For the EYT project we developed a method of deriving the NS- SEC classification from the historic SOC codes in order to derive comparable measures of social class over time (Croxford 2005). The trends in parental SEC - based on the higher of mother's or father's occupational SEC - are shown in Table 2.7. The table shows a steadily increasing proportion of parents in managerial and professional occupations, and corresponding decline in the proportion in working class occupations.

Table 2.7: Socio-economic classification of occupations of parents of SSLS samples (weighted)

Comparison of parental social class with census data

2.22 In an attempt to validate these trends, we compared the social class distributions of parents occupations from the cohort surveys with published data from the census. This comparison was made after omitting "unclassified" cases, because this category has different meanings in the census compared with the surveys. However, we cannot expect the social class distributions to be the same in the cohort surveys as in the census because the likelihood of being the parent of a respondent to the cohort surveys is greater in the middle age-groups and lower socio-economic groups.

2.23 For Scotland, published data from the census compares the NS- SEC categories of the occupations of all people aged 16-74 in 1991 and 2001 ( GROS 2003, Table 16). These data are summarised in Table 2.8, columns 1991a and 2001a. However, the parents of the young people who responded to the Scottish youth cohort surveys are more likely to be in the age-group 30-59, so social class data for this age group were extracted from the detailed 2001 census tables, and are summarised in Table 2.8 columns 2001b. Social class of occupation of mothers and fathers of the SSLS cohorts 1990 and 2002 are summarised in Table 2.8 columns 1990c and 2002c.

Table 2.8: Scotland only: Comparison of social class distributions from Census 1991 and 2001 with Scottish School Leavers Surveys

Notes
a) All aged 16+ in employment
b) All aged 30-59
c) Parents of all ages

2.24 The social class distribution of fathers in the 1990 cohort is very similar to the census distribution for 1991. The subsequent increase in the percentage of fathers in managerial and professional occupations by the 2002 cohort is in the same direction, but a little lower, than that shown by the 2001 census. The downward trend in the proportion of fathers in working class occupations between the 1990 and 2002 cohorts is steeper than the corresponding trends shown by the census.

2.25 Mothers in both 1990 and 2002 cohorts were less likely to be in managerial and professional occupations than is the case for women in the census. However, these differences can probably be explained by the fact that women from managerial and professional backgrounds are less likely to have children than women from other social classes, and therefore less likely to have children included in a survey of school leavers.

2.26 We conclude that the distribution of social classes in the SSLS is not exactly the same as the distribution in the census, but differences are not very extreme, and may to some extent be explained by social class differences in age-related birth rates and their consequent effect on the sample of school-age children. However, we must be cautious in interpreting findings based on survey data.

2.27 Some questions about family background have been asked consistently of all SSLS cohorts. For example, the SSLS have consistently asked detailed questions about parents' main activity (full-time job, part-time job, unemployed, retired, doing full-time unpaid work in the home, other) which provided valuable information about socio-economic characteristics. Other questions, have been asked in slightly different ways in each survey, so that it is more difficult to construct comparable variables. For example, all SSLS from 1984 to 1998 cohorts included the question: How old were your mother and father when they left school? This provided a consistent measure of the proportion of parents who had experienced post-compulsory education, so that we could analyse trends over time, including the effects of parents' education on the educational experiences of their children. However the question asked of the 2002 cohort has been changed to: Did your parents (or step parents) get one or more Highers (or equivalent)? Thus, we no longer have a consistent time-series for parental education.

Distribution of SSLS samples

2.28 The SSLS aims to present a nationally-representative picture of the experiences of young people in Scotland, and the sample includes all schools except special schools. The majority of young people included in the SSLS samples attended schools in urban areas (Figure 2.1). The vast majority (95%) attended state-funded comprehensive schools with most of them attending non-denominational schools (Figure 2.2). These characteristics did not change over the period of this study, so Figures 2.1 and 2.2 show the distribution aggregated across the whole time series.