Class Sizes, Staffing and Resources Working Group: Final Report

Listen

Chapter 2 - Class size research

Introduction

1. The CSWG realised from their earliest discussions that there were gaps in the research and data available regarding issues of class size. A starting point was the need to be aware of the main messages arising from existing research in this area so a literature review on class sizes covering the last 25 years was commissioned. A gap in knowledge was identified regarding the role of education authorities, head teachers and class teachers in making decisions about class sizes so a second qualitative review was undertaken. Finally, given the lack of evidence available for secondary schools a quantitative pilot study exploring the impact of class size on standard grade results was carried out.

2. This chapter presents the findings from these reports and summarises what we have learnt. It also identifies gaps still remaining in the knowledge of the CSWG and considers what additional research would be necessary to fill such gaps and the practicalities of implementing such research.

Class size literature review

3. The (then) Scottish Executive Education Department ( SEED) commissioned the Scottish Council for Research in Education ( SCRE) to review the literature on the effects of class size on teaching practices and pupils' attainment, attitudes and behaviour 6. The original review was conducted between September and November 2001. It included UK and international literature, mainly from the USA, published between 1980 and 2000. In 2006, the review was updated to include research published between 2001 and 2006. Evidence from research using different methodologies both qualitative and quantitative and using primary and secondary sources of data is presented here 7. A significant study of the effect of class size in primary schools in England is included, but no relevant Scottish studies emerged.

4. The literature review was based around seven research questions. These are:

  • Which class sizes provide the greatest benefits and what are the problems presented by other class sizes?
  • Which stages of education benefit most from different class sizes?
  • What is the impact of class size on the teaching process?
  • What is the impact of class size on pupils' learning?
  • What is the impact of class size on pupils' attainment?
  • What is the impact of class size on pupils' behaviour?
  • What is the impact of class size on characteristics such as attendance and pupil motivation

5. The criteria for inclusion of studies in the review were that studies should be about primary and secondary aged children and that they should be about class size, but not school size. Studies were required to be well designed and statistically valid and preference was given to articles and papers that had been subject to peer review. In total over 1000 articles were reviewed between the original and follow up study although many of these articles were based on the same relatively limited number of empirical studies.

The key findings from the research

6. The key findings from the literature review are replicated below organised under a number of key headings.

How good is the evidence?

7. The evidence is extensive and some of it is very good. However, it has to be accepted that no research is perfect and questions remain about the research designs and the statistical analyses employed, and also the generalisability of the findings. Benefits in most studies were measured by a narrow range of outcome measures, i.e. progress in reading and mathematics. In addition, some scores from different tests across different classes and year groups were 'pooled'. Few had baseline assessments for the participating children and attrition rates for follow-up studies were often high.

8. The Student Teacher Achievement Ratio ( STAR) project in Tennessee is often regarded as the 'gold standard' of class size research. However even this study demonstrated problems in research design as:

  • Not all students joined the experiment at the same time.
  • There was a sizeable rate of attrition for the experiment and the impact of this was never explored although it is likely that it was skewed to lower socio-economic groups.
  • Some parents were unhappy with class allocation and pressured schools to reassign their children.
  • Children were transferred between classes during the experiment.
  • There was an upwards drift in class sizes.

9. A large-scale study of the effects of class size in England - the Class Size and Pupil Ratio ( CSPAR) project - was undertaken by the London University Institute of Education in 1996. It aimed to overcome the difficulties encountered by other researchers by employing multiple methods. It followed two cohorts of children through their first three years of school. It used baseline measurement and adopted a multi-level modelling approach to examine the impact of school and pupil level variables. This potentially offers a more robust methodology and more reliable results.

Does class size impact on pupil attainment?

10. The evidence from studies conducted in the USA, in particular the large state-funded experiments, claim to have demonstrated an association between class size and pupil achievement, i.e. as class sizes reduce, pupil attainment rises despite weaknesses in research design around high attrition rates, a lack of baseline measures and a drift upwards in class sizes. The best known of these studies, the STAR study, following children from kindergarten, aged 5 to Grade 3 aged 8. There is some disagreement amongst researchers about how much classes must be reduced in size to achieve significant improvements in pupil performance: some argue that benefits are most marked in classes of fewer than 15 pupils (Achilles et al, 1993); while others (Glass & Smith, 1978) suggest that the major benefits from reduced class size are obtained as size is reduced below 20 pupils.

11. Evidence from the CSPAR study of reception, year 1 and year 2 classes in primary schools in England broadly confirms American results, reporting a decreasing score in pupils' literacy with increasing class size, and little apparent change in performance between class sizes of about 18 and 25, with low achievers benefiting the most.

12. In both the USA and England the evidence of lasting benefits beyond the early primary years seems to be weaker than for initial effects. Evidence from the STAR project in Tennessee claims that the benefits gained from being in smaller classes are still evident in later grades. This is not confirmed by English evidence, which found no evidence of an effect when pupils (aged 10-11 years) were assessed at Key Stage 2 ( KS2).

13. Some evidence from secondary schools in England shows that pupil attainment is higher in larger classes, but this is probably due to teachers assigning more able pupils to larger 'sets'.

Which stages of education benefit most from class size reduction?

14. Evidence from the STAR project in Tennessee showed that the benefits of class size reduction are most marked in the early stages of a child's schooling, i.e. kindergarten through Grade 3 (5-8 years), and with children from minority ethnic backgrounds. The impact on younger and less able children is confirmed by English evidence.

15. English evidence also shows that there is a possible 'disruption effect', in which benefits are lost, when children who have experienced small classes in Reception class move to larger classes in Year 1.

16. At the secondary stage evidence is inconclusive because of the tendency for schools in Britain to teach less able children in smaller sets. However, a study of General Certificate of Secondary Education ( GCSE) examination results in England reported higher results from larger sets, composed mainly of more able pupils.

How does class size manipulation impact on teaching practices?

17. Teachers in various studies in the USA and England believe that class size affects their teaching practices, in particular the way they organise within-class groups and the amount of time they can devote to individual children.

18. Project STAR in Tennessee was not set up in a way that collected observational

information about teaching practices. Better evidence about teachers' actual behaviour emerged from the various stages of the CSPAR project and The Primary School Grouping project in England. Based on quantitative measures of within class grouping 8, teacher questionnaires and teacher interviews, these projects report that:

a) The number of within-class groups increased with the increasing size of the class: small classes (under 20 pupils) had an average of just 3 groups; in larger classes it approached 6 groups.

b) Overall, the most common group size experienced by pupils was of 4-6 pupils. However, in classes of over 25, pupils were more likely to be in larger groups of 7-10 (such groups found in 24.3% of classes with 26-30 pupils compared to 6.3% of classes with 0-20 pupils) while in class sizes under 25 there was more likelihood of a pupil being in very large groups of 11 or over, including being taught as a whole class. There is also a tendency for the youngest children ( i.e. the Reception class) to be taught in fewer, larger groups.

c) More whole class teaching took place in small classes.

d) From teacher interviews and questionnaires 9 teachers expressed the opinion that being in groups of 7-10 pupils had a negative educational effect in terms of the quality of teaching, pupils' concentration and their contribution to group work.

19. Researchers in both the USA and the UK suggest that there is a difference between the way teachers indicate they would organise their classes if class sizes were reduced and their actual classroom practices. Researchers in both countries suggest that teachers need to modify their classroom practices, particularly the number and size of within-class groupings, to take account of different sized classes. Further training may be required.

20. Teachers in numerous studies in the USA and England report that smaller classes are easier to manage and that they are less concerned about discipline than in larger classes. There was no evidence to show that Teaching Assistants in England had had an impact on pupils' attainment. It is suggested that they have an indirect effect by allowing teachers to focus more on teaching. This is broadly supported by Scottish evidence (Wilson & Davidson, 2006) which notes that the majority of local authority and headteacher respondents reported that additional support staff funded by the Teachers' Agreement had made an impression on teachers' administrative workload.

What effect does class size reduction have on pupils' learning?

21. Teachers in USA and England claim that smaller classes afford them more opportunities to get to know children and devote more time to pupils' individual learning needs. Evidence from the USA suggests that small classes increase students' engagement with learning and reduce anti-social behaviour. The findings on prosocial behaviour ( i.e. students assisting, supporting and caring for each other) are less complete.

22. Evidence from the CSPAR project in England found that pupils in small Reception classes were more likely to be on-task than those in larger classes, but against expectations they found that class size did not affect pupils' on-task behaviour or peer interactions in Year 6 (10-11 year olds). The researchers suggested that this was likely to be due to the uniformity and intensity of curriculum coverage leading up to the end of KS assessments.

23. Observational studies of within-class groupings show little evidence of collaborative learning taking place amongst pupils: most appear to learn individually while sitting within groups. Pupils usually have more physical space within which to learn in classes composed of fewer pupils. However, little attention has been devoted to the impact of the classroom environment, space and furniture on pupils' learning in the research literature.

What is the impact of class size reduction on pupils' behaviour, attendance and motivation?

24. Teachers in a number of studies of smaller classes in the USA report that small classes are quieter and more easily managed than larger ones. Therefore, potential discipline problems are prevented from arising. In the STAR project, direct evidence of pupils' behaviour from their disciplinary records was absent. Most studies resort to proxy measures of behaviour, such as exclusion, 'drop out' and attendance.

25. Researchers in the STAR project, however, claim that fewer pupils who experienced smaller classes in the early years of schooling subsequently 'dropped out' of school at Grade 10 (16 years). In addition fewer are excluded and their average number of days absence was less than for those who have not experienced smaller classes.

26. Evidence from England shows that pupils in small primary classes have a more interactive relationship with their teacher, are more often the focus of the teachers' attention, but have fewer classmates from whom they can learn. There is some European evidence to show that the number of incidents of pupil pushing, crowding and other aggressive behaviour increases in larger classes within larger schools.

27. Overall, research suggests a complex inter-relationship between pupil behaviour and their attitudes towards learning and their attainment. Class size may be one influential factor but the evidence is inconclusive.

How much does it cost to reduce classes?

28. There is a continuing interest amongst policy-makers, practitioners and parents in many countries in class size reduction. In 2006, the maximum class size in Scotland is 30 for a single stage class P1-P3; 33 for a single stage class P4-7; and 25 for a composite stage class (Scottish Executive, 2006).

29. Over the past decade average class sizes in Scottish primary schools have been falling, as has the number of primary schools and pupils. The Scottish School Census 2005 shows that the average primary school class size was 23.6 pupils (compared to 23.9 in 2004). Composite classes had an average of 19.9 pupils (compared to 20.2 in 2004).

30. The average primary school class in the UK was 26.0 pupils, compared to an average of 21.9 in the countries in the Organisation for Economic Co-operation and Development ( OECD, 2004): class size ranged across the OECD from 15.6 in Luxembourg to 41.5 in Egypt.

31. Although researchers disagree about the outcomes of class size reduction, there appears to be a consensus that reducing class size is expensive. Some suggest that it is the most expensive educational policy option that can be chosen. Some other countries, particularly the USA, have allocated billions of dollars to class size reduction.

32. Economists seem to be divided in their opinions as to whether a policy of class size reduction is a sensible use of resources.

Conclusion

33. A far from straightforward picture emerged in 2001 as much of the existing evidence was at best confusing, sometimes even contradictory. By 2006, although the class size debate had tended to polarise researchers, more of a consensus was emerging. While most researchers agree that there is a relationship between small classes, especially in the early years, and pupil attainment, many accept that this is only part of a complex picture. Classroom processes, the quality of teaching, the prior attainment of the child and parental background, are all likely to contribute.

34. Other researchers, while accepting that class sizes should be reduced, claim that there are more cost-effective ways of providing young children with individualised attention when they most need it. Class size reduction is attractive because it maintains the existing structure of schools while simply adding more resources. This may be a necessary step, but there is no evidence that in the long-term it will be sufficient to raise the attainment of all pupils.

Source: Wilson, V. (2006) Does Small Really Make A Difference?

Class size decision making research

35. York Consulting Limited ( YCL) was commissioned by the Scottish Executive Education Department ( SEED) to explore the process of making decisions about class sizes in Scottish education authorities and schools in May 2006, to inform the work of the Class Size Staffing and Resources Working Group. The purpose of the research was to explore the processes and reasons behind the decisions taken by education authorities and school managers and to explore how these are played out at a classroom level by teachers.

36. The methodology comprised of 8 case studies of Education Authorities ( EA) across Scotland. Within each EA, the case study involved one interview with an EA representative and visits to four schools: two primary and two secondary schools. Within each school, up to four individual interviews with teachers were undertaken. 101 school interviews were completed in total.

37. YCL facilitated a workshop event towards the end of the study to discuss the emerging findings emanating from the research. The workshop involved research participants and members of the Class Size Staffing and Resources Working Group and the Research Advisory Group. The final report from this study was not published separately, instead the main findings are reproduced below.

The key findings from the research

Class Size Policy

38. All interviewees were aware of current national policy and statutory limits on class sizes. National class size policy has a significant influence on the decisions made by EAs and schools in relation to class sizes. All EA and school representatives reported that they comply with the national framework and follow the guidelines as routine. Class size policy is extremely influential because EAs and schools fear negative staff reaction and potential trade union activity. The policy has limited influence in small schools with small classes, because they operate well within the maxima.

39. No EA has developed an internal policy in relation to class sizes, but all have internal procedures and processes in relation to allocating staffing and structuring classes. The procedures are generally consistent across EAs.

40. Class size policy had only been breached on a small number of occasions. In all cases, this was a rare event and occurred due to exceptional and unforeseen circumstances. Class sizes were usually breached for a short period of time, until a particular problem was resolved. In all but one case, the respective EA was aware that the class size limit had been breached.

41. All EAs have some process for review of class sizes. Only one EA has conducted any local research in relation to class sizes and/or the development of policy.

Decision Making Process

42. The decision making process is fairly consistent for all schools involved in the study. The decision making process involves the following key stages:

Stage 1: The school predicts the pupil roll for the following session and submits to the relevant EA. The pupil roll is estimated by identifying: the new intake at P1 or S1; the leavers at the end of P7 or S6; and the staying on rate at other stages.

Stage 2: The EA determines the number of FTEs and school budget. The number of FTEs is determined by a staffing formula, which takes into consideration time for management, non-contact time; deprived area enhancement; school capacity; absence cover and CPD. The formula used varies by EA area, sector and the size of the school. Schools receive a fixed budget in relation to the number of pupils and FTEs they are entitled to. The Head teacher may request an adjustment if they disagree with their allocation. Schools receive a devolved budget for absence cover and CPD and may choose to use this to enhance staffing, if required. They may also use funding from specific government initiatives to enhance staffing.

Stage 3: The Head teacher feeds back to the School Board the budget, predicted school roll and staffing allocation. The Board may raise concerns but they have no influence over the decision making process.

Stage 4: The Head teacher, Depute or Time-tabler allocate staffing across the school. Schools have considerable autonomy and flexibility in allocating staffing within the school and the EA would rarely encroach on this process. Lack of suitable accommodation emerged as the most significant factor affecting the decision making process. Recruitment was reported by interviewees as a significant problem in some schools, particularly those in less desirable, isolated or rural locations.

Stage 5: Departmental Heads or Principal Teachers allocate teachers and pupils to classes. The allocation of teachers and pupils to classes varies by school and largely depends on the pupil roll and the number of pupils per year group. The method of dividing a year group into classes (sections) also varies by school. Some schools split by pupil ability; others prefer mixed ability classes and split them on a more random basis. The method of dividing a year group can also vary within a school. When schools split classes by pupil ability, many will attempt to fill higher ability classes to the maximum to enable smaller classes for pupils with lower ability.

Stage 6: Class lists and timetable are distributed to class teachers for discussion/consultation. The level of consultation with class teachers varies by school, but appears to be greater in the primary sector. Most class teachers have some mechanism to feedback to the Head teacher, Depute or Principal Teacher any concerns with the suggested class structure. In a small number of occasions, this has resulted in changes to class structures.

Stage 7: Pupils and parents are informed of the class allocation. Parents and pupils have no influence over the decision making process, but they may raise specific issues after a decision has been made. Parents/carers are more likely to complain about the allocation of their child to a particular class, rather than class size per se. On a small number of occasions, parents have put pressure on the Head teacher to make a change.

Stage 8: The school roll is confirmed during Census Week. The final roll is based on the number of pupils who enrol at the school during Census Week. The predicted roll is usually fairly accurate and therefore only minor changes are needed at the start of term. Changes usually occur due to fluctuations in the resident population, largely as a result of new employment opportunities, new house building activity and travelling families moving into the area.

Stage 9: The EA may enhance or remove staffing based on the actual number of pupils. If the pupil roll exceeds, or is less than, the predicted roll, the EA may enhance or remove staffing as required. This may require a re-organisation of class structures.

Stage 10: Staffing and class structures are reviewed by the school throughout the year. Schools take account of any late intake, pupil movement during the year and staff changes. Schools usually leave space for late intake at the start of the academic year and therefore there is usually room to accommodate. In a school at maximum capacity, any late intake may require a class restructure and/or the recruitment of an additional teacher. Pupils move between class sections throughout the year, but this rarely results in a class reorganisation. Staff changes or staff absence may put short term pressure on class sizes.

Stage 11: Special needs teachers, learning support teachers and classroom assistants provide additional support where required. Support teachers can provide invaluable support to teachers but they are not used to cover classes or as an attempt to reduce class size limits.

43. Most EA and school representatives reported that the decision making process was generally very effective.

Teaching and Learning Strategies

44. Without exception, class teachers believed that smaller classes improved the quality of the teaching and learning experience. However, they also recognised that very small classes can be incongruous to learning and that larger classes are required for certain activities.

45. Teachers have considerable autonomy to organise a class as they see fit; to introduce within-class groupings; and to employ a range of teaching strategies, as required. Class size is just one of a number of factors that influence their approach. Other factors include: physical space; type of class; subject area and the demands of the curriculum; year group; composition/dynamics of the class; resources available; and the experience and confidence of the teacher.

46. All class teachers agreed that smaller classes provide greater choice and flexibility in the use of teaching and learning methods. Smaller classes are believed to improve the quality of teacher and learning for a number of reasons: they provide greater individual attention; they provide greater opportunity for practical and interactive activities; they allow greater interaction and participation of all pupils; they involve closer supervision; they provide greater physical space; and they provide more opportunity for formative assessment.

Concluding observations

47. Although the decision making process in relation to class size was regarded as generally very effective, a number of key observations emerged:

  • EA representatives, Head teachers and class teachers would like staffing to be further enhanced to reduce class sizes across the board and enable greater flexibility;
  • Head teachers would like greater clarity about the extent to which they can vary the class size maxima to accommodate local needs and requirements;
  • EA representatives and Head teachers would like a policy statement from the Executive setting out the longer term strategy in relation to class sizes;
  • class teachers would like the level of communication between the Executive, EAs and schools to be improved;
  • class teachers would like greater consultation regarding the introduction of new policies that directly affect them;
  • EA representatives and Head teachers would like the wider factors, such as accommodation and staffing, to be considered when implementing new policies;
  • one EA would like new legislation to prevent appeal panels from placing pupils into classes at the maximum limit.

Source: York Consulting Ltd (2006) Exploring the process of making decisions about class sizes (unpublished)

Exploring the impact of class size on standard grade performance in Scottish Schools - a pilot study

48. The SEED Class Sizes, Staffing and Resources Working Group commissioned the design and piloting of a quantitative study of the effects of class size on pupil attainment at Standard Grade, based on existing administrative data during 2006. The aim of the exercise was to:

  • critique the rationale and methodology adopted;
  • identify and implement methods for analysing and presenting the data from the pilot;
  • set out any apparent conclusions from the findings;
  • make recommendations on the desirability and feasibility of mounting a larger-scale and definitive exercise in 2007, and
  • if recommended as desirable and feasible, prepare an outline specification for the larger-scale exercise.

49. The work was undertaken by Linda Croxford at the University of Edinburgh. As with the above study on class size decision making, this study has not previously been published so the main findings are presented below.

50. Reviews of previous research show that in order to make causal inference between class size and pupil attainment the design of an observational study of class size should:

  • be multilevel, that is, with measurements taken at different levels of pupils, classes and schools to control for variation between schools, classes and pupils;
  • include measures of baseline attainment prior to class allocation;
  • collect robust measures of actual class size experienced by pupils;
  • collect longitudinal data over two years in order to capture changes in class size experienced by pupils.
  • Design issues for the pilot and proposed larger-scale study

51. As this pilot needed to be completed by mid 2007 to enable the SEED Class Sizes, Staffing and Resources Working Group to report by Autumn 2007 the timescale required to provide evidence was a major constraint on design. Therefore the pilot used pre-existing data over 3-4 months rather than a longitudinal study over two years as recommended by the research literature. Hence within the proposed tight timescale it was not possible to produce data that is sufficiently valid and robust to provide "definitive" evidence of the effects of class size on attainment.

Methodology adopted

52. The methodology was based on comparison of a pupil's attainment in the subject of focus with her/his average attainment in other subjects, which is the "relative values" approach used by Standard Tables and Charts ( STACS). This method is not as appropriate as a "value added" approach (which measures pupils progress after taking account of prior attainment), but relative values are used because no systematic measures of prior attainment are available.

53. Differences in practice by schools and local authorities with regard to presentations for Standard Grade and other National Qualifications ( NQ) created complications for the design of the study. Not all pupils take the same exams at the same time.

54. The study assumed that pupils had experienced the same class size throughout S3 and S4, and that this class size was represented by data recorded in the February prior to the S4 examinations. However, measures of class size at a single time point may misrepresent the pupils' real class size experience if there is movement of pupils between classes during the course of S3-S4, especially after preliminary examinations.

55. A key feature of the study was that it is based on existing administrative data, with very little additional data collected from schools. By definition, administrative systems are designed for the day-to-day running of schools, and they do not collect information about classes and pupils in the form that is needed for research. Administrative data required considerable manipulation by local authorities in order to try to create variables for analysis, with consequent time and resource implications.

56. The design of the pilot Scottish study had some advantages, especially the fact that it had a multilevel design that reflected the hierarchical structure of the school system within which it was based. The design of data collection and analysis allowed for the fact that pupils within the same classes and schools will obtain more similar results than pupils in different classes and schools. Further advantages are that the study includes a range of subjects. There are, however, major weakness in the design, arising from:

  • Inadequate measures of class size.
  • Lack of data on changes in class size over S3-S4
  • Lack of measures of baseline attainment prior to class allocation.

57. These weaknesses cause serious doubt to be cast on the findings of the analyses as providing definitive answers on the impact of class size in respect of attainment outcomes.

Pilot findings

58. These findings are for illustrative purposes only, and do not provide evidence of the effects of class size on attainment. The findings refer only to S4 in North Lanarkshire in 2006, and should not be generalised to the whole of Scotland.

59. From the pilot study data there appears to be no evidence that smaller class sizes are associated with higher attainment.

60. The effects of class size appear to differ between the three subjects. In the pilot study, maths attainment is significantly lower in smaller sized classes than in average or larger classes even when pupil characteristics, level of study and class context are taken into account; this result is similar to findings by Massey (1997) for GCSE. In chemistry and modern studies class size does not appear to have an effect on attainment once pupil characteristics, level of study and class context are taken into account.

61. Pupil's own concurrent attainment, as measured by Mean Exclusive Points ( MEP), and level of study, are very strong predictors of attainment in all three subjects. In addition, the contextual effect of the class, as measured by average MEP, has a strong influence on attainment. The strongest effect of class composition is found in mathematics, and this is probably associated with a greater degree of setting by ability in maths.

62. Schools make a difference - there is significant variation in attainment at school-level, after taking account of all the factors in the model. It is possible that some of the variation might be explained by socio-economic factors that are not included in the model.

63. Teachers make a difference - there is significant variation in attainment at class-level, after taking account of all the factors in the model. This variation is smaller in modern studies than in maths or chemistry. In addition, for maths and chemistry, the relationship between MEP and attainment is not the same in all classes: in some classes pupils with high MEP gain a considerably greater advantage than in others.

64. Pupils make a difference - there is a great deal of variation at pupil level, after taking account of all the factors in the model. This may be associated with attendance, attitudes to the subject in question, self-esteem, behaviour etc that are not currently measured.

Source: Croxford, L. (2006) Exploring the Impact of Class Size on Standard Grade Performance in Scottish Schools: Report on the Pilot Study (unpublished)

The way forward

65. It was agreed by the SEED Class Sizes, Staffing and Resources Working Group that the work commissioned by the CSWG to date still had not provided answers in relation to optimal class sizes overall due to the complexity of the questions asked and the limitations in the analysis possible based on the available evidence collected. While some limited information from the USA and England existed to show that certain groups of pupils 10 benefited from reduced class sizes early in their school career, research regarding the impact of class sizes in secondary schools was lacking either in Scotland or abroad. This means that the CSWG was not in a position to make definitive recommendations about the class sizes that should be adopted throughout the secondary school years. While the CSWG is of the view that available evidence from England could be used within a Scottish context to make comments about the primary years, at secondary level no such evidence exists so specifically Scottish research would be useful. Therefore it was decided by the CSWG to focus future research efforts on Scottish secondary schools and to develop a proposal for a larger piece of longitudinal research to be undertaken as secondary level. This section of the chapter outlines what such research might look like.

Methodological problems

66. In developing a possible future research specification, we drew heavily on the pilot study undertaken by Linda Croxford (described above) and Peddar (2006 )11 who presents a valuable summary of the key methodological difficulties present in the main research designs used to assess the impact of class size. Even the most influential studies such as the Student Teacher Achievement Ratio ( STAR) project in Tennessee and the Class Size and Pupil Adult Ratio ( CSPAR) project undertaken by Blatchford et al in 1996 contain methodological particularities that make it difficult to generalise their work outwith the context of their particular setting. Hence before starting to plan for an new study to examine class size in Scottish Secondary Schools it is necessary to examine these methodological difficulties and consider how they might be overcome.

Measures of class size (allocation, changes, absences)

Croxford's (2006) pilot statistical study noted that class size measures were derived from school administration systems which did not always collect data in the way most suited to research. Numbers recorded in the school administrative systems did not always match up with the numbers presenting for exams (may be some drop out) at a particular point in time. In addition the numbers officially registered did not always relate to the numbers actually in the class and the numbers that turned up in class. Records of absence were not always sufficient to ascertain the impact on specific classes as they tended to record a pupil's absence on the day rather than the specific classes affected. .

What constitutes a class was also subject to some discussion - how are composite classes to be treated? What is the difference between a section and a class?

Changes in class size over the time of the study were not captured in the level of detail required for research purposes. The level of pupil movement between classes over S3-S4 was noted by some schools to affect up to one third of classes however such movements were not routinely captured by existing administrative systems.

The Student/ Teacher Achievement Ratio ( STAR) Project in Tennessee attempted to over come some of these problems by adopting a controlled experimental design which randomly assigned children and teachers to differently sized classes ranging from 13 pupils to 25 pupils. However while there may have been clarity at the start regarding class size this soon broke down as:

  • Not all students joined the experiment at the same time.
  • There was a sizeable rate of attrition for the experiment and the impact of this was never explored although it is likely that it was skewed to lower socio-economic groups.
  • Some parents were unhappy with class allocation and pressured schools to reassign their children.
  • Children were transferred between classes during the experiment.
  • There was an upwards drift in class sizes.

Baseline measures prior to class allocation

To measure the impact of class size, a baseline measure of attainment before class allocation is required to estimate value added. In Croxford's (2006) study this required a baseline assessment measure to be available at the end of S2 for each pupil before being allocated to S3 classes. Such a measure was not available. Therefore an alternative measure, Mean Exclusive Points ( MEPs) was used instead to control for pupil characteristics. This type of measure is used to measure general attainment and to compare this to relative attainment in the chosen subjects. It is not regarded as a suitable substitute for prior attainment as it is influenced by pupils other learning experiences in S3 and S4. In addition it was noted that class size bore some relation to the MEP score of the class, with larger classes having higher MEP scores. This makes the analysis more complex if the research regarding the negative impact of allocation to a lower set is taken into account. Therefore it is difficult to control for selection bias without some measure of prior attainment and some understanding of setting or streaming policies.

This is also one of the key weaknesses of the STAR project which did not start with a baseline measure of the children's attainment or ability.

This is potentially the most difficult problem to overcome in trying to design an extended project on class size. Without some measure of baseline attainment, there is nothing to measure a key component of the progress being made by individual pupils and a key element of the added value of schooling.

Problems measuring NQ attainment

The increasing divergence in practice amongst schools regarding the presentation of pupils for standard, intermediate and access exams in S3 and S4 provides a number of complications in how pupil attainment should be measured at a point in time. If we are interested in the impact of class size from the beginning of S3 to the end of S4 how do we treat pupils who present for exams at different times and hence reflect the impact of different amounts of teaching. We need to consider how the attainment of such pupils be recorded and what level of attainment should be used?

Sorting out some of the above problems was time consuming at individual school level. It may be that the SQA is best placed to provide the required attainment data.

The suitability of school administration data

Schools' administrative data is maintained as appropriate for the day-to-day running of the school and as such is not in a form or always of a quality to support this research. The pilot research required considerable manipulation of data. While this is inevitable when using data collected for one purpose for another, it adds to the complexity of the research and the burden on schools.

The classroom as a 'black box'

Pedder's (2006) paper on class size studies highlights the conceptual problem of treating the classroom as a "black box' and inadequately examining classroom processes as possible mediating factors between class size and pupil attainment. He makes a persuasive case for the need to gain a detailed understanding of what actually happens in the classroom in terms of pupil teacher interaction and teaching approaches and criticises many other studies for paying inadequate attention to such issues. This is in line with Croxford (2006) who suggests that a review of learning and teaching processes in differently sized classes could add to our understanding. Wossman and West (2006)12 also add to this line of argument by indicating that class size effects are mediated by teacher quality. Even the STAR study can be criticised for its lack of identification of teacher effect and teaching practice. This entire issue is encapsulated by Wilson (2006) who comments that

Researchers in both the USA and the UK suggest that there is a difference between the way teachers indicate they would organise their classes if class sizes were reduced and their actual classroom practices.

The objectives of the research

67. To address our primary interest - the link between class size and attainment the need to define suitable questions. A longitudinal study should be based around the following:

1. Can we identify a causal link between class size and increased attainment?

2. What do we mean by attainment, academic results and/or improved attendance, behaviour etc?

3. Do we understand how and why such causality might work (more individual attention, different teaching methods, less class disruption, etc)?

4. If there are increases in attainment, of what scale and what are the resource implications?

5. Is there an optimum class size in terms of attainment?

6. Does this optimum change for different subjects or for different age groups?

68. To put in place a research project which has even a chance of delivering conclusions which are specific and relevant a design must be found that addresses the problems and weaknesses of previous studies listed above. Drawing on Wilson's (2006) literature review which contains a section examining the quality of the evidence captured by the most authoritative studies in this field, we suggest that a multi-method approach with a sufficiently longitudinal element and a set of baseline measures is needed to produce robust research. We need to consider how such an approach might be operationalised.

69. The following section contains an outline of a potential approach. It outlines a study that would:

  • take place over the period 2007 to 2010
  • contain quantitative measures of attainment based on a baseline collected in S2 compared to standard grades in S4 for selected subjects
  • involve the detailed collation of class size data
  • collect background pupil data to enrich the possibilities for analysis
  • involve a qualitative element though classroom observation to get inside the 'black box'.

It is extremely difficult to predict the cost of such a study with the number of unknowns we currently have. Not only is there the actual cost of researchers undertaking the work but there is also a significant cost in terms of burden and impact to the schools involved. If we decide to proceed, the first stage is to commission a scoping study to properly focus and cost the research.

Developing a large scale study

Introduction

There are a number of stages to developing a research design. The main issues needed to consider are summarised below. Each requires a significant amount of further discussion and debate to arrive at the optimum solution. The comments below are offered as initial prompts to start the discussion.

Asking the right questions

The question we are interested in is the link between class size and attainment. There are a number of aspects to this question:

7. Can we identify a causal link between class size and increased attainment?

8. What do we mean by attainment, academic results and/or improved attendance, behaviour etc?

9. Do we understand how and why such causality might work (more individual attention, different teaching methods, less class disruption, etc)?

10. If there are increases in attainment, of what scale and what are the resource implications?

11. Is there an optimum class size in terms of attainment?

12. Does this optimum change for different subjects or for different age groups?

Defining our sample

We also need to make decisions around what age range of pupils to include in the study and what subjects to focus on. Regarding age range, given the lack of research on secondary school pupils, it would seem appropriate to concentrate on secondary school. To provide us with a standardised measure of end attainment, we could use Standard Grade results, hence our study would end at the conclusion of S4 13. Therefore the best approach would be to study classes over a four year period from S1 or a two year preparation period from the end of S2. For reasons of timing and budget and the changes in class size coming about in S1 and S2, the latter proposal appears more realistic.

Our next consideration would be around the subject range we wished to cover. According to Croxford's pilot study there are 16 standard grade subjects with at least 10,000 candidates, all of which could usefully form part of the research. In addition we need to consider how if we want to use subjects with entrants at access or intermediate level at the end of S4. Croxford suggest that we choose based on the modes of the curriculum framework or on different teaching methodologies. Popularity might be the best basis for a choice or perhaps subjects presenting a wide range of attainment levels? The greater the number of subjects covered the greater the cost and the greater the burden on schools. However some element of comparison between different subjects would be useful.

The study population also needs to be defined. Croxford suggests a study population of 5-6 local authorities. However our choice of study population and sample depends on the level of analysis we wish to undertake. Our choice would be dependent on the extent to which we wished to be able to generalise across specific subjects or participant characteristics ( e.g. ethnicity, socio-economic characteristics) or perhaps local authority class size policies or teaching approaches. There are around 61,000 pupils in S3 and S4 in Scotland. We need to consider if we want to base our sample on random classes across the country adequate to achieve a representative sample of pupils or whether we wish to focus on a small number of authorities and include all eligible pupils ( i.e. those at the required grade studying the required subject(s)).

Obtaining the baseline

Obtaining a baseline whether we start at the beginning of S1 or the beginning of S3 is problematic. We do not have a consistently applied standardised testing regime that applies across the entire school population. There are three main options we could adopt to solve this problem.

1) Focus our attention on those authorities that already test their children at a suitable age. This option may skew our sample and reduce the reliability of the data obtained. It would also be unlikely that each authority would use the same testing tools so we would not be comparing like with like. In addition 'testing' authorities may be different to 'non-testing' authorities in terms of their focus on attainment. We would also need to demonstrate that such tests showed some link with eventual S4 results.

2) Carry out our own testing using tried and tested materials ( e.g.SSA materials). This would ensure that the testing was consistent however it would be expensive and burdensome on schools to undertake a testing exercise just for this purpose. A test of attainment and detailed socio-economic and contextual information would be needed.

3) Using SSA data for S2 to form the baseline. This potentially has greater benefits. The testing will be carried out anyway so there will be no duplication. If we used SSA 2008 we would avoid burdening schools and there are no other studies planned for that year. However we would need to look closely at the content of SSA 2008 and in particular the pupil questionnaires, used to obtain contextual information, to see if any rewriting was needed. We would also need to ensure that we do not reduce the response rate in SSA 2008 by requiring more of schools or local authorities. There would be serious implications for the SSA including boosting the sample which would need to be very carefully considered.

What ever method we choose we also need to consider the predictive power of the baseline we adopt. For example, does a good performance in a baseline test of English have any relationship with S4 performance in economics? There is research that shows the correlation between success in different subjects and it may be that this also influences the subjects we choose to include in the study.

Measuring class size

Adequate measures of the actual class size and not the nominally registered class size throughout the period of the study are essential as a basic measure for the study. This requires a standard definition of a class to be agreed across all participating organisations. Maintaining up to date measures of pupils officially registered for a class and the numbers actually attending will involve redesigning school administration systems to ensure that:

  • A correct record is kept of the numbers of pupils actually registered to attend the class
  • The actual number attending the class is recorded on a regular basis (or absences noted)

Without a correct record of class size throughout the duration of the period of the research, measurement of the dependent variable becomes less reliable which impacts significantly on the usefulness of the research. However ensuring that a correct record is maintained will place a considerable burden on each school involved in the research. This can best be minimised by the researcher visiting each school in advance of the research study and helping them to adjust existing systems or implement new systems to ensure that data is collected as required.

A study outline

A robust study would be defined as addressing all of the above questions and ultimately would provide the best possibility of gathered convincing evidence regarding a causal relationship between class size and attainment . To achieve this aim we need to develop:

1. A shared understanding of terminology and definitions based on existing literature.

2. A longitudinal study with a representative sample enabling us to generalise across Scotland (covering an appropriate mix of schools and pupils).

3. Reliable, up to date school data around class size and allocation, changes and absences.

4. A range of contextual and socio-economic data to allow us to control for exogenous factors (poverty, ethnicity, home circumstances etc).

5. A range of attitudinal data to allow us to control for endogenous factors (efficacy, ambition, self-esteem etc).

6. A measure of baseline attainment to provide a starting point.

7. A measure of subsequent attainment to measure 'value added'.

A research design based on the above requirements has the potential to provide us with causal evidence of the link between class size and attainment. However it will not explain how or why class size might make a difference. Accompanying qualitative research around learning and teaching processes, as described above, would be necessary to understand this difference.

The above requirements would result in a longitudinal study of some years duration with considerable input required from schools, local authorities and possibly the SQA. As an example of what this might look like, a suggested timescale for such a study is set out below, assuming commencement in Autumn 2007.

Sept 2007-June 2008

  • Finalise research designs with decisions made about research questions, definitions, exogenous and endogenous variables to be collected, methodologies (qualitative and quantitative) and the sampling strategy to be adopted.
  • Obtain final buy-in to the study from relevant representative bodies for the quantitative and qualitative studies.
  • Select local authorities and schools and obtain their buy-in for the quantitative and qualitative studies.
  • Work with local authorities and schools to establish data collection systems for the data required for the study (or to establish subsidiary systems if necessary).
  • Choose pilot sample and obtain permissions.
  • Develop an S2 baseline.
  • Select student sample and gain agreement to participation (based on pupils in S2 during this period).
  • Undertake testing of students (within a defined timeframe) at the end of S2.
  • Collect background data on students from ScotXed/local authorities.
  • Collect personal data from students via student questionnaires.

June - December 2008

  • Set up database and record initial test results, background and personal data on a pupil basis.

August/early Sept 2008 and on a termly basis thereafter

  • Collect class size data based on original registration for each pupil and add to database.
  • Repeat exercise at the start of every subsequent term noting reallocations.
  • Collect data on pupil absence at the start of every subsequent term.

Sept 2008 - Dec 2008

  • Undertake interviews and obtain data regarding class size decision making, setting, streaming etc.
  • Make arrangements for the qualitative site visits.

Jan 2009 - July 2009

  • Undertake class visits to observe (or record) teaching and learning in action in S3.
  • Undertake related teacher interviews.
  • Undertake second round of pupil questionnaires re personal factors.
  • Check background information with schools for any changes.

Sept 2009 - Dec 2009

  • Continue with analysis of the first round of qualitative site visits.
  • Make arrangements for the second round of qualitative site visits.

Jan 2010 - July 2010

  • Undertake second round of class visits to observe (or record) teaching and learning in action in S4.
  • Undertake related teacher interviews.
  • Undertake final round of pupil questionnaires re personal factors.
  • Check background information with schools for any changes.

Autumn 2010

  • Obtain NQ results from SQA.
  • Match results to pupil database.
  • Undertake analysis.

Spring-Summer 2011 - report

Other research proposals

70. The study could look at the before and after effects of class size reduction and also examine impacts on attendance, discipline and truancy.

Alongside the major study of class size and attainment within the secondary sector there are other aspects of class size policy that require evaluation. Firstly, the previous Scottish Executives commitment to reduce class sizes in maths and English classes to a maximum of 20 class size reductions are already taking place. The CSWG considered that it would be helpful if a small scale study was undertaken into the impacts of these reductions particularly in deprived areas 14.

In addition the current Scottish Executive policy of phased reduction in P1 - P3 class sizes to 18 will also require ongoing evaluation throughout its lifespan.

Below is a summary of how these policy initiatives could be evaluated.

Summary description of the proposed short-term study

A small scale study will be developed which measures the impact of the August 2007 reductions in S1/S2 English and Maths class sizes across Scotland. This study will use the experiences of students moving from S1 to S2 in a selected number of case study schools, chosen using measures of deprivation and HMIE inspection information, and their teachers to explore changes in attendance, discipline, attitudes and achievement. This study will commence in Autumn 2007.

Position re the long-term monitoring of the nursery, P1 and P2 class size reductions

As the government begins to implement its planned reductions in early years class sizes to 18, we will develop an approach to research and evaluate the impact of these changes on children's attainment, development, motivation and behaviour. This should be regarded as a longer-term project with results taking some time to emerge. In the shorter-term we will also seek to monitor and evaluate the implementation of these changes by schools, teachers and teacher training institutions.

Page updated: Friday, November 16, 2007