Scottish Government

EXECUTIVE SUMMARY

Background

1. This report presents the findings of an 18-month evaluation of three pilot initiatives financed by the Scottish Executive Education Department ( SEED) and implemented in three local authorities: North Lanarkshire, East Ayrshire and Glasgow City.

2. The pilots aimed to improve the transition between primary and secondary school, and were introduced in response to anecdotal and research evidence that suggests there is a dip in pupil attainment at the point of transition. Although the pilots were designed primarily to support teaching and curricular transition (with a particular focus on literacy and/or numeracy), they also recognised the social and pastoral issues involved, and adopted teaching and learning strategies suited to the needs of pupils.

Overview of the pilots

3. The pilots in North Lanarkshire and East Ayrshire began in August 2004. The third pilot, the ENABLE project at Eastbank Academy in Glasgow, had been in operation for two years prior to the other pilots. All three pilots aimed to improve the transition from primary to secondary school; however, each had a different curricular focus, and differing methods were introduced in each area, as summarised below:

Aims and objectives of the evaluation

4. The overall aim of the research was to conduct a process and outcome evaluation of the three pilot projects, to assess different approaches to improving transitions between primary and secondary and to suggest where different approaches would be most likely to be successful.

5. The process evaluation focused on implementation and operation, examining the factors that contributed to smooth implementation for each pilot, any obstacles encountered and how they were addressed.

6. The outcome evaluation focused on the specified objectives of each pilot individually, and the outcomes that were common to all three pilots. These were the extent to which the pilots led to improved performance in literacy and numeracy, eased the transition from primary to secondary school, and increased pupil motivation, self-esteem and confidence of pupils in making the transition to secondary school.

Research methods

7. The research took place between March 2005 and September 2006, and involved two main tasks: qualitative consultation, and quantitative secondary analysis of existing data relating to attainment in literacy and numeracy.

8. At regular intervals over the duration of the pilots, in-depth interviews were conducted with key pilot staff, Head Teachers, promoted staff, and other teaching and non-teaching staff at each of the schools involved. Primary teachers were also interviewed in at least two primary schools per secondary school in the literacy and numeracy pilots. (As ENABLE only operates at Eastbank Academy, no research took place in primary schools in the area).

9. Qualitative consultation was also carried out with pupils. In the literacy and numeracy pilots, three cohorts of pupils were consulted at various stages of the transition from P7 to S2. Focus groups explored the pupils' feelings about and experiences of the transition to S1, and their views about working with key pilot staff. At Eastbank Academy, four pupil cohorts were consulted during the evaluation. Focus groups were conducted with S1, S2 and S3 pupils, to explore their experiences of ENABLE and the transition back into mainstream S3 classes.

10. Quantitative analysis of education authority collated attainment data was used to estimate the impact of the pilots on performance in literacy and numeracy. ¹ Additional data were analysed for the ENABLE project, including individual pupil records for all ENABLE pupils, and Standard Grade results for the first ENABLE intake.

Implementation of the pilots

11. In North Lanarkshire, the experience of implementation varied considerably between the three learning partnerships involved.

12. In the first two learning partnerships, the implementation of the literacy pilot went smoothly. The Literacy Development Officers ( LDOs) at these schools were previously Assistant Principal Teachers ( APTs) of English, and had status and credibility within their departments. For the most part, communication of the aims of the pilot and how it would operate were effective in these schools.

13. Some obstacles to smooth implementation were encountered at one of these schools; for example, a lack of available space meant the LDO had no classroom. There was also some disruption to teaching at the start of the pilot, as there were problems finding a suitable replacement to take over the LDO's class teaching.

14. In the third learning partnership, there were particular problems in implementing the literacy pilot which led to its termination after one year. These problems centred around dissatisfaction among the English teachers about the appointment of the LDO, highlighting the importance of having credible and experienced staff in place.

15. In East Ayrshire, teachers had substantial doubts about the numeracy pilot. Primary teachers were concerned about working in the secondary sector or having secondary Maths specialists in their class, while secondary teachers were concerned that the primary teachers would think they were there to analyse or criticise their methods. These doubts were eventually overcome through close cross-sector liaison, but they represented a barrier to smooth implementation at the outset of the pilot.

16. Staffing issues also caused some difficulty in East Ayrshire: for example, the long-term illness of a key member of staff in one learning partnership soon after the pilot's inception led to a cessation of visits to the primary schools, slowing down the process of building relationships and gaining trust. This learning partnership had further problems with staffing which affected its operation throughout the two-year pilot.

17. The commitment and involvement of senior management was also important to the implementation process, which went most smoothly when the Head Teacher and Principal Teacher were actively involved in developing the initiative, ensuring that there was consensus from the outset about the aims of the pilot and the methods that would be used to accomplish them.

18. However, teachers across East Ayrshire generally felt that there was insufficient information and ineffective communication at the start of the project: some primary teachers said they did not receive all the necessary information relating to the pilot. Communication was recognised as an area that needed to be improved, and this aspect of the project was successfully developed over the course of the pilot.

19. A key factor in the implementation of the ENABLE project at Eastbank Academy was the careful selection of staff to be involved. Primary specialists were chosen who were skilled in encouraging the least able children. The project also required secondary teachers who would be able to engage with pupils of low ability, and who were interested in adapting their curriculum for the ENABLE classes.

20. No problems were reported in relation to primary teachers working in the secondary setting. Teachers at Eastbank therefore felt that ENABLE is not difficult to implement, as all education authorities have primary teachers available, and most have schools grouped into clusters, allowing resources to be reallocated between schools.

21. Pupils were placed carefully in ENABLE classes, based on information from their primary schools and the results of a range of ability tests. ENABLE pupils had lower ability, a slower pace of learning, and required more focused support. Only pupils who did not have serious behavioural difficulties were chosen. Eastbank experienced some problems obtaining up to date information from primary schools, where National Assessments tend to be used to confirm judgements later in the year. However, teachers generally felt that the selection system worked well, and the right children were placed in the ENABLE classes.

Operation of the pilots

22. In all three local authority areas, the research indicated that the pilots took time to become embedded within each school; changes were made during the course of the projects, as staff learned from their experiences and settled into their new roles.

23. The work of the LDOs in North Lanarkshire also developed differently based on the needs of individual schools. There were no firm guidelines to determine how the pilots should operate: teaching staff emphasised that the pilot was therefore 'custom built', and saw the flexibility of the LDO role as one of the most positive aspects of the scheme.

24. The LDOs contributed to their English departments in a variety of ways, including team teaching, extracting pupils from classes, introducing new teaching and learning approaches (such as cooperative learning, reciprocal reading and formative assessment), conducting research and developing teaching resources, implementing personal reading schemes and involvement in National Assessment testing. They also introduced a range of voluntary activities such as book clubs. The LDOs' work in the primary schools began with observation and 'fact-finding', and then developed in the second year of the pilot as they became more involved in class and small group teaching. They specifically focused on topics such as Writer's Craft and introduced reciprocal reading techniques to primary classes.

25. In East Ayrshire, too, the pilots had the flexibility to develop in response to local needs. Here, there was a greater emphasis on inter-sector working than in North Lanarkshire.

26. Firstly, secondary Maths specialists were involved in primary classes. They said they gained from watching how primary teachers interacted with their pupils, and applied these teaching methods to S1 and S2. They also discovered that there was a discontinuity in the use of Maths language between the primary and secondary schools, and that primary teachers did not always use the most useful methods for teaching certain topics, such as equations and long division. The increased understanding of cross-sector differences led to mutual learning in both school clusters.

27. Secondly, primary teachers were involved in secondary Maths classes. However, this aspect of the pilot differed between the two learning partnerships. In one school, two primary teachers taught in secondary classes on a regular basis over the two years of the pilot, eventually taking full responsibility for their own classes there. In the other school, the focus was more on observation and cooperative teaching. Having two teachers in the class facilitated practical work, and the sharing of teaching methods.

28. Staffing problems affected the operation of the numeracy pilot in both learning partnerships. In one school, this meant that only two primary teachers were able to visit for most of the second year of the pilot. Some teachers also expressed concerns that the role of these primary teachers when visiting secondary classes was not clearly defined. In the other school, there were concerns about the suitability of some appointments, which led to several changes in key project staff throughout the pilot.

29. As well as cross-sector working, a significant aspect of the numeracy pilot was the development of a new Programme of Study for levels D and E Maths in the pilot schools. This aimed to develop consistent practice in Maths teaching, in order to ensure that all pupils would reach S1 having experienced a common curriculum.

30. ENABLE, which established separate classes for the least able and most vulnerable pupils in S1 and S2, used primary trained staff to teach basic literacy and numeracy skills before pupils returned to mainstream classes in S3. There was a more focused and concentrated input for the pupils, both academically and pastorally. ENABLE pupils spent two hours a day with their ENABLE teacher, and other secondary teachers took ENABLE classes for subjects such as Social Studies, PSE, Technical, PE and Science.

31. Efforts were made to ensure that there was no stigma attached to being in an ENABLE class. Teachers felt that this had been successful, citing examples of pupils who they believed were unaware that the classes were based on academic ability. However, it could arguably be more hurtful for children to find this out from their peers than from their teachers, so there are perhaps implications here in terms of the school's communication of what exactly ENABLE is.

32. Some changes were made to the post- ENABLE S3 courses in recognition of the problems encountered by ex- ENABLE pupils accessing mainstream courses. The continued support available to them from their ENABLE teachers was also formalised.

33. Parental involvement was difficult to achieve, but new ways of engaging parents were developed over the course of the project.

Impacts of the pilots

34. All three pilots were reported to have had a positive impact on the schools involved. Importantly, teachers in both sectors in North Lanarkshire and East Ayrshire reported that they had learned new teaching techniques as a result of working together.

35. Primary teachers in both of these pilots reported that working with secondary teachers had increased their pupils' motivation. P7 pupils enjoyed working with the secondary staff and said they had learned new skills in Reading and Writing or Maths. Data analysis showed that in North Lanarkshire, P7 Reading attainment improved overall during the pilot, although no consistent pattern of change occurred in Writing attainment. Although teachers in East Ayrshire were able to give examples of specific groups of pupils making 'huge jumps' in Maths attainment, there was no evidence of a measurable overall improvement in Maths attainment among P7 pupils at the primary schools involved. Teachers in both pilot areas suggested that the real impact on pupil attainment was more likely to be seen when P7 pupils made the transition to S1.

36. At the secondary schools in North Lanarkshire, teachers emphasised the positive effects the LDOs had on their English departments, particularly in terms of offering support and training to colleagues, helping to monitor and administer National Assessments, and introducing new teaching and learning methods to English classes in S1 and S2. These changes in how the curriculum was delivered affected outcomes for pupils.

37. Teachers reported an increase in attainment after the introduction of the literacy pilot, and the quantitative evidence broadly supports these comments. In particular, analysis of data for each pupil cohort showed that one cohort improved their Writing attainment compared to North Lanarkshire as a whole over the two years of the pilot, and Reading attainment improved in both cohorts compared to the authority average during the same two years. Involvement in voluntary activities also encouraged pupils to read more often, and more widely, than they had before.

38. In East Ayrshire, teaching methods in S1 and S2 were influenced by the cross-sector work with primary colleagues, including a focus on more interactive methods and Maths games. These changes, and an increased focus on the use of technology such as Promethean Boards, led to a reported increase in pupil motivation. The numeracy pilot also led to improved pupil/teacher ratios at both secondary schools, with smaller class sizes particularly benefiting the least able pupils.

39. Teachers believed the numeracy pilot had improved pupil attainment, but also stressed that the impact on attainment was likely to be seen in the academic year 2006/07, when the new Programme of Study had been fully implemented and the new S1 intake would have been working on the programme in P7. Quantitative analysis showed that Maths attainment had declined in one school, and improved in S1 (but not in S2) in the other. This was partly attributed to the particular cohort of pupils involved, and the decision to delay testing while the new Programme of Study was implemented. Cohort analysis showed that while one pupil cohort's performance improved compared to the East Ayrshire average over the course of the pilot, the other cohort's Maths attainment declined in relation to the education authority as a whole.

40. Cross-sector work in the literacy and numeracy pilots was reported to have a positive impact on pupils' experiences of transition from primary to secondary school, both socially and academically. In both North Lanarkshire and East Ayrshire, pupils felt reassured knowing a teacher at secondary school, and their positive experiences with secondary teachers during P7 had dispelled any fears they had had about the standard and quantity of work in S1.

41. Importantly, the cross-sector transfer of information about pupils improved as a result of the literacy and numeracy pilots. In North Lanarkshire, the LDOs' primary liaison provided information to secondary staff about individual pupils and the P7 curriculum in general. In East Ayrshire, improved cross-sector liaison, and particularly the introduction of the new Programme of Study, led to better transfer of information about pupils, allowing more accurate setting in Maths from the start of S1. This was viewed positively by both pupils and teachers.

42. ENABLE was reported to be very beneficial for pupils at Eastbank Academy, particularly because of the fact that it provided a secure environment for the child. Smaller class sizes, and the level of social and academic support available, had a favourable effect on pupils' experiences of the transition to S1. Pupils were positive about their experiences of ENABLE, liked their teachers and felt able to go to them for help with any aspect of school life.

43. Teachers generally felt that ENABLE pupils coped well with the transition into mainstream S3 classes. Pupil consultation supported this claim: children felt they had been well prepared academically for S3, and were confident that they could maintain close links with their ENABLE teachers.

44. In terms of attainment, many ENABLE pupils had mastered basic literacy and numeracy skills by the end of S2. Furthermore, analysis of Standard Grade performance in 2005/06 showed that ENABLE pupils had done better than a comparable group who sat exams in 2004. Importantly, ENABLE succeeded in keeping pupils engaged in mainstream education, with the drop-out rate by S4 improving markedly for ENABLE pupils compared to comparable pupils in previous year groups. Teachers also observed an increase in the ENABLE pupils' confidence and personal development. Pupils in mainstream S1 and S2 classes also benefited from the introduction of ENABLE and the resulting reduction in class sizes.

Conclusions and recommendations

45. The three transitions pilots shared the general aim of improving existing experiences of transition, and attainment at secondary school, following the move from P7 to S1. Consultation evidence suggests that all three pilots were largely successful in meeting this aim, and represented an improvement on the existing transitions arrangements in the schools involved. Although the pilots contributed to some minor modifications to these existing arrangements, it is likely that the majority of the observed changes in the transition experience for pupils were a direct result of the pilots.

46. In terms of the pilot models, each project was designed to meet its aims in quite different ways as regards the curricular focus, the pupils targeted, how staff were deployed and the extent of cross-sector working. Feedback from staff and pupils, supported in some respects by attainment evidence, suggests that each model had positive results and none was more or less successful than the others: staff in each area had developed the projects based on local need, and each model appeared to be largely successful in meeting its particular aims.

47. It therefore appeared it was not so much the precise model that affected the smooth running and success of the projects, but other factors and processes. These included issues relating to staffing, the importance of cross-sector liaison and interchange, effective communication, and the ability of the pilots to be flexible and responsive.

48. Broad lessons for the potential future implementation of similar transitions arrangements are as follows:

• Pilot models
• • It would be helpful to make transition projects with dedicated resources more generally available, with the precise model dependent on local needs. Any of the three pilot models can be successful, provided they are implemented with appropriate staff resources and commitment.
• Staffing
• • Head Teachers and key project staff ( e.g. Principal Teachers and literacy/ numeracy specialists) should be committed, enthusiastic and have 'vision';
  • Those with responsibility for the pilots ( e.g. Principal Teachers) should have control or influence over decisions related to staffing;
  • For projects targeting the least able, most vulnerable pupils, teachers should be chosen carefully based on their skills in encouraging pupils and willingness to adapt their curriculum;
  • Having staff in place for the duration of the project is more successful than appointing teachers on short-term contracts, particularly in terms of maintaining cross-sector relationships;
  • Lead literacy/numeracy officers should be experienced, well respected and seen to be knowledgeable and credible;
  • Other staff should be given control over their level of involvement - asking for volunteers will attract those most committed to cross-sector working; and
  • A reliable pool of supply teachers is required, during implementation of the project and throughout its operation, in order to address staffing problems as they occur.
• Cross-sector liaison and interchange
• • Transitions projects can be used to enhance/improve existing transitions arrangements and cross-sector relationships. This can lead to more effective teaching in both sectors, and also provides pupils with some familiarity with secondary work and at least one teacher when they arrive in S1;
  • When secondary teachers are given the lead role in cross-sector liaison they should be sensitive to the feelings, needs and strengths of primary colleagues - it should be an equal partnership;
  • Consultation between primary and secondary staff is required to ensure the correct balance is reached between initial observation and involvement in teaching;
  • A well-specified, purposeful observation period should work both ways, with teachers from each sector observing in the other; this leads to true reciprocity of learning from each others' teaching methods;
  • Organising Continuing Professional Development ( CPD) courses is a good way of sharing new learning and teaching methods across both sectors;
  • Consultation should be undertaken when organising events for primary and secondary staff ( e.g. are colleagues in both sectors able to attend CPD courses at a particular time of year?);
  • For ENABLE-style projects (where work is not conducted within primaries), good working relationships with associated primary schools is important to have in place at the start;
  • A history of collaboration and good working relationships is helpful, but not essential, for the success of cross-sector work; and
  • Initial doubts about cross-sector working can be overcome through effective communication.
• Communication
• • It is important to ensure that the role of the lead officer is clearly defined and well communicated;
  • There should be consensus about the aims and methods of the project, which should also be communicated effectively to all involved, ideally before the project starts;
  • Projects should build in adequate time for discussion and feedback between primary and secondary teachers; and
  • Social events can successfully improve communication.
• Flexibility and responsiveness
• • Lead officers should be given flexibility within the guidelines to develop their roles in response to local needs;
  • Where parental involvement is required (in projects such as ENABLE), teachers should be flexible in their approaches to engagement with parents;
  • Projects targeting the least able pupils in S1 and S2 should also be responsive to these pupils' needs (both social and academic) further up the school; and
  • Such projects should also consider the issue of openness about the nature of special classes very carefully: a wish to eliminate stigma, although positive in itself, may have negative implications in terms of a lack of openness and honesty.
• What works where
• • A model that targets the most vulnerable, least able pupils is particularly well suited to areas of deprivation, schools with a large proportion of children arriving without sufficiently developed skills, or where pupils present with emotional difficulties; and
  • The models including all pupils making the transition can work in any area, providing the key factors above are taken into consideration.