Scottish Government

3.4 Status of projects at start of, and key developments during, the evaluation project

This section presents, for each project, the project descriptor aim (as found in the programme Action Plan), any further statement of objectives in the documents, progress against milestones/timescale and early emerging issues discernible in the project documents.

At the start of the programme, schools participating in school-based projects were requested to complete a baseline audit which addressed issues of planning and practice of assessment and reporting. The audit tool was developed in order to help schools identify areas to be addressed in their project and so include them in their action plans. Some of the schools completed the audit but it was reported that it was not particularly useful as it fell outwith the normal school development cycle.

Project 1: Support for Professional Practice in Formative Assessment

The aim of Project 1 is to 'investigate classroom approaches to formative assessment and produce guidance for schools and teachers about assessment policies and practice to raise attainment'.

Project 1 introduced and took forward strategies for formative assessment as proposed in the 'Black Box' materials (Black and Wiliam, 2001). This was supported through conference input by staff from King's College, London, and some teachers from England who had participated in developments in formative assessment with King's College. Project 1 teachers were encouraged to take part in action research and where possible to set up 'control groups' within their teaching context to see if any differences emerged between groups of pupils who were being introduced to formative assessment strategies and those being taught in the 'normal' way. Teachers were able to share their experiences through the 'recall' days organised by the development officer.

Thirty-five schools were involved in the first phase of the project: 16 primary schools, 14 secondary schools, 3 junior high, one 5-14 school and one special school. A second group of 10 schools became involved in April 2003:2 primary and 8secondary. Schools from all local authorities participated in Project 1. It was noted that the Project 1 schools had, in fact, completed their action plans before the audit tool became available.

It was proposed that the first phase schools would have submitted action plans by June 2002 and case study reports of developments in their schools by March 2003; the second phase group had a proposed date of June 2003 for the submission of their case study reports. Study of the documentation indicated that the majority of the schools had produced action plans as required, though of varying quality, with additional support from the development officer where required. Many were on track to produce the required case study reports, though, where there were delays, deadlines were being treated flexibly, with a small number of the original schools extending to June 2003 to report.

Analysis of the documentary evidence indicated a great deal of enthusiasm and commitment amongst teachers taking part in Project 1. Where there were delays or obstacles, these related to changes in school management, to the roles of the teachers or to external factors such as lack of supply cover and HMIE inspections. Changing classroom practice raised a number of challenges that caused schools to reflect and modify their plans, learning from the issues that arose through review and reflection. At this stage, there was no evidence of links being made to the other projects by the teachers involved and no mention of personal learning plans was noted.

Project 2: Personal Learning Plans

The aim of Project 2 is 'to draw together existing work on Personal Learning Plans ( PLPs) and other existing records and design a single recording framework'.

Personal Learning Plans were a key aspect of developments in the New Community Schools initiative and the schools participating in the AifL project were able to draw on the existing work from that initiative. The outcome of drawing together the work of all the schools was to contribute to 'a single recording framework' which would be transferable on a national basis. This project was most closely linked to Project 3.

Seventeen schools were involved across 6 local authorities: 11 primary schools and 6 secondary schools. Only 3 schools completed the baseline audit; it was noted that most schools felt it was inappropriate as it did not fit within the school development planning cycle.

Schools produced action plans which identified key areas for development relevant to the personal learning planning process: making learning intentions and assessment criteria explicit, pupil self-assessment, identifying own learning needs, setting targets, planning next steps, and dialogue between pupils and teachers. Some of the schools were concentrating on personal learning planning in specific aspects of the curriculum: for example, language and maths; others were taking a whole curriculum approach. Some secondary schools focused on core skills with planning taking place within Personal and Social Education ( PSE).

It was originally proposed that schools would produce case studies by March 2003. Again this timescale was treated flexibly, with schools aiming to produce a PLP framework by that date to be piloted with pupils and a revised PLP to be available by June 2003.

The documentary analysis revealed a number of challenges faced by the schools and also challenges to the overall aim of the project. There was a clear perception that working with personal learning plans was time consuming and led to an increased workload. Participants also had a range of views as to the purpose of PLPs: some held the view that it should address all aspects of learning including test results as well as personal objectives; others focused on specific curricular targets; others viewed them as being appropriate only for personal development and learning.

There was evidence of some clusters of schools working together to have a common approach within the cluster, particularly for P7 to S1 transition. There was also evidence that some schools were recognising the importance of formative assessment practices in relation to PLPs. Some schools were looking for ICT solutions to PLP development.

Project 3: Supporting the Management of PLPs

The aim of Project 3 is 'to produce guidance for schools and teachers on management of the record-keeping process and PLPs, taking into account work on the Progress File and Individual Educational Programmes'.

The schools involved in Project 3 were to develop PLPs with a particular focus on the processes involved: for example, how learning intentions and criteria are made known to the pupils, how teachers engage in dialogue with pupils, how learning targets are agreed, how reporting to parents is carried out.

Fifteen schools from 6 authorities took part in Project 3: 4 secondary, 9 primary (3 incorporating nurseries), one nursery and one special school. Schools were at different stages in the process of developing PLPs - some were just starting and others were in the second or third year of development.

All schools completed the baseline audit and produced action plans. From a review of the audits and action plans it was clear that the main areas identified as being less developed were related to pupil self-awareness, pupil awareness of learning intentions and assessment criteria, individual progress and target setting. Some schools rated themselves highly on relationships with parents; others recognised this as an aspect which needed development and would be addressed as part of the project. As with Project 2, the focus for development varied from one or two aspects of the curriculum to the whole curriculum. Again as with Project 2, the original date of March 2003 for completion of reports was treated flexibly.

The documentary analysis revealed similar challenges to those noted in relation to Project 2. There was particular emphasis on differing departmental responses within secondary schools. References were made in the notes of meetings with schools to the relevance and importance of formative assessment.

Further developments of projects 2 and 3

To seek to address some of the issues emerging in relation to PLPs, a working group involving project development officers and a consultant developed a 'cross cutting skills' model. This was based on the OECD Key Competencies Framework linked to the 5-14 curriculum guidelines, with the aim of providing a 'manageable "spine" of curriculum outcomes for PLPs that all teachers could report against' ( SEED, 2003a, para 24). PLPs were developed around skills at 3 levels: P3-P5, P6-P7 and S1 . These were piloted under the title of "Skills for Living" in 10 of the original AifL PLP project schools from September 2003.

A proposed framework for PLPs was developed and circulated for consultation in September 2004.

Project 4: Gathering and Interpreting Evidence

The aim of Project 4 is 'to produce guidance and exemplification for teachers on the range of assessment evidence to be gathered and retained for each aspect of the curriculum at each stage, and how to evaluate it'.

Schools involved in Project 4 were engaged in identifying opportunities for assessment within normal classroom practice so that assessment is seen as an integral part of the teaching and learning process. Emphasis was on the whole assessment cycle as presented in the 5-14 curriculum guidelines: planning, teaching (including gathering evidence), recording, reporting and evaluating. Developments focused on identifying opportunities for assessment which went beyond pencil and paper tests: for example, observation of children completing tasks, products from class activities, verbal responses. Particular encouragement was given to using digital video as a means of capturing evidence. Establishing clear assessment criteria for each level of the curriculum to assist in the interpretation of the evidence was a key part of the process.

Materials presented at the introductory project seminar highlighted the fact that there are (approximately) 1776 attainment targets across the 5-14 curriculum, including all levels A to F. Clearly, for development purposes, each school had to select specific areas of the curriculum and levels as a focus for the project.

Twenty-six schools across 5 authorities took part in Project 4: one nursery, 17 primary, one junior high, 4 secondary and 3 special needs schools. All schools completed baseline audits and produced action plans. The main areas for development appeared to be clarifying criteria, producing assessment materials and exemplars of work which met the criteria. Some schools emphasised involving pupils more in understanding the criteria, in suggesting suitable evidence and in self-assessment.

Some of the curricular areas mentioned were: reading for information, writing, art, clarifying criteria for ICT and integrating the teaching of it with other subjects, PE and modern languages.

Most schools produced case study reports (or at least draft case studies) within the timescales required. As with other projects, a flexible approach was taken with schools who were not able to do this.

All participants received 'Black Box' literature but at this stage it was not clear that formative practice was a focus of developments. Rather the focus appeared to be on clarifying the criteria and using a wider range of evidence for summative judgements. No reference to PLPs was noted.

Project 5 - Local Moderation (Phase 2)

The aim of Project 5 is 'to investigate ways of "sharing the standard" with other teachers, and to produce local portfolios of examples of assessed work across the various aspects of the curriculum'.

As a second phase development, Project 5 was in its early stages at the beginning of the evaluation project. It was intended that this should build on the work of Project 4, focusing on teachers working together to reach a common understanding of the levels of attainment in different curricular areas and the criteria to be applied in making assessment judgements. Exemplars of work illustrating the criteria were to be developed. Building on the lessons learned from the Phase 1 projects, Project 5 schools were specifically requested to incorporate formative assessment strategies and consider how the criteria they were developing could be used formatively.

Four school clusters from 4 different authorities were identified to participate in Project 5, ie 4 secondaries and a total of 13 associated primaries. This approach was to allow for sharing of standards across schools and across sectors. None of the schools initially involved had been involved in Project 4, although some Project 4 schools subsequently took their developments further into the work of Project 5.

The school clusters produced action plans. Two of the clusters focused on aspects of language development and 2 focused on aspects of environmental studies/science. Although originally planned that Project 5 schools would report by November 2003, they extended beyond this into 2004.

Project 6 - New National Assessments

The aim of Project 6 is 'to create an online "bank" of assessment materials, based on Assessment of Achievement Programme tests and tasks, to replace the current National Tests'.

During the initial stages of the evaluation project, the development work on the new National Assessment Bank was ongoing. The previous bank of tests available to schools had been questioned on grounds of both validity and reliability. The bank was being improved through the incorporation of items used previously as part of the Assessment of Achievement Programme and new items. The principle of the bank is that schools can download assessments for a specific curricular area covered by national testing requirements, at the required level, when they need it. They have no choice as to which precise test they receive. Previously schools had selected tests from a catalogue and ordered them from SQA.

The launch of the new National Assessments had been planned for August 2003; however, due to various delays and issues of suitability of hardware and software in schools, this took place in December 2003.

In September 2003, a consultation on Assessment, Testing and Reporting: 3-14 was initiated ( SEED, 2003b). The findings of this consultation and the Scottish Executive response were not published until November 2004 ( SEED, 2004a and 2004b). During this period many participants expressed uncertainty regarding the future of National Assessments and the Assessment of Achievement Programme.

Project 7 - Assessment of Achievement Programme

The aim of Project 7 is 'to continue and update SEED's existing AAP monitoring programme, improving the monitoring of standards, progression between stages and change over time, and incorporating professional development for teachers'.

The Assessment of Achievement Programme was established in the mid-1980s with the aim of monitoring achievement in key areas of the curriculum at significant stages in the school career of pupils. Initially, the focus was on English language, mathematics and science at the Primary 4, Primary 7 and Secondary 2 stages of schooling. Each subject area was assessed on a 3-year cycle by means of a survey approach involving both assessment materials for pupils and questionnaires for schools. National representative samples of pupils in schools are drawn to participate in the AAP so it was anticipated that only a sub-sample of the AifL schools would have experience of it.

This pattern of monitoring was modified in 2002 when the surveys were moved to a 4-yearly cycle and the stages of pupils involved were changed to Primary 3, Primary 5, Primary 7 and Secondary 2. While the emphasis remained on English language, mathematics and science, the fourth year of the cycle could be used for other subject areas such as social studies (assessed in 2002). In addition, performance on core skills within the context of the subject area was included in the surveys. The role of the AAP and any future developments were addressed in the consultation exercise on Assessment, Testing and Reporting: 3-14.

Some of the AifL project schools participated in the AAP surveys that took place during 2002 to 2004, and some teachers were involved as field officers. This evaluation did not seek evidence directly from schools on their experiences of the AAP but rather has drawn on the internal evaluation undertaken by the AAP itself.

Project 8 - ICT Support for Assessment (Phase 2)

The aim of Project 8 is 'to investigate ways in which ICT can support assessment, record keeping and reporting, and ensure that developments are consistent with ScotXed and NGfL agendas'.

Project 8 was different from the other school-based projects in that, rather than asking schools specifically to develop new approaches, particularly to assessment, recording and reporting processes, the purpose was to investigate what software was already in use, how effective it was, gain schools' views on potential developments and produce a framework document describing software specifically for the purposes of supporting personal learning plans. This document could then be used in discussion with software providers. Views of local authority representatives were also sought as part of this process.

It was planned that one primary and one secondary school be involved from 16 authorities. In the end, a total of 16 schools (12 primary and 4 secondary) from 15 authorities took part in the exercise. The schools were different from those already involved in Projects 2 and 3. It appears that in at least one authority Project 2 and Project 8 schools worked together to ensure consistency.

Initially an audit was carried out of schools and authorities, followed up by meetings with each school at which an authority representative was present. Some schools already had in place electronic record keeping and reporting systems. Some of the early emerging issues related to ensuring that various Management Information Systems ( e.g. school and authority, or even different systems within the school) were compatible in order to avoid having to enter data more than once. Specifically in relation to PLPs, the main challenges, as with Projects 2 and 3, included being clear on the purposes of a PLP and what targets were to be included (curriculum and/or personal). Software was required that enabled a wide range of targets to be effectively incorporated into an electronic system and allowed teacher, pupil and parent access as appropriate. A further challenge was keeping the focus on the process of developing and negotiating the PLP and keeping the IT system as a way of managing the information and not determining it. Some concern was expressed about the lack of clear links being established between schools involved in Projects 2, 3 and 8.

The framework document for ICT software was produced in March 2003.

Project 9 - Reporting to Parents and other Teachers (Phase 2)

The aim of Project 9 is 'to link reporting to the Personal Learning Plan framework and propose a common format for reports'.

The main developments in the project began early in 2003. The original aim was to take forward work based on a Personal Learning Plan framework (or single recording framework) which was proposed as an outcome of Project 2. As noted above, this had not been produced by the start of Project 9. The focus therefore shifted to reporting to parents in a broad sense, including written and face-to-face communications as well as formal reporting systems.

Twenty-one schools from 7 authorities were involved in Project 9: 5 secondary, one junior high, 11 primary and 2 nursery. All schools completed a sub-section of the baseline audit which related specifically to working with parents and a questionnaire was also used to gain parents' views on the effectiveness of the school reporting procedures. The outcomes of these two exercises were used to inform school action plans.

Action plans indicated a wide range of developments, including electronic reporting, trialling of new written reports, using email for communication with parents, involving parents in developing new policy, improving parents' evenings, organising more open events for parents, involving parents in children's learning, and consulting more with parents.

In addition to this project that was targeted at communicating with parents, the programme organised a series of 'Open Space' events for parents, informing them of the wider aims of the initiatives and seeking their views on developments.

Although the original date for case study reports was October 2003, the work extended into 2004, with case studies being produced early in 2004.

Project 10 - Meeting the Needs of Pupils with Special Educational Needs (Additional Support Needs) (Phase 2)

The aim of Project 10 is 'to ensure that all the projects in the programme are inclusive of pupils with the whole range of educational and social needs'.

Schools involved in Project 10 were invited to consider the developments occurring in the other projects and to use them within the context of young people with additional support needs. This category included those with learning difficulties and physical disabilities, those with English as second language, travellers, refugees and asylum seekers, and those learning through Gaelic-medium education.

Sixteen schools from 13 authorities participated in Project 10: 4 secondary, 9 primary and 3 special schools. Additionally an authority-wide service on English as an Alternative Language was included.

All schools carried out baseline audits and action plans. For many, the focus was on encouraging independent learning and improving the planning and management of assessment. The 16 schools chose to develop aspects of AifL as follows:

The main developments began early in 2003, though there was some delay with the Gaelic-medium developments. Schools continued developments into the following session with the production of reports in late 2003/early 2004.

3.5 Summary of key points

This chapter has provided an overview of the development and early progress of the 10 Assessment is for Learning projects.

• From the inception of the programme at the end of 2001, it was proposed that the developments be taken forward based on 10 separate but linked projects. Originally the main focus was on the development of Personal Learning Plans as a means of supporting a unified system of recording and reporting. Although multiple projects were devised to make developments 'manageable', it was considered important that they all be seen as contributing to a coherent national system of assessment.
• All local authorities were involved in Project 1 and at least one other project. Over 170 schools and more than 500 teachers took part in the pilot developments. Each project had an associated project development officer and representative from higher education.
• The school responses to completing a baseline audit varied, with some schools indicating that it was inappropriate and did not fit within the school development planning cycle.
• Milestones were set for schools to produce case study reports and examples of products. Some schools found it difficult to work within the time constraints; however, as growth and the developmental process were fundamental to the programme, the timescales were treated flexibly. Because of this the preparation of outputs for wider use took longer than originally anticipated.
• Those involved in PLP projects varied in their understanding of the purposes of PLPs and the implication for developing ICT support. A common reporting framework was not produced as planned, with resultant consequences for the focus of Project 9, which was intended to build on that framework. Further development work on PLPs was undertaken, with a proposed framework being circulated for consultation in September 2004.
• From the documentary analysis conducted between February and June 2003, it appeared that involvement in Projects 1, 4 and 9 was focused exclusively on project-specific developments (viz: no mention of PLPs in relation to Project 1 or formative assessment in Project 4). On the other hand, some participants in Projects 2, 3 and 8 were recognising links with formative assessment.