Scottish Government

Insight 2
The Impact of ICT Initiatives in Scottish Schools

This document is also available in pdf format (192k)

Welcome to Insight

Insight is a publication of the Research, Economic and Corporate Strategy (RECS) Unit, one of four units in the Information, Analysis and Communication Division, which is responsible for providing analytical services within the Scottish Executive Education Department (SEED). RECS is a multidisciplinary unit (consisting of researchers and economists) which undertakes and funds economic analysis and social research in the fields of: school education; children, young people and social work; architecture; and tourism, culture and sport.

The Scottish Executive is committed to the use of sound evidence in the development of policy and practice as well as in the evaluation of policy and its implementation. We therefore want to disseminate the results of research that SEED has undertaken and funded, in a manner that is accessible, interesting and attractive.

Insight aims to present the essence of research projects in a format that will be useful and informative for practitioners, policy makers, parents, academics, and anyone else who has an interest in economic and social research in these areas.

The views expressed in this Insight are those of the author(s) and do not necessarily reflect those of the Scottish Executive or any other organisation(s) by whom the author(s) is or are employed.

Insight may be photocopied for use within your own institution.

A limited number of additional copies can be obtained from the Dissemination Officer, Research, Economic and Corporate Strategy Unit, Scottish Executive Education Department, Victoria Quay, Edinburgh EH6 6QQ (telephone 0131-244 0092). Copies of Insight and our other publications can also be downloaded from our website: www.scotland.gov.uk/insight/

The Impact of Information and Communication Technology Initiatives in Scottish Schools

Rae Condie (University of Strathclyde), Mary Simpson (University of Edinburgh), Fran Payne (University of Aberdeen), Donald Gray (University of Strathclyde)

Introduction

The Impact of Information and Communication Technology Initiatives in Scottish Schools project was funded by the Scottish Executive Education Department to provide feedback on the effect of the considerable investment made in Information and Communication Technology (ICT) access and provision in schools. More specifically, the project attempted to determine the levels of performance of pupils at Primary 7 (P7) and Secondary 4 (S4) on a range of ICT skills and abilities, their experiences of, and attitudes to, ICT and, in addition, the views of their teachers. As a result, two national surveys of pupils and their teachers were undertaken, the first in 1998-99 and the second in 2000-01.

In 2000, Interchange 63 was published by the Scottish Executive, setting out the origins of the project and its aims as well as the main findings from the first phase (1998-99). This Insight reports on the findings from the second phase in 2000-01, comments on the impact to date and makes recommendations for future development in the area of ICT.

The Study

In both phases the study focused on pupils in P7 and S4 and their teachers. In 2000-01, 1332 P7 pupils in 72 primary schools and 1409 S4 pupils in 80 secondary schools participated in the project. The two year interval between phases meant that it was possible to track some of the P7 pupils from 1998-99 and assess them again in 2000-01 when they were in Secondary 2 (S2). Within each phase three forms of data collection were used. The first consisted of questionnaires to pupils, teachers and school ICT co-ordinators. These were distributed at the same time as the series of written booklets designed to assess pupils' knowledge and understanding of ICT and its uses. The third form of data gathering involved visiting a sub-sample of schools and assessing pupils' skills and understanding as they worked through a series of practical tasks, using a computer. In 2000-01, the written element of the study (questionnaires and written booklets) took place during February to April 2001, followed by the practical component in May and June 2001.

Pupils' ICT Knowledge and Skills

The assessment framework (written booklets and practical tasks) consisted of three categories of performance: Knowledge and Understanding, Personal Appropriation and Practical Competence. Within each category, four sub-categories were identified: Hardware, Software, Communications and Networks and Uses and Impact of ICT. The majority of tasks were concerned with computers, although a few included questions on other forms of ICT.

Knowledge and Understanding

In Knowledge and Understanding of Hardware, tasks typically asked pupils to name parts of computers and their peripherals, to identify desktop icons and to know about viruses, icons, networks and a range of disks. Software tasks focused on using software for specific tasks, including graphics, databases, spreadsheets, CD-ROMs, and recognising the more common icons on software toolbars. The example No Smoking required pupils to recognise various aspects of a word processing package.

Example 1: No Smoking

A total of seven marks was available for this task. On average, P7 pupils scored 5.2 marks and S4 pupils scored 5.4.

At P7, attainment was greatest in Knowledge and Understanding of Hardware and Software, with scores of 63% and 70% respectively for the sample. The S4 papers contained more tasks, and more demanding ones, although the majority of tasks were common to both stages. At S4, pupil performance levels were also better on Hardware (71%) and Software (72%).

Within Knowledge and Understanding, tasks on Communications and Networks expected knowledge of e-mail, video-conferencing, internet search engines and associated terminology such as URL and links. Uses and Impact of ICT tasks focused on the ways in which technology is used in society, including the kinds of tasks that workers in ICT might undertake. On Communications and Networks, P7 pupils scored 53% of the available marks and 70% on Uses and Impact of ICT ; at S4, the figures were 61% and 58% respectively.

Personal Appropriation

Personal appropriation, the ability to identify and use appropriate strategies to complete a task, was also assessed through the written papers. At P7, the success rates were 49% for Hardware, 33% for Software and 35% for Communications and Networks. At S4, the corresponding figures were 58%, 61% and 45%.

Example 2: For Sale

In this task, For Sale, a total of four marks was available. At P7, the average score was 1.6 and at S4 it was 2.4.

Practzical Competence

Incompatibility of equipment, platforms and applications resulted in a smaller data gathering exercise for this aspect than was originally anticipated: 198 P7 pupils and 195 S4 pupils tackled the practical assessment tasks. Assessments were made on a one-to-one basis and the instructions were presented in written form and read out by the assessor.

At both stages, pupil performance was most secure on word processing. Basic procedures for spreadsheets were fairly secure. Competence levels on e-mail and using the internet were difficult to establish accurately due to a lack of available facilities in many schools, but where pupils did have access, they accomplished most of the tasks satisfactorily.

Comparing Primary 7 and Secondary 4

Thirty-six tasks were common to both stages. The pattern of S4 superiority observed in the first phase was repeated in the second phase, with the older pupils achieving higher success rates on 28 tasks while P7 were better on eight tasks. There was no clear pattern to the differences in terms of the kinds of tasks where one group was more successful.

Comparing 1999 and 2001 findings

There was no evidence of improved performance at P7 although there did appear to be some evidence of improvement at S4, primarily due to greater understanding of software applications and procedures.

The Perceptions of School Staff and Pupils

Questionnaires were issued to the pupils involved in the survey, their teachers and the ICT co-ordinators within their schools. The co-ordinators have the main responsibility for planning the implementation of the various initiatives within schools.

The ICT Co-ordinators

Remits

Co-ordinators in both sectors indicated that their remit was escalating in scope and volume. New elements had been added in response to local authority and government initiatives, and the publication of guidelines for ICT had had consequences for their workloads. Dealing with technical problems occupied a great deal of their time.

Sources of advice and information

Both primary and secondary co-ordinators reported that their main sources of support were local authority staff (68% primary; 59% secondary), and colleagues in their own school (41% primary; 59% secondary). The most valued sources of information were mainly within the mainstream educational system - their own schools, the local authority and, for primary teachers, Learning and Teaching Scotland. Although there was a reported increase in the use of the internet, of teachers' networks and the Scottish Virtual Teachers Centre, as in the previous phase these were consulted by relatively few (<20%).

The development of policy and planning

Both sectors indicated that a lack of teacher confidence and/or expertise was hindering the implementation of the 5-14 Guidelines for ICT. Other obstacles included the time to teach, finance for hardware and other resources, and the monitoring of progress. The majority of schools in both sectors had written policies for ICT; approximately half of the secondary schools based this on Using the Superhighways: ICT and Development Planning.

In general, communications about ICT within schools appeared to have become more effective since 1999, primarily through in-school staff development activities and informal contact. More use was being made of e-mail in secondary schools for this purpose.

Progress with priorities

Primary co-ordinators indicated that their priority had been resourcing for ICT, e.g. installing hardware, and the data showed no clear pattern of development in learning and teaching in primary schools. The most securely established practice remained the use of CD-ROM resources in primary classrooms (70%), and this was one area where pupils did appear to have gained in expertise since the first phase. There was little or no change in administrative usage, e.g. enhancing the quality of pupil reports (48%) and centralising the assessment and test results of pupils (25%).

In secondary schools, advances in a range of learning and teaching strategies were reported. These included supporting pupils with learning difficulties (from 56% in 1999 to 74% in 2001) and securing opportunities for internet access (29 to 74%). Administratively, the quality of reports had been enhanced (48 to 60%) and assessment and testing records had been centralised by a greater proportion of schools (8 to 30%).

Using ICT to communicate with others outside the school - parents, professional colleagues, pupil peer groups or the local community - remained the least well advanced in both sectors.

Increased benefits

In both sectors, co-ordinators identified a number of benefits from ICT including aspects of management activity, increased pupil motivation, improved presentation and increased ICT skills amongst pupils and teachers.

The Primary Teachers

Of the primary teachers who responded, 81% had a computer at home and 77% of these were linked to the internet. The computer was used most frequently by our overall sample of teachers for word processing for school (78%), internet searches (65%), using e-mailing for personal (64%) and professional (37%) purposes, and CD-ROMs (62%).

Working with pupils

In school, 93% of primary teachers had at least one computer in their classroom regularly in use by pupils and 54% also used a school ICT base, suite or other shared computer facilities.

The only significant change in pupil use of school computers since 1999 was an increased use of the internet to search for information (from 3% in 1999 to 27% in 2001). Teachers reported that their pupils showed a higher level of independence from the teacher when working on the computer (63%) while 45% said that they had evidence that pupil attainment had improved and 94% that ICT had contributed in significant ways to the learning of their pupils.

More teachers were using ICT to create curriculum materials (43%) or download materials from the internet (26%). There was little change in the proportion using ICT to assist high attaining pupils or for providing differentiated activities, although the proportion giving specialist assistance to pupils with learning disabilities increased from 34 to 47%.

Administrative and management tasks

In terms of management activity, 41% were using ICT to produce pupil records for parents, with around 17% using it for managing other types of assessment data. However, significant percentages indicated that producing records for parents (22%), making profiles of pupil progress (29%), recording and analysing test results (46%) and setting up systems that would allow pupils to use ICT to record and monitor their own attainments (54%) were unlikely to be developed in the foreseeable future.

Fifty-five percent of primary teachers thought that the classroom of the future would be very different from the present while 40% considered that the importance of ICT had been greatly exaggerated.

The Secondary Teachers

Of the secondary teachers, 88% had a computer at home and 75% of these were linked to the internet. The computer was used most frequently by the secondary teachers for the same purposes as their primary colleagues.

Working with pupils

In school, 62% had at least one computer in their classroom regularly in use by pupils while 48% also used a school ICT base, suite, lab or other shared computer facilities. More teachers were using ICT to create curriculum materials (46%) or to download materials from the internet (25%). Increased percentages were using ICT to assist high attaining pupils (40%) and for providing differentiated activities (44%); the proportion giving specialist assistance to pupils with learning disabilities had increased from 15% to 24%.

Little change was reported in pupil use of ICT except for using the internet to search for information (from 8% in 1999 to 25% in 2001) and using CD-ROMs (19 to 28%). However, 66% thought that pupils showed a higher level of independence from the teacher when working on the computer while 41% said that they had evidence of improved attainment and 84% agreed that ICT had contributed in significant ways to pupils' learning.

Administration and management tasks

In terms of administration and management, 28% were using ICT for pupil records for parents and 20% for other types of assessment data management. Significant percentages indicated that producing records for parents (27%), making profiles of pupil progress (29%), recording and analysing test results (30%) and setting up systems that would allow pupils to use ICT to record and monitor their own attainments (73%), were unlikely in the foreseeable future.

Just over half (55%) envisaged that the classroom of the future would be very different from the present while 49% thought that the importance of ICT had been greatly exaggerated.

The Pupils

The responses from 2001 were remarkably similar to those of 1999 and showed little variation with age. Where there was an age difference, the S2 figures fell between those for P7 and S4. Just over half of the pupils in each group indicated that they knew 'enough (about computers) to get by', while around one third indicated they knew 'a lot or were real experts'. 51% (P7) and 65% (S4) regarded learning to use computers as 'very important'; almost all the remainder regarded it as 'quite important'.

In school

At school, 55% of S4 pupils used a computer three or more times a week, while most of the younger pupils used it once or twice a week or less. The range of typical uses in school had changed little, e.g. word processing, drawing and using CD-ROMs. The only changes were increases in the use of the internet and e-mail.

More than half the S4 pupils studying subjects where computers were used regularly (e.g. Business Management, Graphic Communication or Accounting) had used them in the previous week, while 25% of pupils had used them in English and/or Art and Design. The figures for computer use in all other subjects were very low, showing little change since 1999.

The majority of pupils in each age group indicated that they liked using computers because the work was more interesting (86% P7 and 78% S4) and the product neater (75% P7 and 90% S4). They liked being able to find information not accessible through books (76% P7 and 68% S4) and that they got to use their own ideas and imagination (58% P7 and 52% S4). Fewer than 10% of pupils in all three age groups thought that the computer-based work in school was too hard while 39% at P7, 33% at S2 and 17% at S4 thought it too easy.

At home

Across the stages, 80% at P7 and 83% at S4 indicated that they had access to a computer outside school and half used it on three or more days a week. Of the total pupil sample, 73% at P7 and 77% at S4 had a computer in their own home and for 57% of P7 and 60% of S4 pupils this was linked to the internet. Some had greater access, with 26% at P7 and 33% at S4 having a computer in their own room.

There were a number of differences in the usage at home and school, with the most significant (educationally) being the lack of pressure of time at home (78% P7 and 87% S4 of those with access indicated that they could spend as long as they liked doing their work on the computer at home, while at school the corresponding figures were 6% and 12%). In addition, they were able to be more adventurous at home with 89% of P7 and 92% of S4 pupils indicating that they experimented with different applications and activities, while at school the corresponding figures were 42% and 32%.

Summary

This study was set up to provide baseline data at the introduction of a series of ICT initiatives in Scottish schools and to consider, two years later, the extent of the impact of these initiatives. This has now been completed and the findings are complex. The questionnaires have yielded evidence that the use of ICT both in and out of school is increasing, with a substantial majority of pupils and teachers having access to a range of technologies, particularly computers, most of which are connected to the internet. The uses to which they were put varied somewhat, with teachers tending to use word processing and other 'task-oriented' programmes while pupils were more creative and 'experimented' with the technology.

Attitudes to ICT were, in the main, positive although a significant proportion of teachers thought that the benefits to the classroom had been greatly exaggerated. Technical problems and inconsistencies across platforms and applications were sources of frustration for both teachers and pupils. An additional frustration for many teachers was the extent to which staff development in ICT had not kept pace with technological developments.

Pupils' performance levels were little changed across the two surveys, with considerable consistency across stages and categories of assessment. Increases were noted in those aspects which supported 'research,' i.e. using the internet and CD-ROMs. The extent to which teachers considered that some pupils were more advanced than most of their teachers, and indicated concern about such a 'skills gap,' decreased over the period of the study but remained a significant issue. It may be that the time pupils spend 'playing' with the different uses and applications is, in part, responsible for this perceived gap. A further issue of concern is the extent to which those pupils who do not have access to computers at home may be disadvantaged in comparison with those who do.

One of the tensions in the implementation of ICT initiatives lies in the debate as to whether ICT should be considered a subject in its own right or as a means to learning within other subjects and contexts. The development of ICT suites or labs has tended to support the former approach, with timetabled sessions where the focus is on ICT skill acquisition. The measures of achievement used within this study focused on ICT knowledge and skills per se.

An alternative approach considers that the potential of ICT lies in the ways in which it can make a range of information accessible to pupils studying, for example, science, art, the social subjects or modern languages, and facilitate their learning. The Assessment of Achievement Programme, funded by SEED, is currently piloting an alternative approach to the assessment of ICT knowledge and skills. Tasks have been developed to provide evidence of ICT competence within the context of a national survey of performance in Social Subjects 5-14. Assessing skills and understanding within relevant and interesting contexts should help to foster the view of ICT as a source of support in learning and teaching more generally. The findings from the survey should contribute considerably to this debate.

The full report, The Impact of ICT Initiatives in Scottish Schools: Final Report, is available from Dr Rae Condie, Faculty of Education, University of Strathclyde (price 7.00).

The Insight Series

1. Classroom Assistants: Key Issues from the National Evaluation
2. The Impact of ICT Initiatives in Scottish Schools

If you have views on Insight or wish to find out more about SEED's research and economics programme, please contact the Research, Economic and Corporate Strategy Unit, Scottish Executive Education Department, Victoria Quay, Edinburgh EH6 6QQ.

ISSN 1478-6788 (Print)
ISSN 1478-6796 (Online)