Scottish Government

Section 5 Numeracy, Maths Literacy and ICT

Assessing and reporting Numeracy attainment

5.1 Almost 13,000 pupils took part in the assessment of numeracy in this survey, comprising just under half of the pupils in the survey sample at each stage. In primary schools, this was around 3,300 at each stage. At S2 almost 2,800 pupils took part.

5.2 Numeracy attainment was assessed principally through the administration of 80-90 test items at each of Levels A to F. The items were drawn from the mathematics section of the 5-14 national bank of assessments. The selection included items from most strands of the attainment outcome Number, Money and Measurement (excluding algebra) and from the strand interpreting information from Information Handling. Problem-solving was not included in the assessment of Numeracy.

5.3 The numeracy items were distributed among 10 different test booklets at each stage, each containing items from three consecutive levels and spread over the various strands. P3 booklets contained items at Levels A, B and C, at P5 the levels were B, C and D, at P7 levels were C, D and E, and at S2 levels were D, E and F. Items were randomly ordered within booklets, and booklets came in two versions, one version simply reversing the item order of the other.

5.4 In addition to the main numeracy assessment, a small number of 'mathematical literacy' tasks were also administered. There were eighteen tasks in total across Levels A to E, each task presenting pupils with a source leaflet and 12 related 'numeracy' based questions. The sources were representative of the sorts of information that pupils at the different stages might reasonably be expected to access and use in their everyday lives. Pupils were required to use a combination of reading and numeracy skills to find and process information presented as a mixture of text, tables and other graphical formats. The tasks were packaged two to a booklet, at adjacent levels: Levels A and B for P3, Levels B and C for P5, Levels C and D for P7 and Levels D and E for S2.

5.5 Individual pupils were randomly allocated two numeracy booklets and one mathematical literacy booklet. In mathematical literacy whole tasks were classified at one or other 5-14 level. In numeracy, the set of items at a particular level across both the numeracy booklets taken by a pupil together constituted a level-based numeracy test for the pupil concerned. Thus, both numeracy attainment and mathematical literacy attainment are able to be reported in the same way as reading attainment, in terms of level attainment using the same cut-off score criteria.

5.6 Pupils with who had 'made a good start' at a level answer at least 50% of the items at that level correctly but fewer than 65%. Pupils having 'well-established skills' at a level answer 65% or more of the items at the level correctly but fewer than 80%. Pupils with 'very good skills' at a level answer 80% or more of the test items correctly.

Numeracy attainment

Figure 7
Numeracy attainment profiles at P3, P5, P7 and S2*
(% pupils classified into attainment bands within 5-14 level: SSA 2005)

* Pupils have 'very good skills' at a level if they successfully answer 80% or more of the test items at that level across their two test booklets; they have 'well-established skills' if they successfully answer 65% or more of the items but not as many as 80%; they have 'made a good start' at the level if they answer correctly at least 50% of the items but not as many as 65%. Approx. pupil sample sizes: 3,300-3,400 at primary stages, ,2,900 at S2. Margins of error of 1_-2 percentage points for 'well-established skills' or better, varying with stage and level.

5.11 The picture in Figure 7 shows the pattern of attainment for P3, P5, P7 and S2 at Levels A-F. Levels B and C are the levels slightly above those expected for P3 and P5. For P7 and S2 Levels D and E are those expected for their stage. The overall pattern of attainment is similar to the pattern reported for the AAP mathematics survey in 2004, with the proportions of pupils showing well-established skills and very good skills increasing through the primary school to P7 but decreasing markedly from P7 to S2.

5.12 Looking more closely at lower attainment, Figure 7 also shows attainment in P3, P5, P7 and S2 at Levels A, B, C and D, respectively. For P7 and S2 this is the level below that expected for their stage. For P3 and P5, the levels are those expected for almost all pupils (90%).

5.13 Again, the results give a very positive picture for primary pupils. Around eight out of ten of the P3 pupils showed very good skills at Level A, and around six out of ten of pupils at P5 and P7 showed very good skills at Levels B and C respectively, all gaining 80% or more of the available marks. The patterns of well-established skills at P5 and P7 are almost identical, showing that pupils who may be less confident at the expected level are nevertheless making solid progress in numeracy. The proportions not achieving even a good start at these levels are very small, well under one in ten of the sample.

5.14 At S2, the pattern is slightly less positive, but nevertheless shows good attainment at Level D, with over three-quarters of pupils estimated as having well-established skills at the level, and only one in ten not achieving a good start at this level. This is an equivalent proportion to those gaining passes at grades 1-4 in Standard Grade mathematics in S4 in the same year, 2005.

Numeracy attainment in local authorities

5.15 The survey sample was drawn to allow some reporting at local authority as well as national level in 16 of the 32 Scottish local authorities. The results provide a snapshot in time, giving an indication of levels of attainment in numeracy in the summer of 2005. In future years it will be possible to show trends for each authority over two- or three-year periods.

5.16 As for reading, the survey looked at the differences between authority attainment and national attainment in numeracy at the various stages and levels, taking account of income deprivation.

5.17 As for Reading, the patterns of attainment for Numeracy varied across the 16 local authorities in focus in this survey. For example, the proportions of P3 pupils estimated as having well-established skills or better at Level B varied from around 30% to around 50%; at P5 Level C attainment estimates varied from 45% to over 65%; at P7 the variation at Level D was around 55% to just under 80%; and at S2 at Level E the range was just under 40% to around 55%. The variation at every stage and level was steadily progressive, with no clear authority grouping.

5.18 As might be expected, the results suggest that pupils in the two least income deprived authorities achieved relatively high attainment rates. Looking at P3 Level B, P5 Level C, P7 Level D and S2 Level E, 'well-established skills' attainment rates were among the highest in the country at all stages in East Renfrewshire and in P5 and S2 in East Dunbartonshire.

5.19 Among other authorities with higher levels of deprivation, estimates suggest that pupils at all four stages in Highland and South Ayrshire did better than might be expected. South Lanarkshire also showed a very positive picture at the primary stages, as did Renfrewshire at S2. P7 pupils in West Lothian showed relatively low attainment.

5.20 Amongst the most deprived authorities, P5 pupils in Inverclyde appeared to do particularly well, showing attainment of well-established skills at Level C higher than Perth and Kinross, one of the least deprived authorities.

5.21 For the lower attaining pupils in the survey the results suggest that pupil attainment in the three least income deprived authorities was again the most positive in the country. Attainment results for pupils in Edinburgh City and Aberdeen City were less positive than might be expected.

Maths Literacy attainment

5.22 The maths literacy results are as follows:

5.23 These results suggest quite consistent progression through the primary school in applying number and data handling skills in everyday contexts, while there appears to be little progress between P7 and S2 in this type of maths.

5.24 Comparing attainment for maths literacy with attainment for numeracy, the proportions of pupils attaining the different levels vary from around 10% to 30%, with the biggest differences at Level A for P3 and Level D for P7 and S2, at 20% or more. The exception was at P5, where there was no difference in attainment on the two different types of task at Level C.

5.25 These differences are very similar to those between numeracy (number, money and measurement and information handling) and problem-solving in the 2004 maths survey.

Teachers' judgements about attainment in Mathematics

5.26 For mathematics, teachers quite consistently classified more pupils as achieving the various levels than the numeracy survey, by around 10 percentage points. The closest match between the teachers' judgements and the survey was at P3 for Level A. The differences were greatest for S2, where teachers judged rather more pupils to have achieved Levels E and F in mathematics than the survey did for numeracy, by around 15 percentage points at Level E. At P5, however, teachers judged fewer pupils to have attained Level D, suggesting that there may be underestimation of some pupils' capabilities in mathematics at this stage.

5.27 Agreement rates varied across the authorities, with a range of roughly 50-65% at P3, 40-55% at P5, 45-65% at P7, and 45-65% at S2. Agreement rates were significantly better than average in East Dunbartonshire at P3, North Ayrshire at P5, Edinburgh City and South Lanarkshire at P7, and South Ayrshire and Stirling at S2. Agreement rates were relatively low in North Lanarkshire, East Ayrshire and Inverclyde at P7, and in Renfrewshire at S2.

Gender differences in Numeracy and Maths Literacy

5.28 Overall, the survey found no significant differences in attainment in numeracy or maths literacy between boys and girls. In this respect, Scotland differs from many other countries, where boys' attainment tends to be higher. The pattern is of slightly higher attainment for higher attaining boys, especially at the early stages, and slightly better attainment for lower attaining girls at P7 and S2. These findings mirror the results of the AAP mathematics survey in 2004.

Changes in Numeracy attainment over time

5.29 Figure 8 shows the comparative picture of attainment of P7 and S2 pupils in 2000 and 2005. The data for 2000 derive from the AAP survey of mathematics that year - numeracy data have been isolated for this comparison. P3 and P5 pupils were not assessed in the 2000 survey, which included P4 along with P7 and S2, and no numeracy items were included at Level F that year. No comparative data are available from the 2004 AAP survey of mathematics, because the 2004 survey was designed to produce attainment proportions for mathematics as a whole, along with 'average item facility' comparisons for the four mathematics outcomes separately: in consequence, fewer than half of the items administered in the survey were numeracy items, too few per pupil for cut-off scores to be meaningfully applied.

Figure 8
The comparative picture of 'well-established' numeracy skills in 2000 and 2005 *
(% pupils achieving 65% or more of marks at a level)

* Figures are based on sets of 11-12 numeracy items per pupil at a level in the AAP 2000 survey of mathematics and 18 items per pupil in the SSA of 2005; numeracy items at Level F did not feature in the 2000 AAP survey.

5.30 The striking feature in Figure 8 is the improvement over time at Level D. The Level D attainment rates for both P7 and S2 are significantly higher in 2005 than in 2000.

5.31 Figures 9a and 9b compare the trends in Numeracy attainment evident in teachers' judgements (the 5-14 National Survey of Attainment) and AAP/ SSA results.

Figure 9a
Trends in Numeracy attainment at P7
(% pupils with 'well-established' skills at the indicated levels)

Figure 9b
Trends in Numeracy attainment at S2
(% pupils with 'well-established' skills at indicated levels)

Note: National Survey results are 1998/99-2003/04 National Survey of 5-14 Attainment Levels, 2004/05 are sampled teacher judgements as provided in the SSA. AAP/ SSA results are from the AAP in 2000 (no Level F numeracy items were used in that survey) and the SSA in 2005.

5.32 Results from the national survey/teacher judgements show steady increases in achievement over the period, for both stages at all levels, with particularly marked jumps at Level E for P7 and Level F for S2 in 2005. In contrast, the latest SSA results for numeracy show an increase in attainment since 2000 at Level D only. This finding is similar to that of the 2004 AAP mathematics survey compared to the 2000 survey, which found no significant change in mathematics performance at any stage or level.

Assessing and reporting Information and Communications Technology ( ICT)

5.33 Around 200-250 pupils at each stage from over 250 schools undertook ICT assessments, each pupil attempting one task. There were six ICT tasks in total, two of which were used at all four stages. Most of the pupils also completed an ICT questionnaire.

5.34 The framework for the assessment of ICT was developed following consultation with an ICT Focus Group comprising practising teachers from across Scotland. The assessment tasks focused on the assessment of the strands Searching & Researching, Creating & Presenting and Using the Technology, and were designed to allow an overall ICT level to be awarded. The questionnaire explored pupils' attitudes to ICT and gathered some contextual information relevant to ICT learning.

5.35 Since the main focus of the 2005 survey was English Language, the context for the practical ICT tasks was language-based. The tasks were designed to assess pupils' ability to find information, to transform the information into a form suitable for their purpose, and to present the information in a way that took into account the audience for their presentation. The main purpose for pupils was to work with a partner to produce a multimedia presentation for a given audience on a choice of topics. Task resources were provided.

5.36 The tasks were administered by field officers working with pairs of pupils. The field officer introduced the task, familiarised the pupils with the laptop computer and the resources, and encouraged discussion about the task. Pupils were allowed approximately 40 minutes to complete their task. The field officer observed the pupils as they worked and noted observations on a recording sheet. After saving their work the field officer carried out a structured discussion with the pupils, posing set questions categorised into appropriate levels for the stage concerned.

5.37 The field officer used information gathered from observations during the task and from the post-task discussion to award an attainment level to each pupil, using 'best fit' descriptors based on the 5-14 Guidelines for ICT.

5.38 As with other practical assessments in the survey, the numbers of pupils involved in the ICT assessment at each stage were small compared with the numbers involved in the written assessment of reading and numeracy. They were also not fully representative of all pupils at the stages concerned, given that the schools involved in the practical assessments had been selected for their size (at least 20 pupils at a stage in the general survey sample) and location (convenient for field officer visits). These facts should be borne in mind when the assessment results are reviewed.

Pupil's access to, usage of and attitudes towards ICT

5.39 At all stages, more than nine out of ten pupils said that they had a computer at home and almost all said they liked using computers. Just under eight out of ten agreed that they were able to access the internet at home and a third had a computer in their bedroom. These access rates are much higher than those that emerged for the survey sample as whole, where nearly three-quarters of the 26,000+ pupils who completed the main survey questionnaires claimed to have computer access at home, and around six out of ten agreed that they had home internet access (see 5.4).

5.40 Pupils reported that they used computers more at home than at school, with home use increasing with age while school use remained relatively static: around four out of ten of the primary pupils claimed to use their home computer every day, rising to almost six out of ten of the pupils at S2, while fewer than one in ten of the pupils at every stage reported this frequency of use at school. The proportion of pupils reporting 'too little' use of computers at school increased with age, from a quarter of pupils at P3/P5 to a third at P7 to over half at S2. The rest, the majority among primary pupils, thought that the amount of time spent using computers in school was 'just right'. Only a handful of pupils at each stage thought computer were used 'too much' at school.

5.41 Pupils tended to have greater access to the more modern computer technologies at home than at school: in particular, CD- ROMs, DVDs, digital cameras, scanners and webcams were all much more frequently used at home than at school (for example, at S2 'at home' use of these technologies was reported by 55-80% of the pupils compared with in-school figures of 30% for CDROM, 15% for scanner and under 10% for other technologies). Floppy disks were used by half of all the pupils at home (two-thirds at S2) and by under a third at school.

5.42 Accessing the internet and using computers to help with school work or homework ranked highly in both home and school (between two-thirds and three-quarters of pupils reporting these uses). Games playing, on the other hand, was more frequent at home than at school: at all stages over three-quarters of the pupils used their computers at home to play games compared with under half on average at school (over half at P3 dropping to a third at S2). Almost half of all pupils (nearly two-thirds at S2) used email at home, compared with around a fifth doing so at school (one-third at S2).

ICT attainment

5.43 The principal results of the ICT assessment are as follows:

5.44 There is evidence of steady attainment progression through the primary stages, at Levels B, C and D. Between P7 and S2 there seems to be a slight dip, with half of the S2 pupils judged as attaining Level E compared with nearly two-thirds of the P7 pupils judged as Level D. Attainment at Levels E and F in P7 and S2 respectively is comparable, at just under one fifth of pupils, suggesting steady progress for the higher attainers in ICT.