Scottish Government

3. Practical mathematics

3.1 The assessment process

3.1.1 The assessment tasks

The practical assessments conducted within this survey included aspects of the curriculum which are not readily assessed through paper and pencil assessments. Tasks were developed by identifying attainment targets which might be assessed by a field officer through one-to-one interaction with pupils, adopting a format which would allow the field officer to explore a pupil's mathematical understanding in a little detail. Given the constraints of the survey, and the limited number of practical tasks which could be included, particular attention was paid to identifying tasks which showed a clear progression within strands across levels. Consideration was also given to the likely availability of any resources or equipment which would need to be used.

Four resource-based practical mathematics tasks were newly developed for use in the survey. Each task comprised a series of questions and activities, progressively moving from Level A to Level E, and was designed to be undertaken by pupils in an interactive session, with an adult (in this survey an itinerant field officer) guiding and observing the activity, posing the questions and completing an associated checklist.

One task focused on pupils' abilities to handle money. The resources comprised a selection of coins and notes, a series of cards picturing various items labelled with prices, a currency conversion chart with simplified sterling/euro exchange rate, and a 'sale card'. A calculator was available for any pupils who might reach the Level E activities, which involved currency conversions and discount calculations. At Levels A to C, pupils were asked to give the exact money to the field officer to pay for particular objects, and were then asked whether they could have paid for the object concerned in some other way, still giving the exact amount but with a different combination of notes and coins. The field officer would then offer to 'buy' the object from the pupil, giving a higher amount of money than the object's price, and the pupil would need to calculate the appropriate change. Pupils were each time asked to explain how they worked out the amount of change. At Level E the concept of exchange rate was introduced, while at Level F pupils were tested for their ability to calculate sale discounts.

A second task looked at time and measure. The 'time' resources were an analogue clock and a card showing digital displays. The 'measure' resources were a measuring tape, a metre stick, a ruler marked with cm and mm, four different lengths of string, and a card showing pencils of different length. Level A to C activities required pupils to set the analogue clock to various times, while those at Levels D and E instructed them to set the analogue clock to the times shown on one or other of the digital displays. After each activity, pupils were invited to explain what they had done. The assessment of pupils' understanding of measure began by asking the pupils to compare given objects by length, width or height, identifying the longest, the widest and the tallest (Level A), always explaining their choices. They were then asked to measure the length of a given object, and to choose one of the pieces of string that matched a given length (Level B activities). Measurement moved on to the length/width of the classroom (which involved measurement in metres - a Level C activity), and then (Level D) to the mm lengths of the illustrated pencils (this involved, among other things, giving a millimetre measurement as a decimal centimetre measurement).

A third task explored pupils' understanding of fractions, percentages and ratio. The first half of the task, Levels A to C, focused on activities involving paper circles and blank grids (20 square, 36 square, 50 square, 100 square). Pupils were asked to divide the circle into halves (for them 'two pieces of exactly the same size') and then into quarters ('four pieces of exactly the same size'), to name the resulting shapes and to explain their responses. At Levels C to E they had to shade various fractions and percentages of different squared grids, explaining how they worked out how many squares to shade. The second half of the task focused on coloured cubes/counters, with pupils required to give the field officer appropriate numbers of cubes to represent particular fractions (Levels A to C) or percentages (Level D) or to count appropriate numbers of coloured cubes to represent given ratios (Level E).

The fourth task featured shape, angle and direction. The 'shape and angle' resources included a collection of 3D shapes, including spheres, cubes, cuboids, cones and cylinders, triangular prisms and square pyramids. A protractor was also available, along with a ruler and pencil. The 'direction' resources comprised a roamer, floor turtle or similar, an optional floor grid and paper turtle grids, a compass and paper compass rose. Pupils were first asked to identify which of the various 3D shapes would roll, and to name the shapes (Level A). They were then asked (Level B activities) to identify an edge and face of a cylinder, to count the edges and faces, to select a shape with nine edges and five faces, and to name it (triangular prism). At Level C they were to identify 2D shapes in the triangular prism, to count the angles, to identify a 90 degree angle and an angle smaller than this, and to give the names for these (right angle and acute angle). At Level D pupils had to measure one angle and draw another of given size, while at Level E they were asked to draw a triangle with one given side length and two angles, and to measure the third angle. In addition, pupils were asked (Level A to C activities) to programme the turtle to make various movements within the turtle grid (or to instruct the field officer to make the same movements on the floor should a turtle or similar not be available), and then to give (Level D) the compass directions of and (Level E) 3-figure bearings for two given objects.

3.1.2 Task administration

The same four tasks were used at all four stages. Consequently no artificial floor or ceiling was put on a pupil's attainment. Pupils were encouraged to work as far through their task as they could, with the assessment ending as soon as it became clear that the limit of a pupil's knowledge/ability had been reached.

For cost and logistic reasons, the tasks were administered in a subsample only of the survey schools, and, within these schools, to subsamples of the pupils involved in the 'pencil and paper' assessment of mathematics. Typically, four randomly selected pupils in each 'practical' school undertook practical mathematics tasks, each of the four attempting a different one of the four tasks available, along with one or other of the 'mathematical literacy' tasks described in Chapter 2.

The tasks were prepared and presented to the pupils by trained field officers. The 137 field officers were practising teachers, who had been released by their authorities from their normal teaching duties for seven days each to take part in the practical assessment. They attended one day of task orientation in May/June 2004 (different meetings at different times for different stages and locations). They then worked in pairs, visiting five assigned schools, spending one day in each (dates agreed beforehand with the schools concerned), before ending their involvement in a debriefing day in June 2004. Just over half the field officers (71) worked at P3/P5, the rest (66) working at P7/S2.

Both field officers in any pair worked together when administering the various practical assessments, including these practical mathematics tasks. One of the field officers introduced the task to the individual pupil and proceeded to work through the task with that pupil, while the other observed and recorded observations on the appropriate checklist (the guidance given to the field officers for this type of assessment is reproduced in Appendix C).

In the event, 250-450 pupils at each stage (1465 in total) were involved in this particular type of practical assessment, drawn from 299 primary schools and 74 secondary schools. Between 70 and 110 pupils attempted any one task, the number varying by task and stage.

3.1.3 Rating pupil performance

Pupils were rated for the quality of their responses, actions and explanations, where they gave/demonstrated any, in terms of correctness and of amount of support needed from the interacting field officer: 'minimal support', 'some support', 'considerable support'. They were also rated for their use of appropriate mathematical language, and for their competence when using the supplied equipment: 'competent', 'unsure' and 'incompetent'.

Clearly, many of the judgements demanded in this particular assessment exercise were inevitably subjective in nature, and subjectivity in assessment raises questions about the comparability of rating standards. What to one field officer would be considered 'minimal support' might to another be considered 'some support'. What one field officer might consider 'competent use' of equipment might by another be considered less so. What one field officer might accept as an appropriate pupil explanation for an action or calculation might to another be considered wanting. Unfortunately, for a number of reasons, it was not possible to conduct formal rater agreement trials before the practical mathematics tasks were used in the survey. We cannot therefore comment on the extent to which different field officers applied the same standards of judgments when making their real-time ratings in some areas.

Objectivity will have been highest when field officers were simply judging whether or not a pupil had offered a correct answer to a direct question (e.g. 'How many quarters are in a whole?' or 'How much are the trainers reduced by, when there is 10% off £60?'), or whether or not a pupil had carried out an appropriate action in response to a direct instruction (e.g. 'Shade _ of the squares' in a 6x6 grid or 'Set the analogue clock to half past ten'). This chapter therefore focuses on reporting the findings of this particular type of assessment: i.e. correct answers or actions.

3.1.4 Reporting knowledge and skills attainment

Given the nature of the four practical tasks that were administered in the survey, it is not possible to offer attainment results in terms of the proportions of pupils attaining particular levels in practical mathematics, even for the objectively rated aspects of the assessment. This is because the number of level-classified demands (essentially test items) within the tasks varied from highs of 7 to 11 per level to lows of 1 or 2 per level (see Tables 3.1 to 3.4). For the same reason it is not useful to offer task comparisons, in terms of task mean scores. Pupil performance is therefore described for each individual question/activity, for the case where the question was answered correctly, or the activity appropriately carried out, with 'minimal support' from the interacting field officer.

3.2 Overview of pupils' attainments

3.2.1 The attainment picture across the stages

Table 3.1 presents the performance results for the first task, focusing on money.

* In each case the percentage "correct" is based on all pupils embarking on the task, whether or not they actually reached the question/activity concerned.
** This is not the wording actually used with pupils.

As Table 3.1 shows, high proportions of the pupils at P7 and S2, typically 80-95%, were competent in all the money handling activities at Levels A to D, with lower proportions successful at Level E (currency conversion and percentage calculation), especially among the P7 pupils.

High proportions of the P5 pupils were successful in the activities at Levels A to C, with the exception of calculating change at Level C, where the proportion of successful pupils fell to just over 40%. For the Level D activities the proportions of successful P5 pupils were still over half, with the change calculation again the least well done. Very low proportions of P5 pupils successfully managed the Level E activities.

The proportions of successful P3 pupils fell steadily through the levels, from 90%+ at Level A to none at Level E (at which level few P3 pupils would actually have been assessed). Calculating change again proved clearly the most difficult task at every relevant level.

At all stages, 90% or more of the pupils who could successfully calculate change, carry out currency conversions and calculate percentages, could also appropriately explain how they had carried out the tasks.

The second task, on time and measure, is profiled in Table 3.2. The picture of performance is again very positive at P7 and S2, with 85-95% of the pupils successfully completing the various clock setting activities at Levels A to E, with the interesting exception of the one Level D activity, in which pupils were to set an analogue clock to display the time on a drawn digital display (the similar Level E activity was better done). For the measuring activities, high proportions of the P7 and S2 pupils, 80%+, were successful at Levels A and B, falling to around 70% at S2, and 50-75% at P7, for the activities at Levels C to E.

* In each case the percentage "correct" is based on all pupils embarking on the task, whether or not they actually reached the question/activity concerned.
** This is not the wording actually used with pupils.

At P5, 85-95% of the pupils were also successful in carrying out the clock setting activities at Levels A and B, falling to 60-70% for the activities at Levels D and E. The majority of the P5 pupils, 70-90%, were also able to demonstrate a sound understanding of length, and the ability to accurately measure integer cm lengths. Just over half were able to use a metre stick to measure the sides of their classroom, just under 30% were able to measure integer mm lengths, but only 15% were able to express an integer mm length as a decimal cm length.

At P3, success rates for 'time' fell steadily from 90% for the Level A clock setting activity to under 10% for the Level E digital to analogue display activity (not all the P3 pupils would have reached this activity). Similarly, for measurement, an 80% or so success rate for Level A activities fell to 5% or less for Level D/E activities.

As for the task on money, pupils who were able successfully to complete the various activities were generally also able to explain how they did so to the field officer (80-90%, with explanatory ability increasing with stage).

The performance findings for the third task, on fractions, percentages and ratio, are presented in Table 3.3.

* In each case the percentage "correct" is based on all pupils embarking on the task, whether or not they actually reached the question/activity concerned.
** This is not the wording actually used with pupils.

According to the evidence in Table 3.3, the majority of pupils at P7 and S2 (85-95%) were familiar with the fractions _ and _, and could demonstrate/apply these successfully whether using paper circles or cubes/counters (Levels A/B). At S2, 60-70% of the pupils were successfully able to work with the fractions 1/5, 1/8 and _, and with simple percentages, compared with 40-60% of the pupils at P7 (Levels C/D). Two-thirds of the S2 pupils showed understanding of ratio, though fewer could implement given ratios using coloured cubes/counters (Level E). At P7, around one-fifth of the pupils could identify a given ratio and could also implement simple ratios.

At P5, most of the Level A/B activities were successfully carried out by 80% or more of the pupils, with a lower 65% successfully demonstrating an ability to handle the fraction _ with coloured cubes/counters. Level C activities involving the fractions 1/5 and 1/8 were less well done, with a third or fewer pupils showing familiarity with these fractions. Percentages (Level D) were not well handled by P5 pupils either, 15% or fewer successfully calculating given percentages, whether shading squares or counting cubes. Ratio (Level E) was a concept handled successfully by just handfuls of pupils at this stage (most will not have reached the Level E activities).

At least half the P3 pupils were familiar with the fractions _ and _, with the exception of the cube counting activity involving _, where the percentage successfully completing the activity fell to around a third. Other fractions, and simple percentages, were familiar only to very small proportions of the pupils (5% or less) and the concept of ratio to none.

Again, the majority of pupils who successfully carried out the activities could explain their method/reasoning to the field officers.

Table 3.4 completes the picture, giving the performance results for the fourth task, on shape, angle and direction.

* In each case the percentage "correct" is based on all pupils embarking on the task, whether or not they actually reached the question/activity concerned.
** This is not the wording actually used with pupils.

Table 3.4 reveals another clear pattern of stage progression. Typically, 65-85% of the S2 pupils were able to answer the questions successfully and carry out the various activities. The least well done activities were counting the angles on a triangular prism (Level C, 56%), drawing a triangle with given dimensions (Level E, 44%), and giving 3-figure bearings for two given objects (Level E, 53%, 59%). At P7, performance was similar to that of S2 for identifying and naming shapes that roll, recognising edges and faces of objects, programming a turtle and giving compass directions. Markedly lower proportions of the P7 pupils compared with the S2 pupils could count the angles on a triangular prism, identify and measure angles, including right angles, draw a triangle of given dimensions, and give 3-figure bearings.

At P5, between one half and three quarters of the pupils were able to identify and name shapes that roll, and were familiar with the concepts of edge and face; 45-60% were familiar with right angles and acute angles, could programme a turtle and give compass directions. Very low proportions of the P5 pupils (fewer than 5% in each case) reached and successfully carried out the Level D/E activities involving measuring or drawing angles or the Level E activities involving 3-figure bearings. This finding will not be surprising. More surprising is the fact that any P5 pupils actually achieved at these levels in these topic areas, and with 'minimum support'.

At P3, between one half and three quarters of the pupils could identify and name shapes that roll and identify the edge and face of a cylinder. For other activities the proportions were much lower, with none at all reaching or successfully completing the Level D/E activities involving angle measurement and 3-figure bearings.

Only one explanation was asked for in this task, and that was to explain how the pupils could tell that the shapes they identified could roll without actually trying them out. As before, where pupils could identify appropriate shapes they could usually (in 90% or more of cases) explain how they knew.

In all four tasks, the frequency of use of mathematical terminology (e.g. 'equal', 'three quarters', 'one tenth', 'ratio', 'multiply', 'currency', 'minute hand', 'shorter', 'metre stick', 'rectangle', etc) increased with stage.

3.2.2 Gender comparisons

It is difficult to say anything useful about possible gender differences in practical mathematics, given the small numbers of pupils that attempted each of the four tasks (70-110 in total, varying by task and stage, with a roughly equal representation of boys and girls). For the time and measure task there was no particular pattern of gender difference. The same was the case for the fractions task. In the money task, proportionally more of the boys than girls at P5, P7 and S2 succeeded on the two currency conversion activities (Level E), but for only one task at one stage did the difference reach statistical significance. For the shape, angle and direction task there was a tendency, at least at P7 and S2, for the boys to be more successful than the girls in giving compass directions and 3-figure bearings, but the activity differences rarely reached statistical significance. A larger-scale enquiry, in terms of pupil sample sizes, would have been useful in allowing any gender differences in practical mathematics to emerge more clearly.

3.3 Summary

Four resource-based practical tasks were administered at all four stages in the survey. The tasks focused on one or other of 'money', 'time and measure', 'fractions, percentages and ratio' and 'shape, angle and direction', with each task involving activities at increasing 5-14 levels.

Between 70 and 110 pupils at any stage attempted the individual tasks, 300-400 pupils in total at a stage. The pupils were guided through the tasks by itinerant field officers, the assessment ending when pupils were deemed to have reached their personal levels of capability.

Success rates were generally higher at P7 and S2 than at P5, and higher at P5 than at P3, and success rates naturally fell as the level of the activity demand increased.

At P5, P7 and S2, 80% or more of the pupils successfully carried out most of the activities involving money, up to Level B/C at P5, Level C at P7 and Level D at S2. Performance was lower at P3, particularly when pupils were required to calculate and deliver change rather than simply offer coins up to a given price. Discount problems were handled well by half to three-quarters of the S2 pupils, and 30-50% of the P7 pupils.

Time (setting analogue and digital displays) also proved relatively unproblematic for the P7 and S2 pupils, 80% or more of whom successfully completed the activities at all levels unaided. At P3, 50-70% of the pupils managed the Level B activities unaided, whereas 60-70% of the P5 pupils successfully tackled the Level C to E activities. A roughly similar picture emerged for activities involving length measurement, except that metre and millimetre measurement was less well done than integer centimetre measurement, with marked attainment gaps between consecutive stages.

The fractions/percentages task revealed a steady attainment progression through the four stages, with particularly marked attainment gaps between P5 and P7 at Level D (percentages) and between P7 and S2 at Level E (ratio). The majority of the pupils at P7 and S2 (85-95%) were familiar with the fractions _ and _ (Levels A/B), along with 80% or so at P5 and half or more at P3. At Levels C/D, 60-70% of the S2 pupils were also successfully able to work with the fractions 1/5, 1/8 and _, compared with 40-60% of the pupils at P7, at most one-third at P5, and fewer than 5% at P3. The proportions able to handle simple percentages successfully were around the same at P7 and S2, but fell markedly at P5 to around 15%, with 5% or fewer at P3. Two-thirds of the S2 pupils showed an understanding of ratio, compared with one-fifth of the P7 pupils, a handful of the P5 pupils and no P3 pupils.

A variety of activities involving naming and drawing shapes and angles again revealed clear evidence of stage progression. Turtle programming, or giving equivalent verbal instructions, was successfully achieved by 20-30% of the P3 pupils, 45-60% of the P5 pupils and 70% or more of the pupils at P7 and S2. Similar proportions of pupils were successfully able to use a compass to give directions, except for a drop in performance for this activity at P3. At S2, 50-60% of the pupils could give 3-figure bearings for objects, compared with 25-30% of the pupils at P7, fewer than 5% at P5 and none at P3 (if they even reached these activities before assessment ended)

At all stages and levels, the majority (typically 90% or more) of those pupils who successfully carried out the various activities could give an acceptable explanation for their methods and answers.