Scottish Government

B: Science knowledge and understanding

B.1 Assessing and reporting Science knowledge and understanding

The 2007 SSA aimed to:

assess and report pupils' Science knowledge and understanding achievement by 5 to 14 levels at P3, P5, P7 and S2 nationally and for opted-in local authorities.

This was achieved through written testing, covering the three Science outcomes set out in the 5-14 Environmental Studies guidelines. These outcomes are Earth and Space, Energy and Forces and Living Things and the Processes of Life.

Most of the 634 tasks used to assess Science knowledge and understanding in the 2007 survey were newly developed, in many cases by adapting tasks used in the 2003 Assessment of Achievement Programme ( AAP, the precursor to the SSA) survey of Science. Others were carried directly over from the AAP survey. Tasks comprised one or more test items, carried between one and four marks, and had been classified at an appropriate 5-14 Level (A to F) prior to survey use by practising teachers.

At P3, pupils were assessed at Levels A and B, whilst at the other stages pupils were assessed at three consecutive Levels: B, C and D at P5, C, D and E at P7, D, E and F at S2. Booklets were designed to contain tasks meriting the same number of total marks at each of the levels to be assessed at a stage. This avoided the risk of distressing any pupil by facing that pupil with a Science assessment that might be at an entirely inappropriate level. The booklets were also balanced in terms of the 5-14 outcomes mentioned above, with tasks presented in 'themed' blocks (e.g. 'magnetism', 'plant anatomy', etc) and two or three tasks per block. Multiple matrix sampling was used to assign booklets to pupils, with each pupil allocated two booklets. The tasks at a level across the two booklets attempted by a pupil constituted a test at the level concerned. Further details of the assessment materials and procedures are available in Annex II.

The results of the survey are reported using the following criteria for test performance:

• 'very good' knowledge and understanding at a level, indicated by test scores of at least 80 per cent;
• 'well-established' knowledge and understanding, test scores of at least 65 per cent but less than 80 per cent;
• 'made a good start', test scores of at least 50 per cent but less than 65 per cent.

In total, just over 35,000 pupils participated in the written assessment of Science knowledge and understanding. All estimates of achievement presented in the following sections are adjusted for the over-representation of opted-in authorities in the national sample, and also for sample imbalance in terms of gender and deprivation. As with all sample surveys, there is a level of uncertainty inherent in the results. Where appropriate, standard errors are quoted alongside each achievement estimate, and the size of the standard errors should be taken into account when interpreting the data. Further details of the sample design are available in Annex I.

B.2 National Science knowledge and understanding achievement estimates

Chart B1 shows the percentage of pupils in each of P3, P5, P7 and S2 categorised into achievement bands at Levels A-F for Science knowledge and understanding. Detailed results are available in Table B1.

Chart B1
Science knowledge and understanding achievement estimates
(% pupils classified into achievement bands by 5-14 level)

Standard errors are a maximum of 0.8 percentage points for 'well-established or better', varying with stage and level.

In P3, almost 55 per cent of pupils demonstrated well-established or better knowledge and understanding at Level A (the expected level for the stage) with just over 30 per cent already doing so at Level B. For pupils in P5, almost 50 per cent demonstrated well established or better knowledge and understanding at Level B, about 10 per cent did so at Level C (the expected Level for P6), and around 1 per cent at Level D. In later stages, almost 25 per cent of P7 pupils showed well-established or better knowledge and understanding at Level C, just over 5 per cent at Level D (the expected level) and about 2 per cent at Level E. At S2, 19 per cent of pupils demonstrated well-established or better knowledge and understanding at Level D and 17 per cent at Level E (the expected level) and about 2 per cent at Level F.

B.3 National Science knowledge and understanding achievement estimates, by gender

Chart B2 compares the achievement of boys and girls within the survey. Detailed results are available in Table B2.

Chart B2
Science knowledge and understanding achievement estimates, by gender
(% pupils having 'well-established' skills or better at the given level)

The thin black lines associated with each achievement bar are the 95% confidence intervals. These are described in Annexe I.

The clearest picture revealed by the chart is the consistent gender difference in favour of the girls at Level A in P3, switching to a consistent difference in favour of boys at Level C and above in P5, P7 and S2. At Level B in both P3 and P5 there was no significant difference in the proportions of boys and girls achieving expected levels.

B.4 National Science knowledge and understanding achievement estimates, by deprivation category

Chart B3 compares the achievement of pupils from the 20 per cent most deprived areas against those from less deprived areas. Detailed results are available in Table B3.

Chart B3
Science knowledge and understanding achievement estimates, by deprivation
(% pupils having 'well-established' skills or better at the given level)

'Most' means the pupil lived in one of the 20 per cent most deprived areas in Scotland based on the 2004 Scottish Index of Multiple Deprivation ( SIMD), the remainder of pupils are classified as 'less' deprived.

The thin black lines associated with each achievement bar are the 95% confidence intervals. These are described in Annex I.

Pupils from less deprived areas do better than pupils from the most deprived areas at every stage and level. All differences are statistically significant and increase with stage, with the largest differences at P5 Level B, P7 Level C and S2 Levels D and E.

B.5 Teachers' judgements

In addition to the written assessments for Science knowledge and understanding, teachers were invited to submit a level judgement for Science (and Reading, Writing and Mathematics - these are discussed in Chapter D relating to core skills) for each pupil sampled in the survey. Chart B4 provides the results of this enquiry for Science, with detailed figures given in Tables B4 and B5.

Chart B4
Teachers' judgements of their pupils' Science achievement
(% pupils judged to be at indicated level by their teachers)

At all primary stages, teachers judged the majority of pupils to be within one or other of two levels - generally the expected level for their stage and the level either above or below it. For example, at P3, teachers judged 60 per cent of pupils to be at Level A and 35 per cent to be at Level B. At P5, 40 per cent were judged to be at Level B and almost 55 per cent were judged to be at Level C. This is the expected level for P6; there is no expected level for P5. At P7 over 60 per cent of pupils were judged to be at Level D, the expected level, while at S2 over 40 per cent were judged to be at Level E, the expected level, with almost 20 per cent at Level F.

In order to provide corresponding level profiles for all pupils at a stage based on the survey test results, we make the assumption that all pupils who achieve a particular level will be secure at all lower levels: for example, we assume that the 32 per cent of P3 pupils who demonstrated 'well-established or better' knowledge and understanding at Level B would be counted among the 54 per cent who demonstrated 'well-established or better' knowledge and understanding at Level A (see Chart B1 and Table B1). By subtraction of the pupils in the next highest level we get the proportion reaching their maximum levels: so for P3 we can say that 22 per cent (i.e. 54 per cent - 32 per cent) of pupils demonstrated Level A achievement but not Level B.

Comparing the resulting level profiles at each stage with those produced using teachers' judgements we see (Chart set B5) a very clear and consistent difference, in that the test-based results are in every case the less positive. Teachers' judgements and test-based results are not necessarily equivalent indicators of pupils' achievement levels, so it might not be surprising to find differences emerging in the results of the two approaches.

Chart set B5
Teachers' level judgements for Science compared with SSA test-based results*
(% pupils judged to be at indicated levels by teachers and proportion of pupils in highest level from the SSA)

Judgements are based on a range of knowledge that the classroom teacher has about each pupil, which has been gained over a period of time. These judgements will be locally moderated within the school. SSA testing, on the other hand, is centrally administered and designed to provide comparable performance opportunities for all pupils in the survey. In this case, the results are based solely on performance on the day of the test and pupils' responses to the material that is contained within them.

B.6 Detailed results for Science knowledge and understanding

The tables provide detailed results to support the information provided in this section.

Table B1
2007 SSA - National Science knowledge and understanding achievement estimates
(% pupils categorised into each achievement band ^*)

* '< 50%' means fewer than 50% of the tasks in the level-based test answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

Table B2
2007 SSA - National Science knowledge and understanding achievement estimates, by gender
(% pupils categorised into each achievement band ^*)

* '< 50%' means fewer than 50% of the tasks in the level-based test answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

Table B3
2007 SSA - National Science knowledge and understanding achievement estimates, by deprivation** category
(% pupils categorised into each achievement band ^*)

* '< 50%' means fewer than 50% of the tasks in the level-based test answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

**'Most' means the pupil lived in one of the 20% most deprived areas in Scotland based on the 2004 Scottish Index of Multiple Deprivation ( SIMD), the remainder of pupils are classified as less deprived.

Table B4
2006 SSA - National Science knowledge and understanding achievement estimates: teachers' judgements
(% pupils judged to be at indicated levels by their teachers*)

Science achievement estimates: Teachers' judgements
(% pupils judged to be at indicated levels by their teachers)

Most' means the pupil lived in one of the 20% most deprived areas in Scotland based on the 2004 Scottish Index of Multiple Deprivation ( SIMD), the remainder of pupils are classified as less deprived.