Scottish Government

Scottish Survey of Achievement - 2006
Local Authority Report for Dundee City

Introduction

The Scottish Survey of Achievement ( SSA) uses a sample survey to find out how well pupils are learning in Scotland as a whole. The information is used to help plan for improvement to support quality learning and teaching. Alongside the national sample, 16 Local Authorities were selected for Local Authority level reporting in 2006 for certain aspects of the survey.

The 2006 SSA, which took place in the period mid-May to mid-June 2006, focused on attainment in Social Subjects (Enquiry Skills), and on core skills in the context of Social Subjects. The 16 Scottish Local Authorities selected for separate attainment reporting were: Aberdeenshire, Argyll & Bute, Clackmannanshire, Dumfries & Galloway, Dundee City, East Lothian, Eilean Siar, Falkirk, Fife, Glasgow City, Midlothian, Moray, Orkney, Scottish Borders, Shetland and West Dunbartonshire.

The specific objectives of the 2006 survey were to:

At National Level only

• Assess and report pupils' 5-14 level attainment in Social Subjects (Enquiry Skills) at P3, P5, P7 and S2.
• Assess and report pupils' attainment in core skills (working with others, problem solving and ICT) applied in a Social Subjects context at P3, P5, P7 and S2.
• Provide information on pupils' and teachers experience of, and views on, learning and teaching in social subjects.
• Provide information on parents' perceptions of their children's pre school experience, and on parents' experience of contact with their child's primary school.

At National and Local Authority Level

• Assess and report pupils' 5-14 level attainment in reading and numeracy at P3, P5, P7 and S2.
• Report pupils' 5-14 level attainment in writing at P3, P5, P7 and S2, on the basis of teachers' judgements.

Further information about the assessment and sampling approaches adopted can be found in Annex G.

Using this report

The following sections present findings for Scotland and the authority in reading, writing and numeracy/mathematics.

Full attainment results are provided in Annexes A and C for the written assessments in reading and numeracy. Results are broken down by gender in these annexes, although care should be taken in interpreting these figures. Given the small pupil sample sizes involved, any gender gaps in attainment would need to be in the range 10-40 percentage points to reach statistical significance, the size required depending on sample sizes and values of estimated attainment proportions.

The attainment results relating to teachers' submitted level judgements for reading, writing and mathematics are given in detail in Annexes B, C and E.

Where an observation is made that attainment in the authority was 'significantly' higher or lower than attainment nationally, this refers to statistical significance at the 5% level. In addition to highlighting the main features of the authority's attainment picture, each section also offers some questions that the authority might find helpful for self-evaluation.

In using either set of results for purposes of comparison and benchmarking, it is important to bear in mind the relative socio-economic context of the authority. It is recognised that educational attainment is related to deprivation.

Attainment in Reading

The charts below illustrate the profiles of reading attainment at the various stages for Scotland as a whole and for the authority.

SSA 2006 - Reading Attainment Estimates: Scotland
(% pupils classified into attainment bands within 5-14 level: SSA 2006)

SSA 2006 - Reading Attainment Estimates: Dundee City
(% pupils classified into attainment bands within 5-14 level: SSA 2006)

The Reading Attainment Picture for Dundee City Compared with Scotland
(% pupils demonstrating well-established or better skills at the indicated levels)

The national picture for reading attainment results

• 79% of pupils at P3 were well established or better at the level expected of them (Level A) and 48% of P5 pupils were well established or better at Level C (the expected level for P6). However, attainment levels were lower in later stages. In P7, 48% of pupils were well established or better at Level D, and in S2 43% of pupils demonstrated well-established or better reading skills at Level E.
• Considering the levels below those expected for the stage, the proportion of pupils who were well established or better also decreased through the stages, from 84% at P5 Level B to 63% at S2 Level D. On the other hand, many pupils were working at levels above those expected for their stages. For example, 52% of P3 pupils were already well-established or better at Level B and 19% were at Level C. In S2, 18% of pupils demonstrated well established or better reading skills at Level F.
• There appears to be a gender gap in favour of girls in the primary sector, with no consistent picture at S2.

Reading attainment results in Dundee City

• Dundee City is generally performing below the national average in reading across most stages and levels.
• The differences in attainment become large enough to reach statistical significance at P5 Level C, P7 Level C and S2 Level F.
• Girls tended to produce slightly better performances than boys at P3 and P5; at the other stages gender gaps were inconsistent.

Teachers' level judgements: Reading

The charts below compare the reading attainment profiles for the authority and for Scotland, based on submitted teachers' level judgements.

The national picture for teachers' judgements of reading

• In general, teachers judged pupils to be at or around the expected level for their stage. At both P3 and P5, teachers judged at least 85% of pupils to be either at the expected level or the level above. At P7, the judgements are more widely spread but still with just under 80% judged to be at the expected level or the one above. At S2, however, a lower number (65% of pupils) were judged to be at Level E or F and 23% were judged to be at Level D (one below that expected).
• It is also interesting to note that, for pupils assessed at the lower levels by the SSA assessments, teachers tended to have provided slightly higher level judgements. For example, 52% of the P7 pupils estimated at Level C on the basis of SSA assessment results were judged to be at Level D by their classroom teachers.
• There is evidence of a slight but consistent gender gap in favour of girls at all stages in reading.

Teachers' reading judgements in the authority

• Generally, teacher judgements of pupil attainment in reading in Dundee City give much the same picture as nationally.
• The one exception is at S2, where a higher proportion of Dundee City pupils are judged to be at Level F than nationally.
• According to the teachers' judgments girls tended to do slightly better than boys at P3 and P5, with a less consistent pattern at the older stages.

Some questions about reading for Dundee City to consider

• What might explain the relatively large differences in attainment between Dundee City and the nation as a whole in P5 at Level C? Do teachers share a common understanding of standards and expectations in reading at this level?
• Teachers' level judgements in Dundee City indicate higher attainment than the national average for the higher levels in P3, P5, P7 and S2. This is in contrast to the survey results which suggest performance is below the national average across most stages and levels. What programmes and activities do teachers in Dundee City have in place to challenge and support more able readers?
• Are there aspects of the theory and practice of teaching and progression in reading that might be the focus of professional development for teachers?
• What might account for the different pattern of gender performance at P7 - is it due to teachers using strategies that improve boys' performance or girls underperforming? Could approaches that support boys in reading in P7 be shared with teachers in S1 and S2?

Teachers' level judgements: Writing

The charts below compare teachers' judgements nationally and for Dundee City.

The national picture for teachers' judgements of writing

• At P3, 89% of pupils were judged by their teachers to be at or above the expected level
  (Level A) and 46% of P5 pupils were judged to be at or above Level C (the expected level
  for P6).
• Attainment levels are somewhat lower in later stages. Considering the expected levels for the respective stages shows that, in P7, 65% of pupils were judged to be at Level D or above and, in S2, 53% were judged to be writing at Level E or above.
• There was a gender difference at all stages for writing, in favour of girls.

Teachers' writing judgements in Dundee City

• At P3, pupils in Dundee are showing a much wider range of levels than the national average, with a much lower proportion of pupils judged to be at Levels A or B and a much higher proportion judged to be below Level A or at Level C in writing.
• At P5 and P7, the pattern of teacher judgements for pupils writing in Dundee City is very similar to that of the national average.
• By S2, pupils in Dundee City are generally judged to be above the national average, with a higher proportion than average judged to be at Levels E and F and lower proportions judged to be at Levels B, C and D.
• In Dundee City, a gender gap in writing emerged at P3 and P5 only.

Some questions about writing in Dundee City to consider

• What might explain the difference between the patterns of judgements made by teachers in Dundee City and nationally? Why might teachers in the authority have judged pupils to be attaining higher levels than nationally at S2? Why might the same pattern not apply at primary stages?
• Do teachers in primary and secondary schools share an understanding of standards and expectations in writing, and what more could be done to encourage and support local moderation and sharing of standards?
• Nationally, there is a gender difference at all stages, in favour of girls. What programmes and activities have teachers in Dundee City put into place that might explain the lack of gender difference at P7 and S2? If teachers are adopting particular strategies, is this something that could be shared more widely?

Attainment in Numeracy

The charts below illustrate the profiles of numeracy attainment at the various stages for Scotland as a whole and for the authority.

SSA 2006 - Numeracy Attainment Estimates: Scotland
(% pupils classified into attainment bands within 5-14 level: SSA 2006)

SSA 2006 - Numeracy Attainment Estimates: Dundee City
(% pupils classified into attainment bands within 5-14 level: SSA 2006)

The Numeracy Attainment Picture for Dundee City Compared with Scotland
(% pupils demonstrating well-established or better skills at the indicated levels)

The national picture for numeracy attainment results

• 90% of pupils at P3 were well established or better at the level expected of them (Level A). At P5, 81% of pupils were well established or better at Level B (the expected level for P4) and 48% of P5 pupils were well established or better at Level C (the expected level for P6). In P7, 67% of pupils were well established or better at Level D (the expected level for the stage) and in S2 46% of pupils were well-established or better at Level E (the expected level for the stage) in numeracy.
• Considering the levels below those expected for the stage, the proportion of pupils who were well established or better is generally around 80%, although this varied from 85% at
  P7 Level C to 76% at S2 Level D.
• Many pupils were working at levels above those expected for their stages. For example, 33% of P3 pupils were already well-established or better at Level B and 5% at Level C. In S2, 18% of pupils were already well established or better at Level F.
• There was no evidence of any gender gap in numeracy nationally.

Numeracy attainment results in the authority

• Generally Dundee City is performing below the national average in numeracy across most stages and levels.
• While attainment levels at P3 are around the national average at P5 through P7 to S2, attainment drops below the national average. The size of this gap increases through the stages.
• At P7 and S2, Dundee City pupils were performing below the national average at all levels. The differences are large enough to reach statistical significance at P7 Level C and S2 Level F.

Teachers' level judgements: Mathematics

The charts below compare the findings for Scotland and the authority.

National picture for teachers' judgements of mathematics

• In general, teachers judged pupils to be at or around the expected level for their stage.
• At P3, teachers judged 97% or more pupils to be either at the expected level or the level above.
• At P7 the judgements are more widely spread, although 75% of pupils were judged to be at the expected level or above.
• At S2 64% were judged to be at the expected level or above with 23% judged to be at
  Level D.
• There is no evidence of any gender difference in level judgements at any stage.

Teachers' mathematics judgements in the authority

• According to the teachers' judgements, the attainment of primary pupils in Dundee City is broadly in line with the national attainment levels.
• Pupils in S2 are generally judged to have attained a slightly higher level than pupils nationally.
• There were no gender differences in teachers' judgements of pupils' mathematics attainments at any stage in Dundee City.

Some questions about mathematics for Dundee City to consider.

• What might explain the apparent steady decline in numeracy attainment across the stages in Dundee City compared with nationally? Is the decline associated with insufficient pace and challenge to ensure progression through the stages?
• Why do Dundee City teachers judge their S2 pupils' numeracy attainment to be above the national average while the SSA assessment results show Dundee S2 pupils to be below the national average? Do teachers in the authority share a common understanding of standards and expectations in mathematics across stages and sectors, and if not, how could the differences be resolved?

Annex A: Reading attainment results

2006 SSA - National reading attainment estimates
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

2006 SSA - Reading attainment estimates: Dundee City
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

2006 SSA - National reading attainment estimates, by gender
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

2006 SSA - Reading attainment estimates, by gender: Dundee City
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

Annex B: Submitted teachers' level judgements for Reading

2006 SSA - National reading attainment estimates: teachers' judgements
(% pupils judged to be at indicated levels by their teachers ^*)

* The standard error is between 0.1 and 1.1 for non-zero estimates of the "all pupils" group and between 0.1 and 1.9 for each non-zero gender estimate

2006 SSA - Reading attainment estimates: teachers' judgements: Dundee City
(% pupils judged to be at indicated levels by their teachers ^*)

* The standard error is between 0.1 and 1.1 for non-zero estimates of the "all pupils" group and between 0.1 and 1.9 for each non-zero gender estimate

Annex C: Writing attainment results

2006 SSA - National writing attainment estimates: teachers' judgements
(% pupils judged to be at indicated levels by their teachers ^*)

* For 'all pupils' the standard error is between 0.1 and 1.1 for non-zero estimates, and between 0.1 and 1.6 for each non-zero gender estimate.

2006 SSA - Writing attainment estimates: teachers' judgements: Dundee City
(% pupils judged to be at indicated levels by their teachers ^*)

* For 'all pupils' the standard error is between 0.1 and 1.1 for non-zero estimates, and between 0.1 and 1.6 for each non-zero gender estimate.

Annex D: Numeracy attainment results

2006 SSA - National numeracy attainment estimates
(% pupils categorised into each attainment band*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

2006 SSA - Numeracy attainment estimates: Dundee City
(% pupils categorised into each attainment band*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

2006 SSA - National numeracy attainment estimates, by gender
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

2006 SSA - Numeracy attainment estimates, by gender : Dundee City
(% pupils categorised into each attainment band ^*)

* '< 50%' means fewer than 50% of the items in the level-based task answered correctly, 'good start' means between 50% and 64%, 'well-established' is 65% to 79%, and 'very good' is 80% or more. ' SE' is the standard error for the 'well-established or better' estimate.

Annex E: Submitted teachers' level judgements for Mathematics

2006 SSA - National mathematics attainment estimates: teachers' judgement
(% pupils judged to be at indicated levels by their teachers ^*)

* For "all pupils" the standard error is generally between 1.0 and 3.0 for non-zero estimates and between 1.0 and 4.0 for each non-zero gender estimate

2006 SSA - Mathematics attainment estimates: teachers' judgement: Dundee City
(% pupils judged to be at indicated levels by their teachers ^*)

* For "all pupils" the standard error is generally between 1.0 and 3.0 for non-zero estimates and between 1.0 and 4.0 for each non-zero gender estimate

Annex F
Pupil and School Participation Rates

SSA 2006 Pupil and School National Participation Rates

* P5 pupils in schools that were also participating in the PIRLS survey were allocated a single reading task or a social subjects task in place of the usual three reading tasks or two numeracy tests plus a social subjects task
** Pupils contributed to the analysis of numeracy only if they had attempted both numeracy tests that were assigned to them

SSA 2006 Pupil and School Participation Rates: Dundee City

Annex G
Assessing reading, writing and numeracy attainment

Around 34,000 randomly selected pupils throughout Scotland were selected to take part in the 2006 survey, and almost 30,000 of these pupils completed assessments. The intended sample was randomly split between reading and numeracy/social subjects.

'Reading pupils' were each randomly allocated three reading tasks (one task for each of the levels assessed at the relevant stage, drawn from a total of 12 tasks per level). Tasks were presented to pupils in the order of lowest to highest level and breaks were allowed between tasks. The reading tasks took exactly the same form as the tasks that are available to teachers from the 5-14 National Assessment Bank.

'Numeracy/social subjects pupils' were each randomly allocated two comparable mixed-level test booklets and one Social Subjects Enquiry Skills booklet. There were 12 numeracy booklets in total per level and each contained an equal number of test items at each of the three levels assessed at the stage. Within the booklets test items were presented in a randomised order. As the booklets contained tasks at mixed-levels, the booklets used in the survey differed from the single-level units that are available for use by teachers as national assessments. Each Social Subjects booklet presented a single task at a single level.

For reading and numeracy (and Social Subjects Enquiry Skills) pupils were judged as having demonstrated "very good" attainment at a particular level if they answered 80% or more of the test items at that level correctly (that is 80% of the items contained in their task at that level, or included within the pair of numeracy booklets). Pupils who did not correctly answer 80% of the items at a level but did answer at least 65% correctly were deemed to have shown "well-established" skills at that level. Pupils who did not answer 65% of the items at a level correctly but managed at least 50% correct were considered to have made a "good start" at the level.

For writing, a different approach was used. For national reporting, around one-third of the pupils that had been randomly selected for involvement in the reading or numeracy/social subjects assessment were randomly selected for the submission of a piece of extended writing, which was submitted with a teacher's level judgement for the piece of writing, and then moderated at a national event using 5-14 National Criteria.

A total of 3,000 scripts were moderated. Information on moderation is available in section E of Scottish Survey of Achievement: Supporting Evidence: 2006 Scottish Survey of Achievement ( SSA) Social Subjects (Enquiry Skills) and Core Skills (http://www.scotland.gov.uk/Publications/2007/08/15104710/0). Due to low sample sizes in authorities, this information has not been analysed at the authority level.

In addition, all the schools that participated in the survey were invited to submit teachers' estimates of the current levels of each sampled pupil's attainment in writing, reading, mathematics and social subjects (in all relevant forms: social subjects, environmental studies, history, geography and modern studies). Level judgements were submitted for around 75% of the sampled pupils nationally. It is this set of attainment figures that are used to provide the local authority writing attainment results for this report.

Local authority sampling

It was intended in the survey to assess approximately 200 pupils per authority per stage in each of reading and numeracy. The actual numbers assessed varied from authority to authority, depending on school and pupil participation rates, and occasionally the achieved pupil samples in one or other area showed imbalances in terms of the proportions of boys and girls and/or in terms of the proportions of pupils from deprived areas ¹ as compared with the authority as a whole. The statistics on school and pupil participation, nationally and for this authority, are given in Annex F.

This Annex (Annex G) shows the gender and deprivation make-up of the authority's pupil samples. Where a degree of non-representativeness was apparent, this was balanced out through data weighting during analysis, in the same way that authority over-representation in the national samples at each stage was taken into account (see Annex I of the supporting evidence report for further detail). It is important to note that data weighting is not intended to 'iron out' any attainment differences between authorities. It simply adjusts the contributions of the different pupil subgroups so that they reflect their population presence. For example, if girls tended to have higher attainment in reading (which they generally do at the younger stages) and an authority sample contained a higher proportion of girls than it should, given its population proportion, then that authority's overall attainment estimate for reading would be more positive than it should be, had the girls' contribution not been adjusted downwards to reflect the population ratio.

Gender and Deprivation Make-up of Authority Populations and Achieved Samples

Whether a pupil is recorded as being in the 20% most deprived areas in Scotland is determined on the basis of the pupils home data zones, each zone having a ' SIMD rank' indicating its position within a national rank order based on a complex deprivation measure: for full information, see Scottish Index of Multiple Deprivation 2004. Summary Technical Report.

Gender and deprivation imbalances were redressed during attainment estimation, through appropriate data weighting (see section B of the supporting evidence for more information).

It is important to note that the weighting does not correct the results for the effects of deprivation and gender. Any gender or deprivation effects will still be present within the data but the weighting aims to ensure that the results reflect the true gender and deprivation make-ups of the authority rather than the proportions achieved in the sample.

Gender and Deprivation Make-up of Authority Populations and Achieved Samples: Dundee City