STATISTICAL SURVEYS IN SCOTLAND - THE SCOPE OF A STRATEGIC APPROACH
This document is also available in PDF format
Purpose
1. To set out some discussions points on issues concerning a more strategic approach to statistical surveys of households or the household population.
Harmonisation
2. There are a range of possibilities:
Sampling
3. Harmonisation would allow more robust estimates of any harmonised questions at smaller levels of geography.
Question content/design
4. Surveys could include core questions on a consistent basis, and further harmonisation wherever similar questions are asked. Even where the data required are the same, different question design/methods can lead to significantly different results. This issue goes beyond harmonisation - there is a possible need to consider the best way of meeting user needs across a range of surveys and rationalising coverage of surveys as a result.
Classifications
5. Harmonisation of classifications could include those of a relatively formal nature such as ethnicity classifications or less formal classifications such the intervals for income bands, age groups etc.
Timing
6. Timing during the year can affect results, particularly in areas such as labour market and weekly income. However some surveys are carried out over a lengthy survey period, primarily for cost and practical reasons.
7. Given several surveys are not annual, some co-ordination of timing coupled with harmonised questions would give more frequent results for core questions
Quality
8. Harmonisation of quality procedures may be more difficult to define and produce less definite benefits. However there may be some merit in a degree of consistency.
Integration
9. Integration may be considered a step beyond harmonisation involving integration of survey management and conduct. It is being considered by ONS for surveys they run. Clearly in the context of UK and Scottish surveys this raises devolution issues. Whilst not attracted to integration across administrations we will want to be aware of any benefits identified and consider how best they might be achieved. Furthermore elements of an integrated approach may offer benefits.
10. Decisions on survey boosts and harmonisation are not necessarily helped by separate management arrangements. The distinction between statistics and quantitative research findings is a grey area, and distinctions within the Scottish Executive's organisational structure are little or no real interest to data users. The lack of harmonisation/integration is potentially damaging to attempts to develop a more co-ordinated approach to meeting analytical needs.
11. The range of possibilities for integrated or partially integrated management/ decision making includes
(a) survey content
(b) survey management and conduct including user interface
(c) priorities for sample boosts
(d) funding - currently spread across CRU, departmental programmes
(e) integration of publishing programmes - this could range from integrated control to editorial influence
Appraisal
12. Harmonisation is not entirely straightforward and may have significant resource implications. Many surveys are likely to have given at least partial consideration to harmonisation. Differences often exist for good reason. There are many important considerations such as survey cost, consistency over time etc.
13. However even partial harmonisation would appear to offer the prospect of significant benefits.
14. Full integration raises more serious difficulties, including whether it is practical and desirable to create a single resource capable of managing the wide range of issues and consultations necessary to allow the surveys to run effectively. However there are again significant potential benefits from some form of integration. These include
- the harmonisation agenda could be pursued more effectively
- value for money could be achieved more easily
- users would find it easier to be involved in more than one specific survey
The role of surveys in meeting user needs
15. An integrated approach to surveys should be driven by the need to meet analytical needs. There are particular issues with providing local level data with the introduction of policies which rely heavily on the need to local data, and so the strategey should address the extent to which survey data can and should be used in providing local information. These include
- The scope to redesign surveys to produce larger samples for small areas, and how should this be done?
- The need to encourage common survey instruments to be used by those carrying out local surveys, to allow merging of local and national data
- Best Practice guidance in the appropriate use of national data for local analysis.
Links with other funders
16. It should be recognised that there is a range of organisations providing resources for surveys in Scotland. A strategy for surveys in Scotland requires to address liaison between all funders, including those external to Government, in particular the research councils. To a certain extent these links are already in existence through the Scottish Executive, but it will be important for any strategic committee to identify effective communication routes.
Capacity and Skills
17. A longer term strategy for surveys, requires to address the issues the capacity and expertise in Scotland for managing and commissioning conducting and using the results of surveys. The committee may address
- Evidence of shortages in any of these areas
- Review the adequacy of current plans for addressing skills gaps and training by Higher Education, Research councils and professional bodies
- Review the effectiveness of networks for spreading good practice and methodology
Rob Wishart
Chief Statistician
Scottish Executive
20 August 2002