WELL? WHAT DO YOU THINK?
CHAPTER TWO. RESEARCH METHODOLOGY AND ANALYSIS
2.1 The research involved face to face interviews with a representative sample of the Scottish population. Random sampling methods were used to select respondents: addresses for the survey were taken from the small user section of the Postcode Address File (PAF). Sampling points were 96 Enumeration Districts, selected from all parts of Scotland (including Highland and Island areas) with a probability proportional to size of the adult population. It was hoped to achieve a sample size of 1,500 adults aged 16 and over.
2.2 The sample size would not give robust data relating to minority ethnic people in Scotland. The 1991 Census showed that there were only two local authority areas in which minority ethnic people comprised more than 2.5% of the population. In order to increase the number of non-white respondents, a booster sample of 100 minority ethnic people was included in the study design. A separate selection of addresses was made from PAF in what were thought to be areas of relatively high concentration minority ethnic population.
2.3 The questionnaire for the survey was designed, initially, by an advisory group. The group included representatives from the Scottish Executive Health Department, the Health Education Board for Scotland, Glasgow University, Edinburgh University, the Public Health Institute Scotland and the National Anti Stigma Campaign. The draft document was further developed by NOP, in discussion with the Scottish Executive.
2.4 The questionnaire collected information on people's general health and lifestyle, mental health and well being, personal experience of mental health problems and discrimination, sources of information about mental health issues, and attitudes towards mental health problems and the people who suffer from them. Emotional well being and energy/fatigue were assessed using questions from the SF-36 (Ware et al., 1994). Vignettes depicting people with symptoms associated with depression, schizophrenia or stress were adapted from scenarios created by Link et al (1999) in their study of public recognition of mental illness. Each vignette was constructed to meet the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, for the disorder in question. The gender of the person in the vignette was randomly varied, and a name was assigned to each vignette subject (Robert or Shona).
2.5 The questionnaire was tested in a small pilot study with 16 respondents in four areas. Locations used in the pilot study were chosen to provide a mixture of inner city, suburban and rural locations including one area with a high concentration of minority ethnic people. The questionnaire was further refined before the main fieldwork stage of the study. (Further details of the questionnaire are provided in relevant sections of the report and the full questionnaire is reproduced in Annex D)
2.6 All fieldwork was conducted via Computer Assisted Personal Interviewing (CAPI), a technique which involves a computerised questionnaire that avoids the need for interviewers to manually check questionnaire routing and leads to both better data quality and rapid provision of the survey outputs. The average interview length was 30 minutes. Fieldwork was carried out between late July and early September 2002. (Information on survey administration, including response rates, is given in Annex B)
2.7 A total of 1381 interviews were conducted, some 120 less than the original target number. The shortfall may be explained by the fact that fieldwork had to be carried out during the summer holiday season (17% of addresses had no contact after 6 or more calls). The tight timescale for the research and the need to report top-line findings in early September meant that there was no opportunity to uniformly reissue non-contacts and 'soft' refusals for further calls and conversions. The main sample shortfalls appear to be among young people, and young men in particular. For more detail on the response rate, please see Annex B.
2.8 There was a relatively high level of refusals: 23% of all addresses. The rate was compared with refusal rates in two other recent surveys in Scotland: the 2001 Scottish Social Attitudes Survey was carried out during the summer season and the achieved sample size was similar; the refusal rate was 28% (Curtice and Petch 2002). However, findings from the most recent Scottish Household Survey ( Scotland's People: Results from the 2001 Scottish Household Survey:www.scotland.gov.uk/library5/society/spv6-04.asp) noted a refusal rate of 13%.
2.9 Where possible, interviewers coded reasons for refusal: of the 23% who refused to take part in the study, some 7% said they were (always) too busy, 6% were not interested in the subject and 3% did not want to talk about mental health issues. Where the refusing household was observed to include at least one older person, the proportion refusing because they did not want to talk about mental health issues was twice as high as among other refusing households. There was no evidence of visible bias in terms of the noted profile (ethnicity, evidence of children in the home etc) of those people who refused to co-operate with the survey but, of course, it is not possible to tell whether there are attitudinal biases in the weighted sample of participants.
2.10 Although there was some resistance to taking part in the survey, three-quarters of the final sample said that they would be willing to take part in future surveys for the Scottish Executive.
2.11 The ethnic minority booster sample generated only 51 interviews. This was mainly because the proportion of eligible households was considerably smaller than had been expected on the basis of 1991 Census data. The general expectation was that there would have been a rise in the numbers of people from ethnic minority groups living in the sampled areas. However, interviewers reported a number of cases in which properties owned by ethnic minority landlords were let to (white) students or young people. The main sample included 19 people from ethnic minorities and, while this group was included as part of the main sample analysis, the interviews were also added to the booster sample to produce a separate cell of data with 70 respondents.
2.12 Computer tables were prepared to a specification agreed with the Scottish Executive and were revised after the initial verbal debrief of the key findings on 4 September. The basic question responses and various derived variables were cross-analysed by a wide variety of cells, described below.
Sex
Age (7 groups - 16/24 through to 75+)
Age interlocked with sex (6 groups - 16/34, 35/54 and 55+)
IPA social class (4 groups - AB, C1, C2, DE)
Ethnic minority respondents
Working status (2 groups)
Income (5 groups - ranging from less than 5,200 per annum to 36,400 or more)
Ease of managing on income (3 groups- easy, manageable and difficult)
Affluence of area (4 groups - most affluent to least affluent, based on 1991 Census data)
Marital/relationship status (2 groups)
Tenure (2 groups - own/buying and renting)
Qualifications (7 groups - ranging from no qualifications through to professional qualifications)
Urban/rural split (3 groups - urban, semi-rural and rural)
Region (5 groups - Central West, Central East, South/Borders, Highlands and Islands and North-east.
The sample for the survey was not large enough to break down findings by health board, so the regional groupings were developed to give some flavour of any broad differences across Scotland. The concentration of population in the central belt meant that any representative sample of this size would have modest numbers of interviews in the more rural areas of Scotland.
Stress levels (4 groups- completely free of stress through to a large amount of stress)
Long-standing limiting conditions (3 groups- limiting condition, non-limiting and none)
Experience of mental health problems (3 groups - a problem of their own, someone close with a problem and no contact at all)
Claimed level of control over things that affect mental health (4 groups - ranging from complete control to a little/none)
Alcohol consumption was not included in the cross-analysis as the levels recorded were quite low and it was not thought that useful analysis could be conducted.
Cluster solutions based on factor analysis of attitude statements (6 groups, see Chapter 7.4 to 7.6)
Stigma scores based on attitude statements (4 groups- from highest to lowest, see Chapter 7.2 to 7.3)
Health and vitality scores based on an excerpt from SF36 (4 groups - from highest to lowest, see Chapter 3)
Vignette used during the interview (6 groups - examples of the symptoms of mental health problems, see Chapter 7)
2.13 The report highlights significant differences within the overall sample for each question asked. Some variables appear regularly as key discriminators (sex, age and the level of stress experienced by the respondent in the last year, for example) while others are rarely significant (for example, whether the respondent lives in an urban or rural location, or in a particular region of Scotland). Throughout the report, where the data is presented in detailed table format, significant figures appear in bold italics.
2.14 The data was also used by NOP's Statistics department in a series of secondary analyses with the intention of identifying links within the data that might not necessarily emerge in standard cross-analysis. These analyses included the production of an index of mental health and vitality (see Chapter 3.10 and 3.11), an attitudinal or 'stigma' scale (see Chapter 7.2 and 7.3) and cluster analysis (see Chapter 7.4 to 7.6). Other statistical techniques were used to assess which would produce the most powerful discriminators within the data. Having examined various approaches, CHAID and Correspondence Analysis techniques were employed.
2.15 CHAID divides a population into two or more categories that have the greatest difference with regard to the dependent variable and then splits each of these groups until no more statistically significant differences are found. CHAID was used on the stigma scoring and also on mental health and vitality scores. Correspondence Analysis produces a graphical representation of the relationship between the row and column elements of a data table and was used on this occasion in examining attitudes to mental health and also in linking the vignettes of mental health symptoms with possible causes. The CHAID and Correspondence Analysis are detailed in Annex F and, where appropriate, findings are cross-referenced in the body of the report.