5 Methods
5.1 An Audit
The method chosen for establishing the snapshot of Scotland's digital media services in the tourist, sporting and cultural sectors was that of a targeted audit utilising an online questionnaire supplemented with telephone interviews.
The audit had as its audience the tourist, sporting and cultural organisations in Scotland. These were largely public sector organisations as they will be the organisations being targeted by any nascent digital media service strategy developed by the Scottish Executive. The specific organisations to be approached to take part in the audit were agreed with the Scottish Executive at the start of the audit. These consultees were sent a personalised headed letter from the Scottish Executive, signed by Mr. Stuart MacDonald. This was to give authority to the audit and hopefully increase the participation in the audit. Not all prospective consultees were expected to agree to be a part of the audit although every effort would be made to ensure their participation.
5.2 An Online Questionnaire
The favoured method for acquiring the majority of information was an online questionnaire. This was to be accessible via the WWW through using a common browser such as Microsoft Internet Explorer or Mozilla Firefox. This was to improve the efficiency and convenience of the data collection primarily for the consultee, but, being in an electronic format would help analysis also.
The online questionnaire itself was broken down into several sections, each dealing with a different aspect of DMS. Even though the nature of the audit was largely that of asking open-ended questions, it was felt that we should still try and capture some information of a categorical nature (for the sake of clarity and ease of analysis). We achieved this by initially asking questions in a 'yes' / 'no' / 'don't know' style, and then, if the consultee had answered categorically whether or not the question's premise applied in their case, they could tell us more about it in an open-ended way using a text box. Using this methodology, we could get a quantitative summarisation of consultees' responses to our questions as well as acquiring further details the consultee thought warranted through their writing. All of the percentages and charts in this report originate from the categorical answers in the online questionnaire.
The online questions consisted of 43 questions, two of which were a question about names of consultees and their organisations, split into five sections. These were subdivided into further subsections which contained the questions. We do not present the online questionnaire as an appendix as it is replicated verbatim in the body of the 'analytical findings' section, for which it provides the overall structure.
The online questionnaire was live and online for a period of one month. It was accessible via a URL (www.simulacra.com/scotexec ) via the usage of a standard browser. Each contact of the consulted organisation was given a login code (their e-mail address) and a password (a random selection of six letters and numbers, some of which had to be upper case) for them to access the online questionnaire. The online questionnaire was implemented on the open source content management system ( CMS) platform openharmonise 1 . The data, once input via the online forms, were stored in a database supported by the CMS. Out of the around 65 organisations contacted, 42 took part in the audit (represented by 58 people).
5.3 The Interviews
5.3.1 Reason for interviews
Even though the online questionnaire had text boxes to allow for consultees to expand more on the categorical answer already given, it was felt that this would not provide sufficient focus on particular issues that consultees wanted to speak about. That is, each organisation would have certain idiosyncrasies and motivations that could only be dealt with and explored dynamically though an interview (either phone or face to face), be those aspects of their DMS or aspects of DMS in general, including aspects of any strategy developed from this audit.
5.3.2 Scoping the interviews
The interviews were held over the same period as the online questionnaire. It was aimed to interview around a third of the total number of approximately 60 consultees. 24 potential interviewees were initially selected, covering all the main areas of activity in the culture, tourist and sporting sectors in Scotland, as well as a cross-section of organisation types and sizes. The initial selection was adjusted in discussion with the Scottish Executive and invitations to interview were sent by email on Monday 26th May, following confirmation by each of the target organisations that they were willing to participate in the audit. It was felt that by contacting more organisations than were required, it would be more feasible to interview the target number of 20.
5.3.3 Setting up the interviews
The initial response to the invitation to be interviewed was immediate in the case of some organisations (the first interview was undertaken the day that invitations were sent out). However, it became apparent that, predictably, the interviews were not a high priority for many organisations and a substantial amount of effort was subsequently expended in chasing potential participants and encouraging them to find time to be interviewed. Without exception all participants were helpful, constructive and engaged when actually interviewed. The main hurdle was identifying a time when they could be contacted. In some cases, interview times had to be rescheduled more than once; others agreed times but could not be contacted at the agreed time, and on occasion, were unable to be contacted to reschedule. Despite the extension of the deadline for participation, extreme difficulty in contacting some participants continued to be experienced. Ultimately, sixteen interviews were undertaken. Two further respondents responded too late to allow inclusion in the audit even within the rescheduled deadline. Particular issues in establishing contact with organisation have included:
- The timing of the project during a popular holiday period.
- The timing of the project during exam and marking periods (for educational organisations).
- Uncertainty within some organisations as to who should participate.
- Suggestions that initial contact emails were lost in spam filters.
5.3.4 Interview question framework
The interviews were designed to complement the online questionnaire by:
- Expanding on some areas of questioning which could only be pursued in a limited fashion within a questionnaire.
- Tailoring areas of questioning to reflect the type of organisation being consulted.
- Enabling a range of 'soft' information and knowledge to be gathered, for example inferences, political issues and motivations for actions.
- Allowing sensitive and complex information and knowledge to be gathered in a medium more likely to encourage interviewees to contribute than in a formal written submission.
- Providing background information which provided a context for current activities and future plans.
A set of questions was therefore developed which combined basic introductory and 'warm-up' questions and 'leading' questions which would provide opportunities for more in-depth information to be shared, and which could be developed within the interview as appropriate.
Some of the questions were not pursued in all cases: for example, questions about digital business models were not always relevant to smaller organisations where one person held a number of responsibilities including ICT, and where ICT was felt to form part of the core function of the organisation.
The interview question framework is provided in Appendix 1.
5.3.5 Conducting the interviews
As indicated above, some difficulty was experienced in scheduling interviews with many of the participants. However, once contact with participants was made, interviewees were asked to set aside around thirty minutes at a specifically agreed time. Interviews were conducted by telephone and notes were taken contemporaneously by the interviewer. Once engaged in discussion, participants were very communicative and eager to have a constructive discussion at a professional level. In most cases, interviewees were happy to speak for 45 minutes, with many discussions taking up to an hour. This was usually due to the participants' desire to share a full range of initiatives with the interviewer as well as to discuss specific issues in greater depth.
5.3.6 Interviewees
Each organisation approached was invited to select its interviewee; some invitees took some time to decide who should be involved. An initial concern was that organisations might hand the responsibility to staff members who were not in a position to take a broad view of the organisations and its ICT activities. In the event, however, this was not the case, with more than half the interviewees being senior managers or CEOs and the remaining having specific responsibilities either for ICT or new media within the organisation, or for project or planning activities relevant to ICT.
5.3.7 Confidentiality and attribution
As with the online questionnaire, participants were told that their individual comments would only be attributable (either by inference or formal attribution) with their explicit permission. This was to encourage a full and frank discussion with the interviewer. On occasion the interviewer was asked not to document the conversation while sensitive contextual information was shared. This approach was felt to have enhanced the willingness of respondents to cooperate and to share information. Although specific references to organisations have not always been made therefore, the analysis of information and creation of an overall picture has benefited from the incorporation and reflection of issues which otherwise would not have been included in the audit.
5.4 The Time-Line
The interviews were held over the same period as the online questionnaire. There was an agreed project plan.