CHAPTER SEVEN PROCESSES
Introduction
7.1 This section of the evaluation document considers the processes which were in place for Tartan Week 2007 and is, therefore, not an evaluation of the events associated with Tartan Week, rather the processes and procedures which underlie it.
Pre-tender process
7.2 Following recommendations from the 2006 evaluation, detailed guidance notes for organisations wishing to apply for funding for events at Tartan Week were published by the Scottish Executive. The guidance notes were publicised through:
- The Scottish Executive website
- Editorial in the Scotsman
- Email/other communication with former Tartan Week participants and potential 2007 participants identified by the Scottish Executive
- Word of mouth through those previously involved with Tartan Week both in Scotland and in New York.
7.3 Proposals submitted in response to the guidance notes had to demonstrate that the event/s could deliver on one or more of the objectives and to deliver against the theme "Modern Scotland in 2007". Whilst it could be argued that "Modern Scotland" as a theme was vague, overall it provided focus without being too limiting.
7.4 It is generally accepted by those members of the Scottish Executive team and the event organisers who had also been involved in Tartan Week 2006 that the tender process in 2007 was a significant improvement on 2006. Event organisers, in their discussions with evaluation team members, were able to identify the objectives associated with Tartan Week 2007 in contrast with 2006 when the majority of event organisers were unable to do so.
Submissions
7.5 The format of the submissions in response to the guidance notes was left to each individual application. This made it difficult for the evaluation team (and presumably the Executive) to compare the proposals and may be a contributory factor in the wide variation in the detail and quality of the applications received. One area of considerable weakness among the majority of the applications which were reviewed by the evaluation team was the evaluation and success indicators. No guidance was provided by the Scottish Executive on the type of indicators or measurements, eg SMART targets, which should be included in applications and addressed within evaluations.
7.6 The quality of tenders received by the Scottish Executive was highly variable making it more difficult for the Executive to be able to fully compare the merits of the different applications and to reach a fully informed decision. It was also unclear against which set of criteria the Scottish Executive assessed submissions. Selection criteria should be published by the Scottish Executive alongside the guidance notes.
Informing Successful Applicants
7.7 The guidance note specified that event organisers would be notified on the outcome of their submission by 8th September 2006. It was considered that this would allow successful applicants sufficient time to resource and prepare their event/s. There was, however, a delay of several weeks in informing applicants and the majority only received the go ahead in mid October. One event organiser stated specifically that the delay in decision making regarding funding meant that they lost the support of a potential sponsor for their event.
7.8 Event organisers who were requested to submit joint applications once their individual applications had been reviewed by the Executive were particularly critical about the time it took to give them final approval on funding and the impact this had on their organisation of events.
7.9 A two stage application process with potential applicants being asked to declare an intention to tender by a specific date with the full submission required at some point thereafter should be considered by the Scottish Executive.
Communications with the Scottish Executive
7.10 In the lead up to Tartan Week, the Scottish Executive hosted meetings to which event organisers were invited. Event organisers who had attended such meetings as part of their involvement with Tartan Week 2006 considered that the meetings held for Tartan Week 2007 were more focussed and had a clearer structure than previously. These meetings were particularly beneficial to those event organisers who had had no experience of staging events at Tartan Week and/or were not part of a larger organisation and lacked specific expertise, such as marketing.
7.11 Unlike last year, event organisers were fully aware who to contact at the Scottish Executive about issues such as marketing and funding. It was also noted that Scottish Executive officials were accessible and approachable. This year, the Scottish Executive officials made themselves available at the Scottish Village each morning during Tartan Week and all event organisers had a list of contact numbers for the Scottish Executive team. The findings from the 2007 evaluation suggest that the lines of communication between event organisers and the Scottish Executive had been improved upon from last year.
Marketing/ PR
7.12 A total of £70,000 was spent on marketing/ PR support for Tartan Week by the Scottish Executive. The Scottish Executive and VisitScotland's joint funding of the What's On Guide, the www.tartanweekny.com website and light poles around Grand Central Station continued in 2007. In addition, the Scottish Executive funded a my space site (myspace/tartanweek.com) and tartanweekblogspot.com aimed at attracting younger audiences to Tartan Week. The blogspot featured the adventures of Morag (a tartan VW van) as it made its way around New York promoting Tartan Week. Event organisers accompanied Morag on her journey and helped to promote Scotland by talking to those who approached the van and handing out balloons/What's On guides.
7.13 MWW, the New York based PR agency, provided PR support for Tartan Week and its event organisers primarily in the week leading up to Tartan Week and Tartan Week itself. Several some event organisers sought PR support from other organisations/individuals for their events during Tartan Week.
Event organisation
7.14 Event organisers were satisfied that their events were well attended and that they had gone to plan. They were particularly pleased with the venues in which their events were located and with the assistance and support offered by venue managers. In some instances, this support extended to marketing the forthcoming Tartan Week event to members, eg New York Historical Society. The evaluation team noted that the selected venues were well located and easy to reach using public transport.
7.15 A few event organisers spent time in New York prior to Tartan Week in order to build contacts and to gain a greater understanding of their potential audience. Whilst this was beneficial to these event organisers, not all had the time or resources to enable them to do so. Those event organisers who were unable to spend time in New York pre-Tartan Week were more likely, therefore, to require support and guidance from the Scottish Executive.
7.16 The evaluation has highlighted that the assistance, support and knowledge offered by New York based organisations and individuals proved extremely beneficial in terms of marketing/ PR, selection of venue and sharing of relevant knowledge. Some event organisers suggested that a Tartan Week Co-Ordinator with a good knowledge of Tartan Week would be beneficial. This individual would provide advice on venues, marketing, etc and may be the main source of information/assistance for event organisers once they had been told they would be receiving funding from the Scottish Executive.
7.17 The Scottish Book Trust, for whom Tartan Week 2007 was its second Tartan Week, benefited considerably from the knowledge gained from attending the event in 2006. Others such as Mike Maran Productions benefited from working closely with Scots and/or Americans based in New York who provided valuable insight and practical assistance. The Scottish Executive may wish to consider how it can involve the support offered by global Scots and affinity Scots earlier in the process for event organisers.
Post-event evaluation
7.18 The event organisers were required by the Scottish Executive to produce evaluations and accounts relating to their events by 30 April 2007. Very few evaluations were received by the Scottish Executive by this deadline, and feedback from the organisers suggests that this was too short a timescale (two weeks after the end of Tartan Week) to provide a realistic evaluation.
7.19 The evaluations received mirror the submissions in that the information is not uniform and the quality of information received varies widely across the event organisers.
Key conclusions
7.20 Whilst the pre-tender process was much improved for Tartan Week 2007, assessment of the guidance notes has highlighted a number of areas which could be considered weak and where there is a need for revision. Suggested refinement to the guidance notes appears in the recommendations section of this report and will ensure that the submissions, in response to the guidance notes, are structured in a way which can be more meaningfully assessed. This would include the requirement for SMART targets and clearer guidance to event organisers regarding the level of detailed information required.
7.21 The move to inform applicants about the outcome of their submission in early September was welcomed by event organisers.
7.22 The guidance given to applicants on the evaluation of Tartan Week events should be increased to ensure that the event organisers could provide a meaningful measurement of their activities. In addition to clear criteria being shown in the guidance notes, the evaluation team for Tartan Week 2008 should be responsible for attending one meeting with Tartan Week event organisers pre-event to provide guidance and advice about the requirements of the evaluation.
7.23 The lines of communication between event organisers and the Scottish Executive were much improved on from 2006. Meetings were particularly beneficial to those attending Tartan Week for the first time.