Scottish Government

EVALUATION OF THE ZERO TOLERANCE "RESPECT" PILOT PROJECT

SECTION FOUR: IMPLEMENTATION OF THE PROJECT

4.1 Following exploration of the preparation for the project, the implementation was examined, and the majority of views of implementation were positive and supportive. Alongside this general perception, however, a number of suggestions for the future were made in relation to the use of the materials and the practical arrangements which were in place. These provide useful guidance in terms of "what worked".

THE LAUNCH EVENTS

4.2 The project was launched simultaneously in Glasgow and Edinburgh on 31 ^st January 2001, with both launch events having presentations by politicians (Jack McConnell, the then Education Minister in Glasgow, and Jackie Baillie, the then Social Justice Minister in Edinburgh) and Zero Tolerance staff, and input from pupils (in the form of drama or dance).

4.3 It had been intended that the launches would generate publicity and raise awareness of the project and the Steering Group members and Zero Tolerance staff were largely happy with the launches. Positive aspects of these were identified as being the involvement of the young people, the involvement of high profile Scottish Ministers and the awareness raising undertaken in the schools.

4.4 Almost half (44%) of staff, however, were not happy with the launches, however, and only a third of the older participants enjoyed the experience. Although all of the Glasgow primary pupils enjoyed it, many Edinburgh primary pupils did not and there was an identifiable difference between perceptions of the Glasgow and Edinburgh events, with many more of those who attended in Glasgow having enjoyed the launch. Difficulties identified related to the lack of publicity achieved, the "dryness" of the Edinburgh launch and the need for "more colour" and "razamatazz" and a perception that the launch had been anti-men (and had particularly upset one of the Broughton staff). There were problems in both settings with the exclusion of the primary school pupils in the discussions with visitors after the events.

PREPARATORY AND SUPPORTING MATERIALS

4.5 In terms of the actual delivery, however, all of the staff were broadly happy that the materials were useful, with the suggestion that they all gelled together as a coherent message.

The CD Rom and "Z" Screensaver

4.6 Many of the young people had not seen the CD Rom, but perceptions amongst those who had seen it were largely very positive, and all of those interviewed and many of those completing questionnaires considered it to be effective (with young women more likely than young men to express a positive view). All but one (95%) of the staff delivering the project who saw the CD Rom were happy with it.

4.7 Positive comments included that it was user friendly, "quite stylish", interactive, "fun" and professional. Staff generally believed that the young people had particularly enjoyed the quiz, the radio talk-in and the timeline and two thirds of the young people identified the quiz as being the part which they had enjoyed most. All but three of the young people interviewed identified something that they had learned from the CD Rom, including facts about domestic abuse and historical information.

4.8 Less positive comments (made by a small number) were that the timeline was a "bit boring", that the graphics could have been better (in one case) and that the music did not work (in another case). Three suggested that the information focused upon how boys treated girls, and two that the questions were too basic. It was also suggested that it was quite short, that it focused only on heterosexual relationships and that it could not be used as part of the lesson. One staff member pointed to an incorrect reference to a support organisation. It was also noted, however, that the CD Rom should be used alongside the other materials and was not "an end in itself".

4.9 A specific issue arose with the fact that the content of the CD Rom meant that it was unable to be used in the two Roman Catholic schools in Glasgow (although the pack itself was approved). All of the material was passed to the Catholic Education Commission for further comment once the project was underway, but approval could not be provided during the implementation phase.

4.10 There was a very low level of use and awareness of the screensaver.

Posters and bus sides

4.11 There was a high level of awareness of the posters and bus sides by the end of the project. Over 90% of all older participants and most of the primary pupils had seen posters (although there remained 38% in St Bernard's primary school who had not). All of the Steering Group members and around three quarters of staff and young people had seen the bus sides.

4.12 Almost all of the young people of all ages considered the posters to be effective (with a higher proportion of young women than young men who responded in this way). The staff were also largely happy with these (85%). The poster stating that "boys must always be tough, girls just need to be pretty, says who?" was mentioned most frequently as the one which children and young people thought was the best. All of the Steering Group respondents and almost all (92%) of the staff were happy with the bus sides and the majority of older participants and around 80% of primary pupils considered them to be good.

4.13 Positive aspects of the posters and bus sides were that they were:

• Plain and striking, with a clear message.
• "In your face" and powerful.
• Inclusive, involving males and females and people from black and minority ethnic groups.
• Promoting a message of respect, equality and avoidance of stereotyping.
• A useful means of awareness-raising (community-wide in the case of the bus sides).

4.14 Concerns with posters and bus sides were that:

• Views may have moved on, and they may not have offered sufficient challenge or may reinforce existing views if taken out of context.
• Some pupils may find the messages difficult to understand, particularly if they were unaware of the project.
• The posters were not sufficiently eye-catching or were too "glamorous" for the target group.
• The posters arrived very late.
• The bus sides were more easily seen on red buses than others, and were only on one side of a bus, with the bus sometimes moving away before there was time to absorb the message.

THE SESSION MATERIALS AND THEIR DELIVERY

4.15 The main content of the project focused on the sessions themselves and the overall view from most of the staff in all settings was that the session materials had worked well, and they had enjoyed delivering them. Some staff stated that they had "enjoyed every session" and 91% of older participants interviewed and 69% of those responding in the self-completion questionnaires had enjoyed taking part. Young women were markedly more likely to have enjoyed taking part than young men, but the overall response was positive. 91% of primary pupils completing questionnaires also enjoyed the project.

4.16 Young people suggested that the work had been fun, and, most commonly, that they had enjoyed having an opportunity to discuss the issues and that it was different from their usual work. The highest proportion of older and younger participants believed that the project had been better than they had expected and few were able to identify any parts of the work which they did not enjoy.

4.17 A very small number of staff had a less positive overall experience of delivery, and one of the groups of young men in Glasgow dropped out of the project after the third session, but these were the exception. Only one member of staff did not enjoy taking part, and although others pointed out that different groups reacted in different ways, and they were sometimes more and less comfortable, most enjoyed delivering the project overall.The content of the sessions

4.18 The highest number of older participants, when asked to identify the session which they enjoyed best stated "most" or "all of it" and when asked to identify which session they enjoyed least said "none of them" (and one added "and that's amazing for school"). Only around a fifth of young people (19%) were able to identify any sessions which they did not enjoy.

4.19 The establishment of rights and responsibilities at the start was seen as important in all settings, although it was noted that neither the first nor the last sessions had been particularly memorable to the older participants, and this was seen as important in the context of the overall impact of the project.

4.20 For both older and younger participants, the practical sessions worked particularly well and the "It's Not Fair" (poster making) exercise and graffiti wall, as well as the co-operative jigsaws (in the case of primary pupils), were identified as particularly popular. The highest number of staff and many young people identified the poster making as having worked particularly well, demonstrating discrimination and allowing young people to talk about their own experiences. The graffiti wall was also one of the young people's favourite sessions. It was suggested that these findings could help to inform the future development of the first and last sessions, with a need for a focus on more creative and practical tasks from the outset.

4.21 Views of the inclusion of role-playing and brainstorming exercises were more mixed, and some youth groups and schools found role-play more difficult, either because participants were considered to be too young, embarrassed by the content or the materials sometimes went over their heads. A number of older participants, however, remembered the role-playing as one of the methods which they had enjoyed.

4.22 The variations which were used generally worked well (with evidence of the youth groups and some schools using a range of different variations to different sessions). Examples included turning the role-play into a "Jerry Springer" show, changing scenarios, using videos (e.g. Bob the Builder as an example of gender issues), drama based sessions, introducing discussions around other subjects (e.g. body language) and, in one case, making shields on the subject of respect. Staff also used some of the variations which were suggested in the materials.

4.23 Most of the sessions were reported as having achieved their objectives (wholly or partly) and many generated wide-ranging discussion of the issues raised in the project. With the older participants, for example, the discussions covered issues such as the identification of situations that could trigger violence, discussions of how the group related to and respected each other, racial tensions, feelings resulting from not being listened to, being different and treating people differently and issues relating to virginity and peer pressure, "sexually abusive insults", male sexual or violent abuse of power, feeling unequal, general stereotypical images and pressures on young people. Primary pupils also discussed a range of issues, including equality, racism and asylum seeking and issues affecting boys and girls, as well as their own experiences of these.

Issues raised and the further development of materials

4.24 Many participants made detailed suggestions about the ways in which the individual sessions could be improved, and these have been presented in the main report.

4.25 The only sessions with which particular difficulties arose, however, were the graffiti wall with the older participants and the name-calling exercise with the younger groups.

4.26 With the graffiti wall, for example:

"many pupils succumbed to the temptation to insult others in their class on the wall ……. when I discussed this, all pupils involved claimed it was a laugh." (secondary school teacher)

There was also a more fundamental issue raised in relation to the use of a continuum of "seriousness" of bullying with the wall.

4.27 The teachers in both primary schools identified name-calling as a very difficult session for them, for a number of reasons. Staff were not sure that the children had learned anything from the session and were also unsure about how far to allow children to go in identifying words which they would never normally be allowed to use in the school. There was a concern that the discussion in this session had the potential to generate some complaints from parents.

4.28 In terms of more general issues raised, some staff also suggested that they had found some of the views which the young people expressed to be difficult (with a number reporting, for example, homophobia and racism) and they had not always been sure of whether, or how, to challenge these views (and had not always done so). This was a particular issue in the session which focused on violence and abuse, with examples of young people laughing in inappropriate circumstances or retaining inaccurate perceptions of violence against women after the session. It was suggested that, in the future, there was a need for clarity for staff in terms of the types of issues which may arise and the best means of addressing these.

The resources

4.29 In terms of the actual resources provided, all of the Steering Group members and almost all (94%) of the staff were happy with the content of the materials, and the same proportion believed that the materials could be used in other schools and youth groups. Almost all of the young people liked the resource cards which accompanied the sessions. General comments included that the material looked appealing, was of a high quality, durable, flexible, contained a wide variety of materials and was well laid out and easy to use. Young people said that the cards had stimulated thinking and discussion, provided facts, helped them to understand things, had "stories and interesting advice", as well as being colourful and easy to read and able to be taken home.

4.30 The only less positive comments made overall were that the materials were perhaps less "revolutionary" than had been anticipated and "slightly predictable". On a more practical level, staff suggested that the lack of flip-charts, Blu-Tack, scissors, coloured pens and other material had caused problems, and that some of the sessions had also required them to make or bring their own materials (e.g. video clips, worksheets, ice-breakers, graffiti walls) which was "just another thing to do", with a lack of time and opportunity to prepare this. Staff in the Catholic schools also had to check carefully through any material used (such as magazines) for adverts or articles which were unsuitable, or might prompt complaints from parents. It was also noted that some of the materials had been given away to pupils, and a new pack would be required for each new group.

4.31 In terms of suggestions made, staff pointed to the need for the material to be flexible and for the staff to improvise with the project (both to meet the needs of young people in the groups, and to respond to other demands which arose, particularly in the schools). It was also noted that it was important to ensure that the material did not become too directive, and that the explanatory material might benefit from more explicit links to "emotional intelligence", as well as being suitable for young people with a range of skills and levels of academic achievement and relevant (in terms of the scenarios) to young people's experiences. It was suggested that there should have been less written material on some of the resource cards (and a smaller number of scenarios from which to choose). One youth worker suggested that it needed "funked up" a bit, and a small number suggested that there should be greater clarity of when to use variations.

4.32 It was also noted that a wider mix of materials and media could be included, and the need to explore young people's views was also stressed. For the young people, it was suggested that a memento (for example, a keyring) as well as stickers, badges and "wee extras" would be useful.

Practical issues

4.33 Although, overall, the materials worked in all of the settings, there were also a number of practical issues identified in relation to "what worked" which help to provide guidance for the future implementation of the project.

Staffing

4.34 It was suggested that experienced guidance staff were the most appropriate to deliver this project in the secondary schools, although a small number of secondary school staff respondents suggested that other staff could also become involved, where they had an interest in the issues.

4.35 Some settings involved others in the delivery of the material (for example, a student, one of the older participants and an Education Resource Worker in the Glasgow schools), and this was positive (although the importance of their understanding the issues discussed was noted). It was also suggested that it would have been useful, in some of the sessions, for outside organisations to have delivered a specific session (for example, a gay and lesbian organisation or Women's Aid).

4.36 All of the staff were happy with those who had delivered the project in their own organisation, although two of the youth workers had concerns about this material being delivered by school staff and two of the youth group members stated that they were glad that they had not undertaken this with a teacher from their school. 96% of the primary pupils and all of the young people in the secondary schools and youth groups stated that they had been happy to talk with the member of staff who took their group and that, generally, the staff were considered easy to talk to.

4.37 It was also seen to be important for the staff involved to be sensitive and experienced (as there may be "a minefield of issues" and they needed to be "able to think on their feet"). It was stressed that the staff should never be forced to deliver the material and that no-one with "extreme views" should be involved. Within these parameters, it was suggested that the project could involve a wider range of staff in the schools and could successfully involve students and other staff in delivery, with appropriate training.

Selection of groups

4.38 Three quarters of the staff were happy with the way in which selection of the groups had taken place. There was a clear preference amongst the majority of young people to work in groups of friends (with almost three quarters of secondary and two thirds of primary respondents expressing a preference for this), but 93% of young people overall were happy with the combination in their group, whether or not it was a friendship group. It was noted that there were benefits and drawbacks for staff of working with unknown groups in schools, but the main requirement was for staff to discuss and agree the means of identification of groups at the start. In youth groups, it was generally considered easier to work with groups which already existed than to convene new groups. It was also noted that 6VT provided refreshments and reimbursed bus fares, which had enabled and encouraged young people to participate.

Composition of the groups by gender

4.39 More than three quarters (77%) of secondary pupils expressed a preference for a mixed-sex group, in order to hear different points of view. Slightly over half of those in the youth groups, however, expressed a preference for a single-sex group (pointing to difficulties in talking in mixed groups), whereas the majority of the primary pupils (57%) believed that it did not matter whether the groups were boys only, girls only or mixed. Whatever the composition, however, all of the young people interviewed (both male and female) were happy with their groups in terms of the gender mix.

4.40 Around three quarters of the staff (76%) were satisfied with all aspects of the composition of the groups. A number of staff suggested that the material worked well in a mixed group, and that the groups could be split into single-sex groups if required. A number suggested that it was important to have a good balance of boys and girls, and noted that the quieter young women generally participated most fully in small groups.

4.41 Some specific issues were also raised in relation to difficulties with young men in both age groups, whether in a mixed or single-sex group. A member of secondary school staff who reported a particularly poor experience of delivery of the project suggested that:

"I knew the boys hated it, although they may have got more from it, but I hated teaching it",

and a further female staff member also pointed to some antagonism from young men. One of the primary school classes also had problems where the boys had started "mucking about".

Age of participants

4.42 The primary school staff were clear that P6 and P7 pupils were an appropriate age group for the use of these materials, as they were beginning to stereotype, but were still sufficiently young to change their views.

4.43 In secondary schools and youth groups, the involvement of young people aged 14-16 was also seen as appropriate, but it was suggested that the material (in a youth group setting) could also be relevant to older participants. It was seen to be less appropriate for younger people, however, and it was suggested that the boys involved in one Glasgow group (which had not continued) had been slightly too young.

4.44 Many staff pointed to the need for a comprehensive programme which would begin in primary school and continue in the early years of secondary (S1-3) as well as in S4 (by which time their attitudes were seen to be fairly clearly formed).

The size of groups

4.45 Although there was a wide variation in the sizes of groups involved (from 2 young people in one case to 19-20 in others, and often more than 20 in the primary schools), it was found that 74% of the staff were happy with the size of the groups. Amongst the young people, 89% of primary school children felt that the group sizes were "about right" and 96% of young people in secondary schools believed that their group sizes were "about right".

4.46 Groups of 12-15 were seen to be most appropriate in secondary schools and 6-8 in a youth group. In primary schools, it was suggested that groups smaller than whole classes might be preferable, but were unlikely to be possible.

Length of sessions

4.47 Where aspects of sessions were less successful, many staff in both primary and secondary schools suggested that this was because "there was never enough time to do them", with the result that some had to miss parts of the project or to move very quickly through some of the activities. As one suggested:

"the temptation also if you are running out of time is that you tell the correct answer, rather than it being student-led".

4.48 There was also seen to be some inconsistency in the denseness of the content of the sessions, and where a session involved substantial reading, it was suggested in primary and secondary schools that the time allowed was sometimes insufficient. It was noted that a lot of the sessions had a "natural break" and could have been done in two shorter sessions.

4.49 Additional work (such as the completion of the evaluation questionnaire) also reduced the time available and the size of classes contributed to the overrunning of the sessions in primary schools. Primary school and youth group staff had some flexibility, however, in delivering the sessions over a longer period, but secondary staff suggested that there should be more scope to continue their work in future lessons, particularly where this related to vital issues (such as the session with older participants relating to violence against women, where this was central to the project but experienced time constraints).

Timing

4.50 The timing of the work was a major area of concern and many staff identified some problems with this (even where they were generally happy). This was a particularly problematic issue for the secondary schools, where the project coincided with preparation by S4 for exams and with course choices, causing stress for staff and pupils alike. In St Paul's, there was an additional issue which related to the small amount of notice which staff were given prior to participation and to the project coinciding with the lead up to a move of school building. Youth group staff suggested that better weather in the spring could also have deterred people attending in the evenings, and the Easter break was disruptive. It was generally agreed that the winter months were the most appropriate time for the delivery of the project.

4.51 The timing of the "Respect" sessions within the day was also highlighted as important, and both secondary schools agreed that the morning groups worked better than afternoon groups, which were seen to be "much harder work".

The space used

4.52 The layout and setting were also found to be important to the delivery of the project, and classrooms were not always ideal for the work, either because they were too small for the purpose, did not have the appropriate atmosphere, were not sufficiently private, were not available regularly or additional space could not be provided. The space available to youth groups was more appropriate, and the groups with their own accommodation had comfortable rooms and informal spaces. The youth groups in primary schools used the staff rooms and other areas in the school, allowing staff to treat the young people like adults and to have an informal and relaxed discussion.

4.53 Despite the differences in experience, and the perception that the space was not always ideal, most of the staff (79%) and almost all of the young people in all of the settings (93%) were generally happy with the space used. The staff adapted their delivery to suit the space available, and although the rooms were "not always the biggest and not the best" no major problems were reported.

Provision of support

4.54 It was always recognised that this project may raise personal issues for some of the participants, and it had been intended from the start that this should be provided primarily through existing mechanisms in each of the settings (i.e. the guidance staff in the schools, the primary teachers and the youth workers) as well as through referral, where appropriate, to other organisations. No time was built in specifically for the provision of support.

4.55 Staff also mentioned that young people had been given access to helpline numbers and more than half (57%) of the primary school pupils interviewed and 70% of older participants stated that they had been given information about where they could get help.

4.56 Almost half of the staff (44%) were aware of young people who had required support during the project, and issues had arisen in all but two of the settings, with staff aware of young people who had experienced sexual abuse, domestic abuse, other issues in their home circumstances, problems with self-esteem, bullying and other experiences which made them "vulnerable".

4.57 Staff described using their judgement in relation to the content of sessions where young people faced personal issues, "keeping an eye on" individual young people and checking with specific young people that they were coping at different stages in the project. In one case a member of staff contacted Zero Tolerance, through which specialist professional support was provided.

4.58 It was also recognised that staff themselves may require personal support, and most believed that they had access to additional advice from Zero Tolerance staff, as required. In Glasgow, the Guidance Adviser was also available for support, and staff had access to support from each other. This was generally through "ad hoc" arrangements, with only youth groups setting aside time specifically for discussion of the work before and after the sessions. There was little contact between Zero Tolerance staff and those delivering the project during implementation, and some secondary school staff stated that this would have been useful. (The general role adopted by Zero Tolerance, however, was seen as appropriate by the Scottish Executive, who emphasised the need for ownership of the process by the schools.)

4.59 Only 6 staff (mostly youth workers) were aware of any colleague who had required support, but the session relating to violence against women did raise personal issues for two of the staff, who had had direct experience of domestic abuse. A total of 13 staff (38%) stated that they would have found additional support useful (particularly in the secondary schools), in the form of regular meetings or "a regular spot to talk about experiences".

The evaluation

4.60 Staff expressed some concerns about the time involved in the evaluation process, and had neither been aware of the level of input required nor had they anticipated the time required for the questionnaires, completion of session reports and interviews. Almost all of the issues raised related to practical arrangements for the evaluation with the only issues raised in relation to the content of materials being a concern (from two staff) that some of the young people may have had some difficulty understanding the self-completion questionnaires.

4.61 All of these findings will help to inform the design or amendment of the materials and arrangements for this type of work in the future.