Scottish Government

National Review of Assessment: Identification of the Learning and Support Needs of Young People

Chapter 2: National Survey of Further Education Colleges

Structure of this chapter

This chapter describes the survey method used by the research team and presents the detailed findings, structured by key stages: at entry; on course; and prior to leaving.

Procedure

A letter outlining the background, aims and proposed methodology of the research was sent to the principals of all Scotland's further education (FE) colleges. College principals were invited to nominate key members of staff to whom a questionnaire should be addressed.

The questionnaire developed by the researchers was modified following advice from advisory group members and three critical readers with significant experience of special programmes or extended learning support provision in further education ( see Appendix 1). The pilot testers took on average one hour to complete the questionnaire.

The questionnaire was sent in May 2001 to 47 colleges, and responses were received from 31 (66%). Colleges could choose whether to complete the questionnaire from an overall college perspective, or from the perspectives of special programmes (SP) and extended learning support (ELS). The numbers of responses representing these different designations were 18, 12 and 14 respectively. Twelve colleges completed questionnaires for both SP and ELS perspectives. Respondents had clearly taken a good deal of time and trouble in completing questionnaires: this is at least partly a reflection of the importance of assessment as an issue for FE staff.

The question-by-question analysis below represents a compilation of responses from all the questionnaires returned, though where differences in perception resulted from the different perspectives these are indicated. Quotations from the questionnaires are included to illustrate common responses.

The process of assessment on entry to an FE programme

Questions 1 and 2 invited respondents to indicate the kind of information that was available in relation to students' "hard" and "soft" skills.

Table 1: Information about "hard" skills available on entry

Table 2: Information about "soft" skills available on entry

The extent to which information was available varied between colleges. Details of "hard" skills might be more readily inferred from Standard Grade results or other assessments. Information was more likely to be available where college lecturers had good links with school guidance or learning support teachers, for example, from a future needs assessment (FNA) or guidance report. Some respondents said that the quality of information varied between schools: '…those prepared to have on-going liaison with the college provide more beneficial information in a mutually appropriate format'. One respondent complained about schools providing very generalised responses to specific questions about competence in literacy and numeracy. More information seemed to be available where students had attended school-college link courses or where there had been involvement of social work or other welfare agencies. Information was less likely to be available for Christmas leavers, adult returners and overseas students. In general, more information was available for students entering special programmes than for mainstream courses. Some respondents referred to students being reluctant to disclose details of disabilities or other support requirements, even when prompted to do so in application forms.

Interviews and references were cited as being potentially valuable sources of information about certain "soft" skills. Some colleges had developed detailed questionnaires for schools and parents. One college referred to collaboration with the careers service to develop a "transition form" ⁴.

'Where used, this is extremely effective in identifying skills and needs. Not widely used yet, however, and not all students arrive at college via this route.'

Examples of "hard" skills noted in the "other" category included Standard Grade passes, and information about learning support requirements was mentioned in relation to both "hard" and "soft" skills.

Questions 3 and 4 specifically asked about the sources of information on "hard" and "soft" skills available to colleges. The results are shown in the table below.

Table 3: Sources of information on" hard" and "soft" skills

The responses show that colleges typically gathered information on both "hard" and "soft" skills through liaison with schools and other agencies, and through interviews with students, rather than through the use of screening tests.

'Much information on "soft" skills is exchanged in the assessment stages. School and college staff are aware that often this can be a subjective area.'

Most colleges used information from a combination of sources to build up a picture of the applicant. Students entering special programmes were typically assessed by college tutors during transitional/link programmes, and information might also come from social work Adult Resource Centres. Students who attended college link courses should already have had a Personal Learning and Support Plan (PLSP).

The "other" category included a range of agencies, such as social work, voluntary organisations and psychological services. Some respondents pointed out that the quality of information was higher where good relationships had developed between college staff and colleagues in schools and other external agencies.

Questions 5 and 6 invited responses about the use of the National Record of Achievement (NRA) in the entry process. Responses from colleges indicated considerable variation in the extent to which the NRA (or Progress File) was used as part of the entry procedure, and also in perceptions of its value. The variation appeared mainly to be attributable to local practice rather than to be a function of different perceptions by ELS and SP perspectives. The following quotations from respondents in different colleges characterise the varying experiences of the use of records of achievement during negotiations for entry to courses.

'Students tend to bring these to interview for course but also to the interview with ELS co-ordinator to establish needs and draw up PLSP.'

'In my experience very patchily. Many applicants do not bring NRA/Progress File with them. (It seems many have not produced them at school.) Those which are brought are examined politely with positive comments made about achievements.'

Three other important issues emerged from the responses. Firstly, the value of the NRA/Progress File might depend on whether students had been encouraged to use them by schools.

'Many schools do not use NRA/Progress File. The production of an NRA is never forthcoming in the entry process.'

'Where available is discussed at interview. Uneven use in local schools.'

Secondly, the NRA/Progress File might not be available to the college at the time of making an offer of a place.

'The NRA is unlikely to be available to us until after the student has been offered a place. In the last academic year it was less useful since they tended to be incomplete - lack of SQA results. A very few students bring NRAs to our 3-day entry/selection links.'

'Not usually considered and not always available - students rarely provide it. Also many students in FE do not come directly from school.'

Thirdly, perceptions of the value of the NRA/Progress File varied:

'NRA would be discussed with Course Tutor in planning individual programmes. Information on NRA would be conveyed to learning support staff by:

• notification by Course Tutor
• interview with the student
• occasionally copy of NRA
• internal referral forms

This is being used to allow consideration of achievement.'

'It is not used as it is held in low esteem by the students and often contains inaccuracies.'

'Few students bring RoAs ⁵ with them to interview. Dramatic downturn in the use of RoAs generally within college.'

Typically colleges had very little or no communication directly with schools specifically in relation to NRA/Progress File. Communication was limited to transfer of the file if it did not come with the student, or, in some cases, the record of achievement might be discussed at a Future Needs Assessment meeting, or in discussion with guidance or learning support staff.

Question 7 invited respondents to indicate the typical sources of information for identifying learning and support needs. The results are shown in the table below.

Table 4: sources of information for identifying learning and support needs

The most common sources of information were schools. However, there was variation in the perceived quality of contact with schools.

'There is great difficulty in accessing information from the range of over 20 feeder schools in a way which is consistent.'

'Often difficult to obtain Record of Needs from schools due to confidentiality - communication barrier.'

Colleges might also lack information that would help to plan support arrangements for young people where there had been a gap between leaving school and applying for a college place. Other sources of information used extensively by colleges included social workers and other carers, specialist careers advisers, therapists, parents and the students themselves.

'We have a network of contacts with external agencies.'

Question 8 asked how satisfied respondents were with the identification of skills/needs at the entry stage and Question 9 invited suggestions for improving the procedure.

Table 5: satisfaction with the process of identification of skills/needs

The results suggested overall a fair degree of satisfaction with the identification of "hard" skills but a more mixed view of colleges' success in identifying "soft" skills at the entry stage. Comments from colleges underlined a variation in the effectiveness of links with schools; where links were good the quality of information transferred was high and the converse was also true. Suggestions for improvement can be summarised as follows:

• better links between schools and colleges, e.g. beginning planning at an earlier stage;
• extending the practice of liaison adopted for future needs assessment of recorded pupils to include pupils with support needs who aim to access mainstream courses;
• improving the quality of information through better co-ordination of professionals, i.e. learning support and guidance teachers, educational psychologists, careers advisers and college staff;
• colleges could do more to explain to school staff and young people the advantages of having fuller information.

'We have already identified the need to be more proactive. We'll contact schools earlier and do the pre-entry core skills assessment.'

'More consistent approach across the whole of the education sector across our catchment area.'

'Records of needs are usually closed when a young person leaves school. It would be useful if it were kept up to date and passed on to FE colleges.'

'The skills we focus on in college are often very different from a school's priorities. It would be very useful if the school curriculum (in a pupil's final year) incorporated some "lifeskills" assessments rather than providing which chapter of the reading scheme used in the school was reached.'

Question 10 requested information about commercially produced assessment instruments/tools used at the entry stage. The Adult Basic Skills Agency assessments were in use in one third of the sample, and their use was under consideration in two further colleges. The table below shows tests used by two or more colleges. In addition, one college used the Okley College Literacy, Numeracy and General Profile and another used a range of psychological tests with individuals, including Raven's Progressive Matrices, PLATO (integrated learning system for distance learning), Schonnel Reading and Spelling tests, and the Neale Analysis of Reading.

Table 6: Commercially produced tests used on entry to college courses

However, most colleges did not make use of commercial tests, generally favouring locally prepared diagnostic assessments. Nineteen colleges (61%) provided information about their approaches to initial assessment. These included college-devised tests of basic skills, use of SQA national assessment bank items, interviewing and scrutiny of school reports.

'Students are assessed using a holistic approach and by observation in as many different ways as possible, e.g. in class, out in the community and in social areas.'

'We tend not to use commercially produced assessment tools at entry. This is mainly because the process can be intimidating for vulnerable students. We try to identify need with the student not for the student so feel that a more holistic approach is better than specified tests.'

'In consultation with individual departments Learning Support devise tailored assessments for individuals identified as having specific needs - core skills gaps - reading difficulties - students with English as a second language. Results are discussed with student (applicant) and course team leader to assess how much support will be required during the course or whether in fact time spent with adult basic education would be useful with a conditional place offered the following year. Occasionally a potential student will be offered a conditional place and will attend workshops with Learning Support prior to taking up the offer…'

Few colleges engaged in blanket screening of all entrants, though this issue appeared to be on the agenda in at least some.

'The screening is being debated at the moment and we are looking for diagnostic assessment in relation to core skills.'

More typical was the practice of general screening of all students already identified as having support needs.

'… College is in the process of moving towards a single Personal Learning Support Plan for all special programmes and extended learning support. Work has been done in this area by a short-life working group co-ordinated through the SFEU ⁶.'

The process of assessment during an FE programme

Question 12 asked which learning and support needs were identified while students are on course. The results shown in the table below reflect the rather complex situation of on-course guidance and support for learning. The support infrastructure was more developed in some colleges than in others.

'These issues are the core of guidance activities, and form the basis of regular interviews as part of the guidance process.'

Information was more likely to be available for "hard" skills and, as one respondent pointed out, this would increasingly be the case as a result of the assessment of core skills within Higher Still courses. Generally there was on-going monitoring of students who had personal learning and support plans but often colleges were less confident about the robustness of their arrangements for identifying the needs of other students, particularly in relation to health and welfare.

'No standardised multi-purpose cross-college screening programme in place.'

'Many difficulties only come to light when the student is not making satisfactory progress or has high absenteeism rates.'

Table 7: Learning and support needs identified "on course".

Question 13 asked how students' on-course learning and support needs were identified. The responses indicated that interviews, using PLSPs and scrutiny of application forms - in that order - were the most commonly used methods of gaining information about students' on course support needs. Other sources of information included work experience reports, referral by a tutor, student self-referral, course team meetings and information passed on by carers/social workers.

'Everyone in the college at every stage of a student's course can refer a student for help. It could be the librarian, a course tutor, student services or the student can drop in to the learning skills centre and ask for help.'

Amongst the examples of good practice identified by respondents were effective induction programmes, clear referral procedures, actively encouraging students to seek help and good signposting of services. However, these approaches were resource-intensive and might be difficult to maintain.

'This is becoming harder to maintain in college as increasingly tutor hours for personal guidance are being cut and at the same time the demand for "paper based administration" is increasing.'

Table 8: How "on course" learning and support needs were identified

Question 14 asked respondents to indicate how satisfied they were with the identification of learning and support needs at the on-course stage, and question 15 invited suggestions for improving procedures. The responses suggested a reasonable degree of satisfaction but there were issues to be addressed.

'The problem is not necessarily the identification of learning and support needs, but the difficulties of providing an individual programme to address those needs.'

'Only when students ask for or want help. People slip through the net and then leave college.'

Table 9: Satisfaction with identification of learning and support needs on course

Increased staffing levels to take account of the time commitment for individual assessment and support provision, was the most frequently mentioned suggestion for improvement. Also mentioned were better review procedures, improved marketing of services to both students and teaching staff, and staff development.

Questions 16 and 17 asked about the use of commercially produced and in-house devised instruments at the on-course stage. At this stage, information typically came from assessment results in SQA units. For students on special programmes Access 2 or 3 were commonly used, and City and Guilds Skillpower and the ASDAN Foundation Training Award were mentioned as sources of information about competence in basic skills, in each case by one college. The in-house developed instruments tended to be action plans and reviews; one college provided information about college-wide student profiling.

The process of assessment prior to leaving an FE programme

Twenty-seven respondents (73%) said that students' skills and needs profiles are updated prior to leaving college programmes. Of the 11 responses to question 18 from the perspective of special programmes, only one indicated that this did not happen. Practice varied between colleges. Typically students who had attended special programmes or who had a PLSP would have a review report, and core skill profiles would become available for all students undertaking Higher Still Access level units. However, this was not the practice in all colleges.

'Each student has a leaver's certificate indicating the SQA units achieved and a skills profile built up from progress reports.'

'We do not complete exit profiles as such but students have evidence of achievement prior to exit and have access to careers advice.'

The skills/needs highlighted in exit programmes (Question 19) are shown in the table below.

Table 10: Skills/needs highlighted in exit profiles/reports

Question 20 asked respondents to indicate who typically contributed to exit profiles or reports. Lecturers with whom students had most contact were clearly in a good position to provide the most accurate information about their progress and continuing needs. Other important contributors were the students themselves, placement supervisors, parents and carers.

Table 11: Contributors to exit reports

Responses to question 21 indicated a desire for improvement in colleges' exit procedures.

'Fairly satisfied, with reservations. There is no "transition form" document at this stage that can accompany students to the next stage (e.g. employment) that will identify "soft" skills and distance travelled.'

'There is less activity here, but this is a far lower priority than identifying needs at the start of the course.'

Table 12: Satisfaction with the identification of skills/needs on exit

Suggestions for improvement included:

• standardising procedures for all students, with a profile that can accompany students from school to FE and then on to HE and employment;
• encouraging students to become more centrally involved in building a personal record of achievement in both "hard" and "soft" skills;
• better use of external specialists (e.g. HE special needs advisers, disablement resettlement advisers, careers advisers).

Few respondents said that they made use of commercial assessment tools at the exit stage. Two respondents referred to the use of careers software. Typical in-house assessment tools included reports of pre-exit guidance interviews, reviews of PLSP, use of Progress File, and placement reports. Clearly practice was quite varied amongst colleges.

'Use of these tools is rare. Students are more likely to come to conclusions about their job path by a combination of SQA units. The world of work and Finding and Keeping a Job and skills profiles build up over the 2 years of the course.'

'Information from the PLSP, pre-exit guidance, NRA and action plans are passed on to future training provider by student.'

Further Information

Respondents were invited to provide further information that might be helpful to the researchers. The comments received can be summarised as follows:

• requests for exchange of information (e.g. about assessment products, literature on assessment and good practice ideas) between colleges;
• provision of staff development, specifically on the practice of assessing support needs;
• extending the role of the Access Centres (as recommended in the Beattie Report).

'For the purpose of reliability and quality I think it would be beneficial for staff and students to have a standardised format to follow for pre-entry, on-course and pre-exit reports. This could be established with collaboration with careers, FE tutors, students etc.'

'Ability to liaise and discuss with other colleges as to the success and failure of the instruments of assessment being used would be profitable.'

'Publication of resources used by centres in order that best material can be identified for possible purchase.'

Respondents also mentioned the SFEU working party which had been preparing standardised documentation for PLSPs, and a CD-ROM, available from SCET (Learning and Teaching Scotland) for use in initial assessment of adult returners.

Key Issue 1: The interface between FE colleges and other organisations with respect to assessment

Colleges appeared to be linking well with schools and other related agencies in relation to assessment, particularly where young people's needs were recorded, or where they had been on college link courses prior to entry. However, evidence about the learning and support needs of school leavers whose needs had not been recorded was less easy to access. Two solutions were suggested for this. The first was that good practice in liaison for Future Needs Assessments should be extended to those with non-recorded needs. The second was that there should be standardised procedures for all students, with a profile to accompany students from school to FE and then into HE and/or the labour market. In one area a transition form for FE entrants was already being piloted.

In many ways this second suggestion sounds very like an extended Progress File with support needs and targets attached. The evidence from FE colleges showed that the current use of the Progress File was very patchy, even at pre-entry stage. Its use as part of ongoing guidance and review was limited and there was little knowledge of its continuing use by students in subsequent employment or training.

There was clearly considerable variation in practice in whether a Record of Need was kept open when a young person went to college.

While there were many positive aspects in the identification of learning and support needs for those students coming from a school situation, links at the pre-entry stage appeared less effective with employers, training organisations and voluntary bodies.

Arrangements with respect to pre-exit assessments and transfer of assessment information out of the college appeared either unclear or non-existent except through the provision of references from the college to the next provider of education, training or employment support. In general there appeared to be limited awareness of, and arrangements for, pre-exit profiles which would be owned by the student.

Colleges were positive about PLSPs, and the focus of development of good practice seemed to be on their use. The evidence of those who had used PLSPs was that they were useful to students and staff alike in ensuring that individual learning and support needs were addressed while attending college.

Key Issue 1: Recommendations

1. Consideration should be given to developing an FNA or case conference approach for those school leavers entering FE whose needs have been recognised but not recorded. Further development of transition documents which take account of both pre-entry and pre-exit assessment information might be helpful. Relevant to this is the current review by the Scottish Executive Education Department of all transition documents associated with Future Needs Assessment.
2. There should be greater clarity and consistency about whether a Record of Need remains open when a young person goes to college, and if not, how the assessment information and recommended strategies should be used subsequently.
3. Potential ways of linking Progress File and PLSP documents and procedures should be examined in order to provide evidence of learning and support needs that might help ex-students describe and explain their needs and their progress to organisations providing education, training and employment to them in the future.

Key Issue 2: Hard and soft skills

There appeared to be a generally accepted understanding of "hard" skills, but much less consistency in the definition, collection and use of information about young people's "soft" skills. On the whole, FE respondents were clear how to handle educational, literacy and numeracy assessments, as might be expected. Colleges had had many years of experience in supporting the needs of individuals with a range of learning difficulties, and some had considerable specialist knowledge in assessing and supporting the needs of individuals with physical or sensory impairment. However, the needs of individuals with emotional and behavioural difficulties, or with addiction or housing issues, or with motivational problems required a different approach to assessment, and many college respondents recognised a need to develop strategies and the use of tools to measure "soft" skills in these client groups. Many college staff, coming from a professional background in education and training, were able to devise their own assessment tools for literacy, numeracy, educational progress and study skills. In fact it was more common for colleges to use their own assessment instruments for "hard" skills than to use commercially produced measures. Higher Still developments had encouraged and required the assessment of core skills across the whole student cohort, and this included the skills of working with others and problem solving. However, "soft" skills were not so easy to measure and this was recognised as an area for development for the institution and for individual members of staff.

A related issue for colleges, which also arose in discussions with Careers Service Companies, was whether screening strategies to identify the learning and support needs of students should be applied across the whole cohort, or whether actual assessment should be focused only on those whose needs had already been identified. There were clearly resource issues in applying this type of assessment across the student cohort. Several colleges noted as an area for development that students should be more centrally involved in building a personal profile of achievement in both "hard" and "soft" skills and that this was a valuable activity for the whole student group.

Key Issue 2: Recommendations

1. There is the need for staff development in FE focused on the practice of assessing learning and support needs, particularly in the development of "soft" skills. Colleges should consider whether commercially produced tools to assist the assessment and development of "soft" skills might play a part in this process. Funding available for Access Centres following the Beattie report may assist.
2. FE colleges need to consider the extent to which assessment of "hard" and "soft" skills should ideally be applied across the student cohort, and the practicalities of this.
3. Students need to be more centrally involved in building a personal profile of achievement in both "hard" and "soft" skills.
4. Like other professional groups involved in this research, FE staff sought greater opportunities for exchange of assessment good practice with colleagues in other colleges and in other sectors than were currently possible. Opportunities for exchange of good practice should be encouraged and supported.

Key Issue 3: Students whose needs were more difficult to identify

The assessment of the learning and support needs of particular client groups seeking to enter colleges had proved to be difficult. In some cases this was because, despite the theoretical existence of a system for the provision of pre-entry information on needs, particular individuals had slipped through the net. Examples of this included school leavers who left at the end of the December term, and those who had truanted (often this group overlapped).

Even where information of other kinds was available, there were often significant gaps. The assessment of need for young people with health, emotional and behavioural issues was often lacking, and as a result there were gaps in the support provision for these young people.

Many individuals came to FE colleges without any prior information on their needs, some of which were either immediately or quickly evident to college staff. Client groups in this situation included overseas students and the general adult population (for whom, in many cases, any assessment evidence would often be seriously out of date). For these groups, too, "soft" skills and needs associated with health, emotional and behavioural issues were the most critical.

However, it is important to realise that there will inevitably be a number of students with learning and support needs for whom there is no prior information.

Key Issue 3: Recommendations

1. In collaboration with other organisations and agencies in a locality, FE colleges should review the gaps in assessment information and strategies for particular client groups or categories of need.