Scottish Government

4 Methodology

4.1 Project organisation

4.1.1 Organisation

The overall management of this project was undertaken by Scott Wilson Scotland Ltd. The design and analysis of the biodiversity elements of the project were largely undertaken by the Game and Wildlife Conservation Trust and Norsk Regnesentral, the Norwegian Computing Centre. The landscape survey design and analysis was undertaken by Scott Wilson, whilst the archaeological analysis was carried out by CFA Archaeology Ltd. The stakeholder survey of farmers was conducted by the Royal Agricultural College.

The project team attended regular steering group meetings, and other meetings as required, with the Scottish Government.

Seasonal field surveyors were contracted to carry out the various surveys on the farms, under the supervision and management of Scott Wilson.

4.1.2 Quality assurance

Field surveyors were given training at the start of each field season before the first phase of farm visits, and again half-way through (before the second phase of repeat visits), both to familiarise them with the survey methods required and to further their bird and plant identification skills as individually required. Surveyors were then paired in a complementary fashion in accordance with their abilities and worked together for the initial weeks of each field season, so that they could also learn from each other.

During the first two years the field coordinator re-visited a sample of farms to validate field data. In subsequent years, the field coordinator took a more direct approach by physically following every surveyor through the various surveys on numerous occasions throughout the field season.

Throughout the project an annual report was produced which was subject to scrutiny by the Scottish Government's Project Steering Group, and any relevant comments were acted upon to hone the future approach. In addition, the project manager regularly reported to the Chair of the Steering Group and these reports were latterly circulated to the rest of the Group.

During the course of the project a number of recommendations for future projects of this kind, and for future research, were produced. These can be found in Appendix 8.

4.2 Stakeholder survey

This element of the study was conducted in the first year of the project (2004). It investigated the views of farmers themselves, using questionnaires mailed to samples of farmers of RSS, CPS, OAS and non-scheme farms. The aim was to explore and analyse the characteristics of the farms concerned; reasons for joining/not joining schemes; decision-making processes; views on the environmental impacts of schemes; and opinions of scheme implementation, land management impacts and scheme effectiveness. This survey was undertaken by the Royal Agricultural College and the original report is available at http://www.scotland.gov.uk/Publications/2007/12/18103831/0.

4.2.1 Scheme participants

A postal survey of agri-environment participants of the Scottish agri-environment schemes (hereinafter simply referred to as 'participants') was undertaken in April/May 2004. The specific schemes covered were the Rural Stewardship Scheme ( RSS), Countryside Premium Scheme ( CPS) and Organic Aid Scheme ( OAS).

A questionnaire was designed to optimise the opportunity to collect the relevant data from respondents. The first part outlines the general background and demographics of the holding including farm size, farm type, enterprises etc. This was supported by the final section which addressed questions of the business itself including tenure status and employment, and more personal information on the respondent including age, retirement plans, education and income. The second and main part of the questionnaire was focused upon the scheme agreement and participation. It addresses the reasoning, views and decisions of participants of each of the three agri-environment schemes.

The questionnaires for RSS and CPS were identical, with only a scheme name difference within the covering letter. The questionnaire and covering letter for RSS is included in Appendix 2 The OAS questionnaire is broadly equivalent, with some minor adaptations to reflect the scheme's characteristics. It is included in Appendix 3.

The number of participating farmers, after removing any farmers participating in more than one scheme, was 2463. This broke down as 1180 RSS, 952 CPS and 331 OAS. A minimum target of 250 total respondents was required, and assuming a response rate of 20%, the participant numbers were halved and then surveyed. The numbers surveyed and the respondents are detailed in Table 4.1.

Table 4.1 Participants surveyed and numbers of respondents

The total number of respondents was 486. The average response rate for each of the scheme participants was just under 40%. This response rate was considered very good and although well over the 250 required for analysis, the data from all responses was used in subsequent analysis.

4.2.2 Non-participants

A postal survey of non-participants ( i.e. farmers who had not joined any agri-environment scheme) was undertaken in July/August 2004. The questionnaire and covering letter were broadly equivalent to those used for the participants, with some adaptations to reflect non-participation in schemes. It is included in full in Appendix

The total number of farmers in Scotland not participating in agri-environment schemes in 2004 was 13,700. From this total a weighted sample of 2500 was provided by Scottish Government, reflecting proportional numbers from each of three regions, namely:

• South - Ayr, Dumfries, Galashiels, Hamilton
• East - Grampian, Perth
• West/North - Inverness, Lairg, Oban, Thurso

This approach retained a degree of parity with other survey elements of the project. The weighted samples were selected by SGRPID⁴. The selected samples were screened against a list of RSS participants entered in 2003, and checked for replicates between samples 1 and 2. Any farm that this screening identified was replaced.

As with the survey of participants, a higher response rate was considered achievable and, assuming a response rate of 20%, the initial sample was halved. The numbers surveyed and the respondents are detailed in Table 4.2. Unlike the participant survey a target of 250 respondents was not reached with the first half of the survey, and subsequently a further 500 questionnaires were sent out which obtained a total of 353 responses. The data from all these responses was used in subsequent analysis.

Table 4.2 Non-Participants surveyed and numbers of respondents

4.3 Sampling strategy for field surveys

It is necessary for any process of data collection and analysis to establish the sampling strategy: how the data should be gathered and the optimal amount of data required for analysis. The strategy here was based on the biodiversity aspects of the project, which were the most significant in terms of survey time allocation and number of different types of survey, and would also be affected by factors such as geographical location, weather and time of year. These factors were less critical for the relatively quick landscape and archaeological surveys, which would be based on fixed-point photography. The intention was to undertake landscape surveys on all surveyed farms, along with rapid archaeological surveys as required (on scheme farms where archaeological features were managed and on non-scheme farms if archaeological features were present).

4.3.1 General design

For all three schemes the same study design was adopted which was central to the way all the data were collected. Large spatial and temporal variation in biodiversity can be expected in any nationwide agricultural monitoring scheme. Therefore a strategy was adopted by which scheme and non-scheme farms were paired together, such that the paired farms were as close and as similar (in terms of type of farm and size) as was reasonably possible (although ultimately not all farms could be paired in this way, as explained below). This design ensured that any variation arising from geographical influences and broadly-defined farm management (whether a farm was arable or dairy, etc.) would be similar for both scheme and non-scheme farms, so as not to introduce any systematic bias.

Furthermore, to reduce the influence of temporal stochasticity that might be caused by, for example, weather or farming operations the paired farms were visited on the same day, or occasionally as near to this as was possible. Seasonal variation between the south and north of Scotland was reduced by surveying southern farms before those further north, as far as was logistically possible. Surveys began each year in April/May and ended in mid-July. Surveys which would be affected by the season where timed accordingly. Vegetation sampling took place on the second farm visits each season when plants would be easier to identify (with the exception of moorlands, scrub and woodland which were surveyed during the first visits of 2006-2008 to reduce workload on the second visits). Invertebrate surveys were also conducted on the second visits, when invertebrates would be more active. Bird surveys were undertaken on both the first and second visits (April/May and May/June/July respectively) to each farm to pick up the widest range of birds.

4.3.2 Power calculation

The number of pairs of farms for each scheme type ( RSS, CPS and OAS) necessary to be confident in the findings of this study was chosen by making a power calculation. The power of a statistical test is the probability that it will be able to detect a true difference between two or more samples and the accepted standard is 80%. This will depend on the following.

• The size of the actual difference of interest between the samples collected: if the difference, for example, in the number of birds actually present on two groups of farms is small it will obviously be harder to detect than if the difference is relatively large.
• The significance criterion used: all statistical tests of this type provide the probability that the result found was obtained purely by chance. It is down to the observer to decide at what point this probability becomes of interest and the result is 'significant'. This is when it is deemed that the result is so unlikely to have happened by chance alone that something more must be happening. The convention is that when a result occurs with a probability of 1 in 20 (5%) or less it is statistically significant and unlikely to have happened by chance alone. However, this cut-off point can be adjusted to make it more or less likely to detect a significant difference, but this is not usually done because it increases the likelihood of falsely accepting a result as significant.
• The sensitivity of the data: the degree to which the data collected actually reflects the situation in the population at large that is being sampled is also crucial. This will be influenced by the applicability and accuracy of the methods used, and also on the number of times the population is sampled ( i.e. the sample size). The greater the sample size the more likely it is that the information gained from the sample will accurately reflect that in the population as a whole.

It follows from the above that the power of a statistical test can be improved by increasing the sample size employed. Large sample sizes, however, have implications for the resources available for any study, so it is important to know how many samples are needed for statistical robustness so as to avoid wasting precious resources pursuing too many samples that will not provide any additional information.

No information was available prior to the study on the likely variation in the various biodiversity measures between farms or over time (which will influence the sensitivity of the data collected, see the third bullet point above). Therefore the power calculation was based on the assumption that at the end of monitoring it would be possible to make a simple decision about whether or not a scheme farm had improved more than its partner non-scheme farm, taking all measures into consideration. It was decided that the minimum proportion of "successes" ( i.e. scheme farms that had improved more than their pair) that should be detectable as statistically significant was 60%. This is a conservative estimate, being just over half (50% 'successes' would be expected simply by chance). The calculation showed that 173 pairs of farms for each scheme would be necessary. This too was a conservative estimate, because it was anticipated that a more powerful comparison than simply better/worse would be possible when the data were collected.

4.3.3 The sampling approach

A robust means of investigating the impact of a change in farm management on biodiversity is to carry out sampling on both the scheme and partner non-scheme farms before the changes are made to the scheme farms, so that the starting conditions are known and can be controlled for. So for those farms just entering a scheme in 2004 or 2005, the approach was to survey them in those years along with their non-scheme partners, and then survey them again three years later in 2007 or 2008 respectively, when it was hoped the effect of the scheme would be detectable. This was only possible for RSS and OAS farms, as these schemes were still open to new applicants in the initial years of the project. Unfortunately not many farmers applied to join the OAS in either 2004 or 2005, and since it was decided that the sample was insufficient for robust analysis on its own, an additional sample of established OAS farms were visited in one year only, in 2006, in the same manner as the CPS (see next paragraph). A single visit ( i.e. surveys during one year only) allows a description of the differences (if any) between the OAS and non-scheme farms at that point in time, but does not provide any information on the starting conditions when participants joined the scheme, nor on how the biodiversity measures may have changed over time, thus making interpretation of any detected effects less complete. However, by carrying out an analysis on the sample of farms visited once and those visited twice it allows for two independent approaches to the same issue: if the results of the two concur then the confidence in those results would be slightly greater than for either analysis on its own, although the sample sizes were small for both.

For CPS farms it was also not possible to use the approach of surveying the farms twice three years apart, because the CPS had closed to new applicants in 2000, and all existing CPS farms had therefore already been in the scheme for a number of years. Additionally, there was not a sixth year available for re-visits three years later, the project being limited to five years. Therefore CPS farms (and partner non-scheme farms) were surveyed in one year only, the majority in 2006.

The three year period between survey years for the RSS and some OAS farms was the longest that was possible within the terms of the five-year study. It was thought that this would be sufficient to allow real changes in biodiversity to take place. Bird populations (particularly those of small birds) are known to be capable of halving or doubling from year-to-year (Newton 1998) and so could, in theory, respond to appropriate changes in the short-term. For example, several species of birds increased over the same period at Loddington in Leicestershire in response to management changes, particularly those species that were previously declining which increased by more than 100% (Stoate & Szczur 2001). It might be expected that some invertebrates may be able to respond even quicker given their faster generation times. Certainly, Woodcock et al. (2005) found that beetles on arable farms responded to the creation of novel grass margins in the first year. It seems likely that certain aspects of the vegetation on farms might take longer to respond to management changes than three years, for example 'species-rich grasslands' ( e.g. Walker et al. 2004). However, if suitable conditions are created it is possible to detect a response amongst the vegetation if the appropriate seeds are present in the seedbank, particularly so for 'simple' prescriptions such as conservation headlands (Sotherton 1991).

If the schemes were having some effect, then it would be expected to find change over time on the scheme farms relative to their non-scheme partners. This pairing and revisiting of the same farms was desirable as it was known that the scheme farms were likely to be dissimilar to the non-scheme farms even before the scheme had started because of the nature of the selection procedure, which targeted farms that were relatively rich in a variety of habitats and wildlife (see Section 3).

4.3.4 Actual farms surveyed

Reduced access to farms meant that the target of 173 pairs of farms for each scheme was not reached. In total over the five years of the project, 331 scheme farms were surveyed and 240 non-scheme farms. 158 RSS farms were surveyed in 2004/05 and re-surveyed in 2007/08, and of these 88 were paired and 70 were unpaired ( i.e. did not have a non-scheme partner). For OAS the problem was exacerbated because, as mentioned above, there were not enough farms entering the scheme at the start of the study. Therefore as many OAS farms as possible were surveyed in 2004/05 and again in 2007/08 (22 farms, of which 15 were paired and 7 were unpaired). Owing to this small number of OAS farms, an additional 29 OAS farms (of which 22 were paired and 7 were unpaired) that had been in the scheme for some years were surveyed in one year only (2006), in the same manner as the CPS farms. 122 CPS farms (of which 105 were paired and 17 were unpaired) were surveyed in one year only (as explained in section 4.3.3), the majority in 2006. These figures are summarised in Table 4.3 below.

Table 4.3 Numbers of RSS, CPS and OAS farms surveyed. The 'paired' row is for farms which had a surveyed non-scheme partner farm; farms in the 'unpaired' row did not have a non-scheme partner. ' OAS 1-visit' represents OAS farms surveyed in one year only like the CPS, and ' OAS 2-visit' those OAS farms surveyed in two separate years like the RSS.

The ability of the monitoring to detect change was therefore not as high as was originally intended, but given the conservative nature of the power calculation it was still considered adequate, at least for RSS and CPS. The very small number of OAS farms meant that the effect of the scheme would have to be large to be detectable.

4.3.5 Evaluation of the sampling strategy

A classic Before-After-Control-Impact ( BACI) ( e.g. Kleijn & Sutherland 2003) approach to the RSS and some of the OAS monitoring was adopted. That is to say, baseline data were recorded from both control or unchanged farms (non-scheme farms) and manipulated (scheme) farms before any changes in management practices took place; then those same farms were revisited sometime thereafter (in this case, three years later) and the sampling repeated. Also, the scheme and non-scheme farm pairs were matched as closely as possible (in terms of farm type, size and geographic proximity) to eliminate some possible confounding factors. This is far preferable to the simpler 'snapshot' approach used for the CPS and some of the OAS farms where scheme farms had been in their schemes for some time prior to the start of the study. The latter approach does not allow for the control of starting conditions and does not permit the measuring of change over time, although it will permit an accurate assessment of any differences at that point in time.

There was no other viable alternative to the approach taken for the CPS and single-visit OAS monitoring given the constraint of scheme start-times. A BACI strategy would have increased costs considerably because of the extra round of sampling and may not have yielded any great dividend as the greatest change in biodiversity on the farms is likely to have taken place during the first few years of joining a scheme.

The approach for the RSS and re-visited OAS monitoring here is largely in line with the accepted best standard for this kind of project (Klein & Sutherland 2003): the only improvement that could have been made was to ensure the starting conditions on scheme and non-scheme farms were more alike. The best way to have done this would have been to start with a sample of farmers who had all just been accepted for entry into the RSS. Then, using a similar paired set-up to that employed here in order to ensure appropriate geographical spread of both the scheme and non-scheme farms, the farms would have been randomly allocated to either RSS or non-scheme status. This would ensure that all the farms would have been similar to start with (probably relatively rich in biodiversity). This approach was not pursued in this case because it would have involved paying farmers allocated to the non-scheme group to do nothing: this was deemed an unacceptable use of tax-payers money.

4.4 Field survey methodologies

4.4.1 Data collection

Each farm in each survey season was visited on two separate days by surveyors, once in April/May, and again in May/June/July. As far as was possible logistically, farms in the south of Scotland were surveyed first and surveyors were then moved northwards to try to account for differences in the timing of the seasons between the south and north of Scotland. During the first visits to each farm the following surveys were undertaken: bird survey, landscape survey(s), archaeology survey(s) (on Rural Stewardship Scheme, Countryside Premium Scheme and non-scheme farms, if archaeological features were present) and conservation audit. The surveys undertaken during the second visits of the season were: repeat bird survey, vegetation sampling and invertebrate surveys (pitfall traps and butterfly surveys). In years 3 to 5 (2006-08) of the project, in order to reduce pressure during the second visits, vegetation sampling of woodlands, scrub and moorland were undertaken during the first visits. The methods for each type of survey are given in sections 4.4.2 to 4.4.5 below. The distribution of farms surveyed in 2004 and 2005, and re-surveyed in 2007 and 2008 respectively, is given in Figure 4.1 on the next page; Figure 4.2 shows farms surveyed in 2006.

Fig 4.1 Distribution of farms surveyed in 2004/05 and again in 2007/08.

'Paired' scheme farms had a non-scheme partner; 'Unpaired' scheme farms did not. Similarly,
'Paired' non-scheme farms had a scheme partner; 'Unpaired' non-scheme farms did not.

Fig 4.2 Distribution of farms surveyed in 2006.

'Paired' scheme farms had a non-scheme partner; 'Unpaired' scheme farms did not. Similarly,
'Paired' non-scheme farms had a scheme partner; 'Unpaired' non-scheme farms did not.

4.4.2 Biodiversity

Conservation audit

As described above ( Section 3) scheme applicants had to produce a farm audit map showing all major habitats on the farm (less detailed for OAS). This provided a useful record of habitats on scheme farms at the start of this study, so in order to look at changes in habitat composition and area on the scheme and non-scheme farms over time, field surveyors produced comparable conservation audits for each farm (in year 1) and during the resurveys of all farms. The audit maps were also used to mark the location of landscape and archaeology surveys, and on non-scheme farms and OAS farms provided a map of habitats which could be used to identify sites for vegetation and invertebrate surveys on the second farm visits of the season. (On RSS and CPS farms the possible locations of vegetation and invertebrate surveys were defined by the location of the management prescriptions, as shown on the official management maps for RSS/ CPS farms.) Given the time required to analyze this type of spatial data to determine change over time, the audit maps were ultimately not exploited in this way (as agreed by the Steering Group).

The audit maps were produced during the first visits of the season to each farm (April-May), after conducting a bird survey and whilst undertaking the landscape and archaeological surveys. It was done by walking over and viewing as much of the farm as possible in the time available, mapping blocks of visible habitat. A hybrid system of habitat classification was used, based on the Phase 1 habitat classification ( JNCC 1990) and the conservation audit codes used in audit maps for RSS and CPS applications. The minimum mappable area was as for a Phase 1 survey (0.1ha) where time permitted, but frequently where the terrain was difficult or the area large, the surveyed area was necessarily less, and parts of the farm were not always covered or were surveyed through binoculars, as only one day was available for each visit to a farm. The habitat codes used were as follows:

For the purposes of this audit, woodland was described as established if it was two metres or more in height and newly planted if under that height. The condition of hedges and walls was briefly assessed by looking at how intact or gappy they were, and noting on a three-point scale whether the hedge/wall was in good, medium or poor condition (good meaning few gaps/mostly intact, poor meaning very gappy/collapsed, and medium meaning intermediate condition).

Vegetation sampling

Step-point sampling (Evans & Love 1957) was used to quantify the species of plants present in surveyed prescriptions (for scheme farms) or habitats (for non-scheme farms). The prescriptions surveyed were management of species-rich grassland, extended hedge, other hedge, creation of wetland, management of wetland, water margin and conservation headland ( RSS only); other prescriptions were surveyed but proved to be too scarce to permit analyses and so are not mentioned here. The equivalent non-scheme habitats were matched as closely as possible to the scheme prescription with which the data would be compared in the analyses. So for example, extended hedges on scheme farms were compared with hedges on non-scheme farms and conservation headlands were compared with cereal margins on the non-scheme farms. For the OAS farms we simply monitored grass or cereal fields and their obvious equivalents on the non-scheme farms. Prescriptions or equivalent habitats suitable for vegetation sampling were identified by reference to the official farm management maps on scheme farms, and on non-scheme farms with the aid of the conservation audits. The observer took an arbitrary, predetermined route through the habitat and noted the species present at 200 sampling points. The route usually criss-crossed the habitat from one side to the other, to ensure coverage of a large proportion of the total area. The number of points used (200) was determined a priori from the time available for sampling and a power analysis comparing the amount of change necessary before that degree of change becomes detectable with varying numbers of steps in the sampling procedure. So for example, if a particular species of plant were detected at 20% of points in year one, with a 200-step sampling procedure the proportion of points at which the species was present in year two would have to be below 9% or above 31% before that change in frequency were detectable at a statistically significant level (P_0.05). This compares to limits of 4% and 36% for 100 steps and 11% and 29% for 300 steps.

The sampling points were spaced evenly throughout the habitat (minimum of 2 m apart, unless the size of the habitat being surveyed prevented this), with the approximate distance between points determined before the survey of each habitat patch. At each point a pole was placed vertically on the ground and each species touching the pole was noted as present on a pro forma containing a habitat-specific list of the most likely species to be encountered. Maximum vegetation height was also noted. At each sampling point, a species touching the pole in more than one place was only recorded once. This procedure provided estimates of the number of species present in a patch and the proportion of the 200 samples at which each species was present. Where species counts have been combined to construct the indicator groups (see Results, Section 5) the sum of counts can exceed 200.

In the first two years of the project, plants were recorded as far as possible to species level. However, because of the large number of species and difficulties (especially for grasses) with identification (often from vegetative parts only), in subsequent years the list of plants on the recording forms was revised, providing groupings of plants which could be more reliably identified by surveyors, and allowing recording of the data to be more rapid. The various plant groups are shown on the field sheets given in Appendix 7. The groupings included:

• where appropriate, very large groupings such as 'trees and scrub', 'ericoid' and 'fern (not bracken)';
• genus groups, such as 'pondweed' and 'St John's-wort';
• groups within genera, such as 'marsh violet' and 'violet (not marsh)'; 'large docks (not sorrel)' and 'common sorrel'; 'rosebay willowherb', 'marsh willowherb' and 'other willowherb';
• the grouping of sedges into 'large sedges' (bottle sedge size or larger) and 'small sedges' (smaller than bottle sedge);
• the grouping of grasses (for grasslands, wetlands and water margins) into the broad categories of 'very large' ( e.g. common reed), 'coarse' ( e.g. false oat-grass, couch), 'fine' ( e.g. fescues, mat-grass) and 'other' grasses, apart from several more easily-identifiable and key species still identified to species (including rye-grass, cocksfoot, purple moor-grass and tufted hair-grass).

If surveyors perceived, for example, two different 'buttercups' at a sample point, the count for that category was two. This situation was infrequent with broad-leaved herbs, however, since an individual point sample is by its nature very narrow, and thus there was a low likelihood of more than one herb from one of the plant groups appearing at a single point. It occurred more frequently with grasses, but surveyors were still asked to make more than one record for a grass category if they perceived more than one grass of that category at a point. The vegetation data recorded in 2004 and 2005 was converted to the same plant groupings to allow comparative analysis.

This basic procedure was implemented for grassland, wetland and moorland habitats, and their non-scheme equivalents. However, it was adapted for arable habitats, water margins, hedges, scrub and woodland.

• In arable habitats the route taken by the surveyor was constrained to the tramlines so as to avoid trampling the crop. Each sample point was located by stretching into the crop perpendicular to the tramline ( i.e. samples were not located on the tramline itself nor within the small area of crop between the two tramlines). Vegetation height here was the height of the non-crop vegetation.
• In water margins 100 pairs of points were taken at each location: one point approximately 1 m into the water and one approximately 1 m away from the water's edge, resulting in 200 points in total. This was to try and account for both bank-side and aquatic/marginal vegetation. Where vegetation filled a ditch, the boundary was taken as the far side of the emergent vegetation.
• For hedges both the woody hedge and grassy margin were sampled at each sampling point, so the species present and total height were recorded from the margin, and the hedge species in line with the sampling point were noted. In addition, the average size and shape of the hedge was noted along with signs of management (trimming, gap filling, etc.).
• In scrubby habitats the presence or absence of woody species was recorded along with height at each sampling point. For continuous blocks of scrub, cover was recorded as 100% along with an estimate of average height as these stands were often impenetrable.
• Within woodland, species of sapling and mature trees were recorded within 1 m of each point with their respective heights; points were a minimum of 5 m apart, to avoid re-sampling the same plants.

During the subsequent analysis of the grassland, wetland and water margin data an investigation was carried out on 'desirable' and 'undesirable' plants, as described in section 4.5.

Ground-active invertebrate sampling

The aim of this sampling protocol was to monitor any impacts of management on ground-active beetles as many of the prescriptions of interest are designed to increase invertebrate numbers and this group is frequently found in the diets of farmland birds (Wilson et al. 1999, Holland et al. 2006). Also, the same methodology could be applied to any of the habitats of interest.

Ten pitfall traps, each 6 cm in diameter, were sunk into the ground in a straight line at least two metres apart within the prescription or non-scheme equivalent habitat to be sampled (avoiding the edge of the habitat by at least 2 m, except in grass margins where they were placed in the middle of the strip of habitat). The prescriptions sampled were creation of species-rich grassland, management of species-rich grassland, conservation headland and grass margin. The lip of the trap was flush with the ground level which occasionally meant raising ground level around the lip of the trap where it was not possible to sink the trap sufficiently, but this was not restricted to any particular habitats or to any type of farm (scheme or non-scheme) and so did not introduce any potential bias to the results. A small volume of soapy water was added to each trap to help retain individuals falling into the trap. The traps were set upon arrival at the site and the contents were collected at the end of the day. This was at least six hours later and was the maximum possible period with the available resources.

Trap contents were sorted into beetles, including beetle larvae, and other invertebrates, and the number of individuals in each sample was counted at the site.

Butterfly sampling

Butterflies can provide an indication of habitat quality and some prescriptions on RSS and CPS farms were designed to encourage them. Butterflies were surveyed using an adapted form of the UK Butterfly Monitoring Scheme (reviewed in detail in Pollard & Yates 1993). Butterflies were counted during a 15 minute walk through creation and management of species-rich grassland prescriptions, conservation headlands, grass margins, moorland and semi-natural woodland (and their equivalent habitats on non-scheme farms), between 1100 and 1500 hours. Individuals were identified to species where possible; when butterflies were too distant to identify, the number of butterflies seen was still recorded. Surveys were only carried out when winds were below force 5 (Beaufort scale) and when there was no precipitation. These restrictions meant that often a butterfly survey was not possible. This in turn resulted in too few butterfly surveys on OAS farms - which were relatively few in number anyway - to permit robust analysis.

Survey of breeding birds

Birds are good indicators of biodiversity in general, as they are dependent upon various plant and insect food sources, and require various habitats for breeding. The survey method used here was based on the Breeding Bird Survey developed by the British Trust for Ornithology, RSPB and the Joint Nature Conservation Council (the method is described at http://www.bto.org/bbs/take_part/bbs_instructions_2009.pdf). This is the standard method used to monitor UK bird populations and commonly used in modified form for other surveys. Birds were counted whilst walking along a transect 2km in length (3km where this included moorland, to increase the likelihood of encountering birds in this habitat, which could by widely dispersed within what was normally a large habitat block). The route followed boundary habitats and key prescriptions or habitats in order to increase the likelihood of encountering breeding birds and of detecting any response to the implementation of prescriptions, but was not restricted solely to prescriptions or to particular prescriptions. The transect was not necessarily contiguous, but was sometimes broken into two or occasionally three sections where this helped maximize coverage of the target habitats. No part of the route went within 200m of any other part of the route. Counts were carried out before 0900, first between April and May (during the first visit to each farm), and then again between May and July (the second visit). The first count was designed to coincide with the peak abundance of resident and early migrant species, whilst the second count coincided with the peak in later arriving migrants. All birds seen or heard within 100 m of the transect were recorded. The maximum number of each species seen at each site across the two counts was taken as the value for analysis.

4.4.3 Evaluation of biodiversity sampling

Vegetation

Step-point sampling was used to obtain estimates of the number of species present and their frequency within habitat patches. Step-point sampling has been in use for some considerable time and is thought to have been first formally described by Evans and Love (1957). Since then it has been employed for many purposes with most references originating in the USA. It is generally thought to be most appropriate for use in relatively short vegetation as this avoids some of the complications caused by multiple 'hits' at any sampling location. However, this is only really an issue when time is short - equally applicable to quadrat sampling (see below). The primary advantage of step-point sampling over the alternatives - and the reason it was chosen for use here - is that it gives far better coverage of the patch of habitat being surveyed, in a given time, than is possible via quadrats (or any area-based sampling procedure) and therefore provides a better reflection of the nature of the vegetation in that habitat. Put another way, step-point sampling requires far less time to provide the necessary amount of information in order to understand the large habitat patches encountered during this study. Also, it is a quantitative method which only measures the frequency of occurrence of plant species, therefore removing the subjectivity and additional time required to estimate cover.

The primary alternative approach to surveying vegetation that could have been employed here was via quadrats. This utilises a frame of standard area which is placed on the ground at a random point and various parameters recorded from within it. For example, it is possible to record the number of plants of each species present, the frequency of occurrence or area covered for each species ( e.g. via the Domin scale, Currall 1987). There are several disadvantages to the use of quadrat sampling which do not apply to step-point sampling, in addition to the issue of coverage and sampling speed mentioned above. Firstly, careful consideration, and justification, has to be given to the size and shape of the quadrat used; ideally a pilot study would compare different sizes to empirically determine the appropriate dimensions for the task in hand (Whalley & Hardy 2000, Hill et al. 2005). Secondly, care must equally be taken in positioning the quadrat so as to minimize disturbance to vegetation and therefore biasing the data collected, and also in ensuring the quadrat is placed randomly. Thirdly, there are often edge effects reported with quadrat sampling as it can sometimes be a subjective matter deciding what is inside or outside the quadrat. Finally, when monitoring vegetation change over a period of time (as for the RSS and some OAS farms, and their non-scheme pairs in this study) an alternative to standard quadrat sampling that has been employed elsewhere is to use permanent quadrats (or quadrats sampling the same location each time). It is often assumed that the use of permanent quadrats reduces the number of sampling points that are needed, but this is not the case as it would still be necessary to cover a greater proportion of the habitat patch than is usually the case with any form of quadrat sampling given a limited amount of time. Furthermore, permanent quadrats would have been inappropriate here because of the high degree of disturbance typical of agricultural habitats making it impossible to use permanent markers for quadrat locations and therefore difficult to accurately relocate the required sampling location.

Ground-active invertebrate sampling

Pitfall traps were used to estimate the abundance of invertebrates on the ground. Pitfall trapping is the standard technique for sampling ground-active invertebrates. It is cheap (large numbers of small, simple traps can be deployed at little cost) and relatively quick to carry out. The only other plausible alternative here that might be used to quickly sample invertebrates at ground level is suction sampling. However, this is widely recognized as being far less reliable as it is unlikely to pick-up animals from the ground, particularly when the vegetation in the habitat is long and/or dense (Southwood & Henderson 2000).

There are no hard and fast rules regarding the application of pitfall traps as the details will vary depending on the aims of the study. However, it is generally recommended that 10 traps are laid out in a line within any habitat patch of interest (Hill et al. 2005), as was employed in this study. The number of animals caught is influenced by a number of factors: their population density; their movements; the boundary of the pitfall trap; and the nature of the boundary of the area being sampled, and the ability of animals to cross it. The movement of ground-active invertebrates will be influenced by temperature, moisture (and sometimes other weather variables), food supply, habitat and the characteristics of the individuals being targeted (age, sex, health, etc.) (Southwood & Henderson 2000). The pitfall trap itself, in particular its size and the material from which it is made, can influence the catch. Generally, the larger the diameter of the trap, the more individuals caught. There are no general principles with regard to trap material, as different species have been shown to react differently to a variety of materials (Southwood & Henderson 2000). In addition, the precise level of the trap in relation to ground level can be important - for example whether the trap is at soil or litter-level, and also the ability of the trap to retain individuals can have an obvious impact on trap efficiency (Southwood & Henderson 2000).

Most of these influencing factors are irrelevant to this study as any imperfections were common to all situations, as the same traps were used in the same manner throughout and so cannot introduce any systematic bias. However, it is possible that differences in the nature of the vegetation between in-scheme prescriptions and their out-of-scheme equivalent habitats may have introduced some inconsistencies if they affected the movements of individuals, thereby potentially introducing some degree of bias. For example, when comparing conservation headlands with a conventional cereal margin on a non-scheme farm, it is possible that the weedier conservation headland could have impeded the movement of invertebrates enough, relative to the more open cereal field, to have reduced trapping efficiency in this habitat. It is not possible to determine, without further study, whether, and to what degree, this was a factor.

There is no standard regarding the period of time that traps should be left in place. However, it is usual to leave them for longer periods than a matter of hours. Generally, the longer traps are set, the more animals they catch, although the rate of capture decreases sharply, perhaps as local resident animals are caught early on (Southwood & Henderson 2000). The period of time employed here (a minimum of 6 hours, but generally no more than 7 hours) was very much constrained by the time available.

Butterfly sampling

Timed counts were used to assess butterfly abundance whilst walking through a variety of habitats. This approach is an adapted form of the UK Butterfly Monitoring Scheme (reviewed in detail in Pollard & Yates 1993). Usually a transect is established at a site and butterflies are recorded along the route weekly under reasonable weather conditions for a number of years. For population monitoring purposes, transects are chosen to sample evenly the variety of habitat types and management activity on sites. The same route is walked each year. Transects are typically about 2-4km long, taking between 45 minutes and two hours to walk, and are divided into sections corresponding to different habitat or management units.

Butterflies are recorded in a fixed width band (typically 5m wide) along the transect each week from the beginning of April until the end of September, between 1045 hrs and 1545 hrs and only when weather conditions are suitable for butterfly activity: dry conditions, wind speed below force 5 and temperature 13°C or greater if there is at least 60% sunshine, or more than 17°C if overcast.

There were no obvious, viable alternatives to the approach adopted here for the survey of butterflies.

Breeding bird monitoring

The method used here was based upon the British Trust for Ornithology's ( BTO) Breeding Bird Survey ( BBS). This is currently the standard way in which national and regional populations of all birds are monitored in the UK. For the BBS, a 1km square is surveyed by simply walking along two parallel transects, each of 1 km in length, approximately 500 m apart and 250 m in from the edge of the survey square. Deviations from the ideal route are permitted (to allow for difficulties with access) but no two parts of the transect should be less than 200 m apart. All birds seen or heard are recorded within different distance bands: 0-25 m from the transect line, 26-100 m and over 100 m from the transect line (including outside the survey square). Birds flying over the route are also noted as such. The transect route is divided into 10 sections and the habitat characteristics of each are noted at the beginning of the season. Each square is visited twice, first between April and mid-May and then again between mid-May and the end of June. The first count is designed to coincide with the peak abundance of resident and early migrant species, whilst the second count coincides with the peak in later arriving migrants. Counts are carried out early in the morning - usually starting between 0600 and 0700 hrs, but no later than 0900 hrs. The maximum number of each species seen at each site across the two counts is taken as the value for analysis.

The method adopted for this study followed these basic rules except that (as described above) transects were deliberately placed so as to follow likely breeding habitats and birds were simply recorded within 100 m of the transect route, not within different distance bands.

The latter alteration makes no difference to the analysis of the results as BTO analytical methods ignore the distance bands in their estimation of annual indices. The first amendment to the methods increases the likelihood of detecting most species of breeding birds present compared to the standard BBS ( e.g. Bradbury & Allen 2003, Stevens & Bradbury 2006), such as finches, sparrows, buntings, warblers, thrushes, etc. Only truly open-field species, like Skylark Alauda arvensis and waders, might have been less likely to be recorded, although the breeding behaviour of the Skylark and many waders is quite conspicuous, compensating somewhat for this difference in approach. Having said that, the difference in approach is only relevant when comparing data from this study with BTO monitoring data: for the comparisons within this study it has no impact because the methods used were the same for all sites. This amended approach has been used elsewhere for monitoring the impact of agri-environment schemes on farmland birds (Bradbury & Allen 2003, Stevens & Bradbury 2006) and it is not uncommon to survey birds in this way more generally (Bibby et al. 2000). The precise timing of counts was also slightly later during the study here. This simply reflects the later start to breeding in Scotland than in southern England, where the guidelines are most applicable, as recommended by the BTO ( http://www.bto.org/bbs/take_part/bbs_instructions_2009.pdf).

The other potential alternatives to this approach were the use of point counts or territory mapping methods. Point counts require the observer to stop at predetermined locations along a survey route for a fixed period of time, usually coupled with the recording of birds seen or heard whilst moving between points (Bibby et al. 2000). This has the obvious disadvantage in this instance of taking considerably more time to complete than transect counts. Similarly, territory mapping methods require a much greater investment of time and resources as each site should be visited at least four times, preferably 6-10 times, during the course of the season in order to build up a picture of the location of birds from which the approximate location (and number) of territories can be estimated (Bibby et al. 2000).

In sum, the approach taken here was deemed to be the most appropriate given the time and resource limitations, and the methods should not have influenced the results because, most importantly, the same methods were used in the appropriate prescriptions/habitats on all farms throughout the study.

4.4.4 Landscape

There were three elements to the methodology: to record landscape change, to interpret the change, and to identify the impact of the scheme measures on landscape. The survey recorded individual landscape components and important landscape features within the farms. The primary technique was the use of fixed-point photographs to record landscape changes. A landscape pro-forma was also developed (see Appendix 7); a pilot survey was carried out with a landscape architect and an ecologist to test the survey forms in relation to the quality of the data collected and the time taken to complete them. To ensure consistency in approach, training of field surveyors was given by a landscape architect, which included a visit to one of the farms in the scheme to test the data collection methodology. Viewpoint locations for landscape surveys were chosen by surveyors in the field on the basis of what were considered good vantage points and representative views of the farms. At least one and up to five landscape surveys were conducted per farm, depending on the size and landscape diversity of each farm. The locations of landscape surveys were marked on the conservation audit maps, and GPS grid references taken of the vantage points where surveyors stood.

The data from the landscape field sheets were analysed in a similar manner to the biodiversity statistical analysis, looking for change between schemes and over time. The fixed-point photographs were examined by two landscape architects, comparing records from the first visits to each farm with those gathered during the second visits three years later, using a five-point scale to record any changes. This methodology was devised by Scott Wilson Scotland Ltd and is based on the recommendations contained in the following documents:

• The Landscape Institute & Institute of Environmental Management and Assessment (2002). Guidelines for Landscape and Visual Impact Assessment (Second Edition). Spon Press, London/New York. (For assessment guidance.)
• Swanwick, C. & Land Use Consultants (2002). Landscape Character Assessment Guidance for England and Scotland. SNH & Countryside Agency. (For advice on recording landscape change.)
• Transport Scotland (2008). Scottish Transport Appraisal Guidance. Scottish Government. (For a useful simple scoring system.)

The landscape changes looked for included increases or decreases in vegetation cover, new habitats or removal of existing ones, erection or removal of fences, and any other landscape changes that might be apparent from the photographs. All comparable farms were assessed over a three year period, i.e. two sets of farms, first set 2004-2007, and second set 2005-2008. The intervening year, 2006, did not have a comparable year, as explained in section 4.3.3.

The five-point scale used during the analysis is shown below:

• +2 Notable beneficial change
• +1 Minor beneficial change
• 0 No change
• -1 Minor adverse change
• -2 Notable adverse change

For farms with notable beneficial or adverse changes, the landscape architect then:

• made an assessment of the sensitivity of the receiving landscape (high, medium or low) based on the capacity of the landscape to absorb change, e.g. varied topography or vegetation patterns which provide screening;
• from the relevant national landscape character assessment prepared by Scottish Natural Heritage ( SNH), identified the key characteristics of the character area within which the site was located;
• assessed the changes against the sensitivity of the receiving landscape and the SNH key characteristics;
• made a judgement about the significance of the changes and the impact on the landscape.

The value of the changes (adverse or beneficial) was a judgement discussed between the landscape architects undertaking the assessment. New fences were generally regarded as beneficial changes if they were replacing old broken down fences or introduced to fence off ditches etc.; however, a new fence introduced into a former open area was regarded as an adverse change.

To ensure consistency in approach in the analysis, two landscape architects carried out the survey data interpretation, working alongside each other and discussing issues as they arose. In order to provide a second opinion, a third landscape architect reviewed a sample of the interpretation. The lead landscape architect and second opinion landscape architects had extensive experience in landscape character assessment and an understanding of Scottish landscapes. The interpretation of the results concentrated on the magnitude of change across the board and related to landscape character rather than qualities of designated sites or scenic value.

Criteria for the assessment of landscape change were intended to refer to changes to key characteristics as identified in the SNH Landscape Character Assessment documents ( LCA). However, none of the recorded changes fell into the notable category which triggered comparison with the LCA key characteristics, except changes which were not part of the scheme, e.g. introduction of farm buildings.

Two issues raised by SNH which could not be incorporated into the methodology are as follows.

• Ground-truthing of a representative sample by a landscape architect. This was not possible owing to budget constraints. Instead, the pilot survey and training exercise included visits to farms in the scheme (see comments above).
• Weighting of landscape surveys close to e.g. tourist routes, public paths or designated sites was not given. This was outside the scope of the assessment, and budgetary constraints would have precluded a detailed analysis of location.

4.4.5 Cultural heritage

Archaeological sites (up to a maximum of 5 per farm) on Rural Stewardship Scheme ( RSS) and Countryside Premium Scheme ( CPS) farms were selected from those archaeological features under management as identified in the official farm management maps and schedules. On partner non-scheme farms archaeological sites (usually up to a maximum of 5 per farm) were identified using Pastmap (an online map-enabled query system for Scottish national and regional archaeological and architectural datasets, hosted by the Royal Commission for Ancient and Historic Monuments in Scotland). Rural Stewardship Scheme and partner non-scheme sites were assessed in two years, either 2005 and 2008, or 2004 and 2007; Countryside Premium Scheme farms and partner non-scheme farms were surveyed in 2006. Archaeological surveys were not conducted for the Organic Aid Scheme ( OAS), which unlike RSS and CPS does not have prescriptions for archaeology preservation, but is concerned with promoting organic farming practices.

The field archaeological surveys were undertaken by field surveyors who were given training. The methodology required the surveyors to visit individual archaeological sites on a farm and collect data via fixed point photography, and complete an archaeology pro-forma (see Appendix 7). A grid reference of the photograph position was taken with a GPS and this point marked on the conservation audit map.

The analysis was based on the assessment of archaeological features in the photographs, augmented by information from the field sheets. A numerical scale (-2 to +2) was used to determine whether the change had been positive or negative and also to give an indication of the magnitude of the change.

Rural Stewardship Scheme farms

The RSS farms and paired non-scheme farms were surveyed over two seasons 3 years apart, with the first survey carried out just after the farms joined the scheme. Thus it was possible to identify the baseline condition of each archaeological site on a farm on the first visit and any observable change on the second visit. Each archaeological site was then scored based on the observable change that had occurred between the visits. This provided a score for each archaeological site on a farm.

The assessment criteria for each value on the numerical scale were as follows:

• +2: Site is in better condition, possibly managed, vegetation cleared, no rubbish, possibly repaired/renovated, protected from further damage incurred through farming practice ( e.g. livestock damage).
• +1: Site is in better condition, with some of the above changes but not all.
• 0: No visible change is observed (this is sometimes attributable to a lack of photographic comparability).
• -1: Site is in worse condition, rubbish around, or erosion through livestock, previous protection measures ( e.g. fencing) no longer operational.
• -2: Site is in worse condition, has been damaged, parts removed/knocked down, or is being used for something else, e.g. a standing stone as a gate post/part of fence.

Countryside Premium Scheme farms

The Countryside Premium Scheme farms and paired non-scheme farms were only surveyed in one year in 2006, as explained in section 4.3.3. This meant that it was not possible to make an assessment of the change in condition of an archaeological site over time. All that was possible was to make a broad comparison of the general condition of all the archaeological sites present on one farm and mark this on the numerical scale. Broad comparisons of the general condition of sites between scheme and non scheme farms could then be made by comparing the difference between these scores. Again the majority of the useful information was provided by the photographs.

The assessment criteria for the numerical scale were as follows:

• +2: Site is in fair-good condition, cleared of heavy vegetation, upstanding, little sign of erosion/damage etc, with evidence of management, and/or protective measures in place where appropriate, e.g. fencing.
• +1: Site is in satisfactory-fair condition and may have protection measures.
• 0: Site is in satisfactory condition and may not have any evidence of protection measures.
• -1: Site is in satisfactory-poor condition, showing signs of erosion, tumbling, damage etc, with no evidence of protective measures.
• -2: Site is in poor-bad condition with clear damage to site, removal or destruction of site.

4.5 Statistical analysis

Bird data

For birds the measures analysed were: total number of species, number of species of birds in various groups ( BAP species, red- and amber-listed species and various functional groups - see below), total number of individual birds, number of individual birds in the aforementioned groups (all of which were transformed to log(number+1) in order to normalize the data and therefore make it suitable for analysis via the tests used), and the Shannon index. The latter (also known as the Shannon-Weiner diversity index (Magurran 2004) is a means of comparing diversity between groups: it is a step up in sophistication from simple species richness estimates ( i.e., the number of different species present) as it looks at how the number of individuals are distributed amongst the different species present (Magurran 2004).

BAP species are those that have a designated Biodiversity Action Plan as part of Government plans to conserve these species. Red and amber listed species are those designated as being of 'greatest' and 'moderate' conservation concern respectively by the RSPB, BTO, Wildfowl and Wetlands Trust, Countryside Council for Wales, Environment and Heritage Service (Northern Ireland), Natural England and Scottish Natural Heritage ( SNH). The birds were also divided into functional groups for analysis on the basis of shared ecological traits. This allowed exploration of potential mechanisms behind possible major effects: for example, if only seed-eating birds showed a response to the schemes then it might be suggested that the schemes were perhaps providing something only the seed-eaters could exploit, like more seed-food. The groups were:

• 'Crows' - Carrion Crow ( Corvus corone), Rook ( Corvus frugilegus), Jackdaw ( Corvus monedula) & Magpie ( Pica pica);
• 'Pigeons' - Woodpigeon ( Columba palumbus), Rock/Feral Pigeon ( Columba livia), Stock Dove ( Columba oenas) and Collared Dove ( Streptopelia decoacto);
• 'Tits' - Blue Tit ( Cyanistes caeruleus), Great Tit ( Parus major) and Coal Tit ( Periparus ater);
• 'Finches' - Chaffinch ( Fringilla coelebs), Greenfinch ( Cardeulis chloris), Goldfinch ( Cardeulis cardeulis), Linnet ( Cardeulis cannabina), Bullfinch ( Pyrrhula pyrrhula), Yellowhammer ( Emberiza citrinella), Reed Bunting ( Emberiza schoeniclus) and Corn Bunting ( Emberiza calandra);
• 'Hirundines' - Swallow ( Hirundo rustica), House Martin ( Delichon urbicum) and Sand Martin ( Riparia riparia);
• 'Woodland warblers' - Blackcap ( Sylvia atricapilla), Garden Warbler ( Sylvia borin), Wood Warbler ( Phylloscopus sibilatrix), Chiffchaff ( Phylloscopus collybita) and Willow Warbler ( Phylloscopus trochilus);
• 'Farmland warblers' -: Whitethroat ( Sylvia communis), Sedge Warbler ( Acrocephalus schoenobaenus), Grasshopper Warbler ( Locustella naevia) and Reed Warbler ( Acrocephalus scirpaseus);
• 'Waders' - Curlew ( Numenius arquata), Lapwing ( Vanellus vanellus), Oystercatcher ( Haematopus ostralegus), Redshank ( Tringa totanus) and Snipe ( Gallinago gallinago);
• 'Ground feeders' - Blackbird ( Turdus merula), Song Thrush ( Turdus philomelos), Mistle Thrush ( Turdus viscivorus), Starling ( Sturnus vulgaris), Robin ( Erithacus rubecula), Dunnock ( Prunella modularis) and Wren ( Troglodytes troglodytes);
• 'Gamebirds' - Pheasant ( Phasianus colchicus), Grey Partridge ( Perdix perdix) and Red-legged Partridge ( Alectoris rufa).

Vegetation data

The measures analyzed here were the total number of species or groups of species (see Section 4.4.2) present and the Shannon index. For grasslands, wetlands and water margins an additional analysis was made of indicator species. Species were designated as 'desirable' or 'undesirable' (or neither) using information supplied by the Steering Group, supplemented by information from the JNCC Common Standards Monitoring ( JNCC 2004a,b). Briefly, a desirable species is one that is thought to indicate conditions typical of that habitat and so might indicate the persistence of species richness. By contrast an undesirable species is one that might indicate conditions likely to result in reduced species richness in a particular habitat, like the presence of high nutrient levels. Lists of these indicator species can be found in Appendix 6. Whilst some species were classed purely as desirable or undesirable (depending on the habitat), others were classed as undesirable only if present in large quantity (for this analysis meaning present in 30% or more of sample points), in small quantity merely adding to the floristic diversity. For this analysis the number of indicator species present was analyzed as was the number of step-point samples at which indicator species were present.

Ground-active invertebrates and butterfly data

For all the ground-active invertebrates, the measures analyzed were mean number of beetles and 'other' invertebrates per trap. For the butterflies the mean number of individuals seen was analyzed.

Analysis of RSS and re-surveyed OAS farms

For the basic analyses (simply comparing the biodiversity measures between the scheme and non-scheme farms and the change over time) of RSS and OAS farms which were re-surveyed after three years, an Analysis of Variance ( ANOVA) was performed for each measure of biodiversity. This is a test that will consider the distribution of the variance in the data between various explanatory factors and indicate which of those factors (if any) is significantly influencing the dependent variable ( i.e. which one might be having an effect over-and-above that which might be expected by chance alone). Here, the ANOVA models included a Pair effect (identifying each pair of farms in the sample), Time effect (visit 1 or visit 2), Scheme effect (scheme or control) and the Scheme by Time interaction. The interaction measures whether the change in biodiversity over time is different between the scheme and non-scheme farms, and is the effect of most interest in this study. The pair effect reduces the residual variance (that which is unexplained) by accounting for differences between the different pairs. This effect is almost always highly significant and reflects both regional effects and the influence of both the weather and farming operations at the time of sampling.

The ANOVA would be straightforward with complete data ( i.e. all farms paired, with two visits for each), but the amount of unavailable data complicates matters. The most powerful analysis uses all the data (including unpaired farms and those with appropriate data from only one visit), but can be misleading if the Scheme or Time effects are very different between the farms. Therefore two analyses were done: one on the complete data only (where the scheme farms were paired with non-scheme farms and the appropriate data were collected from both visits to each), and one on the full data set (using all the data). If these disagree the results should be viewed cautiously. The most common outcome is that the direction of the effects is the same in both analyses, but the full data analysis shows more significant results.

The potential influence of Farm Type and Region was considered via a further ANOVA on the bird data from RSS farms. This was only possible with the bird data because the other data sets were too sparse (sample sizes were too small) to allow the further sub-divisions of the data required to incorporate these factors. Farm Type was defined as either livestock (dairy, LFA sheep & cattle, lowland sheep & cattle and pigs & poultry) or crop (unknown mixed, mixed, cereal, general crop, horticulture and other) based on the definition assigned to each farm by SGRPID. These combinations were most practical because of the small number of farms in many of the basic categories. Three different regions were also defined, based on the farms' SGRPID area offices: 'Highland' region = Inverness, Lairg, Thurso, Oban, Orkney, Portree, Stornoway; 'North East' region = Elgin, Inverurie, Perth; and 'Southern' region = Ayr, Dumfries, Hamilton, Galashiels. Figure 4.3 below shows the SGRPID area offices (it is labelled as ' SEERAD area boundaries and offices', but these are still the same under SGRPID). The ANOVA for this analysis included the factors Farm Type and Region, plus their interaction. In this case the dependent variable was the difference in the change over time between the RSS and non-scheme farms: this would highlight whether joining the scheme had impacted on the birds present in a way that was dependent on these factors; that is, whether the scheme only 'worked' in a certain region or a certain type of farm.

In a final analysis the biodiversity measures on each RSS farm and its non-scheme pair were combined into a single composite measure for each pair. This was then averaged across all pairs and allowed an estimation of the overall effect of the RSS on biodiversity. So, on each pair of farms a single measure was created for each type of biodiversity data (birds, beetles, crops, moorland, wetland, hedgerow, grassland, butterflies). For birds this was simply the total number of species. For all the others except beetles it was the number of species averaged over all prescriptions that existed on both the RSS farm and its partner. Total number of species was used because this measure was available for most groups. For beetles it was the mean number of beetles per trap, again taken over all prescriptions that existed on both the RSS farm and its partner.

For each of these new measures, the RSS farm was considered better than its pair if the number of species (or individuals) increased more between the two visits on the RSS farm (or decreased less). Where the RSS farm was better that measure was assigned a '1'; where it was worse it was assigned a '0'. The composite measure for the pair was then found by taking the average of these measures, so that if there were four composite biodiversity measures, two indicating a greater increase on the RSS farm (two '1s') and two indicating a greater increase on the non-scheme farm (two '0s'), the average would be 0.5. This was done for all paired farms with two visits. A t-test was then done on the composite measures for all pairs comparing the data with an average of 0.5: this is the overall proportion of measures that would be better on one half of partner farms in a random sample of paired non-scheme farms. A confidence interval was also found for the "scheme effect". The confidence interval is important as it relates the sample mean to the 'true' mean in the population as a whole: for example, a 95% confidence interval defines the limits within which there is a 95% probability that the true mean would be found (Fowler et al. 1998). Therefore the narrower the confidence interval, the more accurately the sample mean reflects the true mean. If this analysis shows there was no difference between the mean composite measure for all pairs of farms and a hypothetical population of farms with a mean composite measure of 0.5, then it could be concluded that joining the RSS made no further difference to biodiversity than would have been expected by chance.

Analysis of CPS and single-visit OAS farms

For the CPS and single-visit OAS analyses, univariate tests were used. Paired t-tests were carried out on the complete data (where the appropriate data had been collected from both visits): these explicitly compare the scheme and non-scheme farms whilst considering the paired organization of the data. Unpaired t-tests were carried out on the full data set (using all the available data, including farms where data were lacking from one of the visits) as these do not require the data to be organized in pairs. Binomial tests were also performed, but the results of this test, which is not very powerful, agreed with the t-tests where they were significant, and are not mentioned further.

An additional ANOVA of the CPS bird data was undertaken to look at the potential influence of the time since the farms joined the scheme on the biodiversity measures. Again, only the bird data were suitable here because of small sample sizes for the other groups. The only factor in the ANOVA model here was Year which referred to the year in which the farm joined the CPS. This ranged from 1997 to 2000 (35 farms joined in 1997, 55 in 1998, 23 in 1999 and five in 2000), so at the time of sampling (2006) the farms could have been in the scheme for between six and nine years.

Fig 4.3 Boundaries and offices of SGRPID (formerly SEERAD) areas

Analysis to compare the CPS, RSS and OAS

An analysis was also carried out comparing the three schemes to see if there were any consistent differences between them in the birds present (this data set being the only appropriate one again for the aforementioned reasons). Again, ANOVAs were used to look at i) the differences in the numbers of birds or species between the scheme and non-scheme pairs and how this varied between the three schemes, ii) the differences between the scheme farms only and, iii) the differences between the non-scheme farms only. In the case of the RSS and OAS farms that were visited in years 1 and 4, data from the latter year was used in the analysis ( i.e. after the farms had been in their scheme for three years). The factors in this ANOVA were Scheme ( CPS, RSS or OAS), Farm Type (livestock or crop) and Region (Highland, North-east and Southern) in order to fully explore the latter two factors further.

For all the above analyses, the tables of means and their standard errors presented have been calculated from the complete data set for the relevant combination of scheme and time.

Landscape and cultural heritage statistical analysis

The results of the landscape photographic analysis were not subjected to statistical analysis as the results were very clear, with very little perceived change. The data from the landscape field sheets were analysed in a similar way to the biodiversity data. For those farms which were visited in two separate years with a three year interval, an ANOVA was conducted on the landscape data, including the factors of Pair, Scheme, Time and the Scheme-Time interaction. For those farms visited in one year only ( i.e. without a time factor) a comparison of scheme and non-scheme farms was conducted using paired t-tests.

The results of the RSS archaeological analysis were not subjected to statistical analysis because, again, there were so few observed changes. However, the CPS archaeology results showed sufficient observed differences between scheme and non-scheme farms to warrant further investigation: a t-test was used to determine the significance of these differences.