Monitoring and Evaluation
Measurement of innovation, and its economic impact, is not a simple process. The OECD paper " OECD Work On Innovation - A Stocktaking Of Existing Work 13, published in February 2009 states that:
"much work, both theoretical and empirical, has already been done to identify the policies, institutions and framework conditions that can provide the most effective means of supporting innovation. However, evaluation of specific government support policies and their impacts on innovation is generally sparse and there is a need for more and better evidence on the costs and benefits of government support for innovation. Measurement of innovation is also an area where further work is likely to yield benefits, particularly in better understanding innovation in the service sector and better capturing the increasingly international nature of innovation activities."
Some of the difficulties raised in the paper include:
"It is clear that aggregate measures of R&D spending are a misleading indicator of innovative activity for two reasons: they are influenced by industrial structures; and much innovative activity is not captured by R&D spending 14. In any case, looking at the amount of resources devoted to R&D is not sufficient to assess a country's innovation outcome, since this depends on how efficiently resources are used. Measures of innovation performance, such as patents, get closer to this. It is also clear that better statistics for services are required. A move from traditional sectoral approaches to definitions that better capture the interaction of services with other industries and the characteristics of innovation in services appears to be required 15.
The new realities of science, technology and innovation are also challenging the statistical STI indicators' focus on the activities of specialised R&D institutions and the particular national location of such activities 16. It is now more difficult to differentiate between "novelty" and "routine", and improvements are often made outside formal R&D departments. In addition, over time, the complex sectoral origin and nature of innovation has made popular technological classifications of high/medium/low-R&D intensive industries somewhat less useful. A low-R&D industry may be a large user of knowledge generated elsewhere - this holds true for many service sectors, where the introduction of new processes, organisational structures, or product innovations, is unlikely to involve much formal R&D investment. Rendering statistical systems more flexible and responsive to the introduction of new and fast evolving concepts that are typical of the STI field is crucial to keep up with the changing context.
In addition, the link between the location of "national" firms' private R&D activities and a country's productivity gains appears increasingly loose, for small and large countries. The largest part of worldwide productivity growth in the last decade has been associated with acceleration in the diffusion of technological change and with global access to codified knowledge. One of the greatest challenges to policy makers is to acknowledge more fully the global public good of STI."
Service based sectors account for a significant share of Scotland's GVA and employment: however traditional definitions of 'innovation' and measures of innovation (i.e. spend on R&D, patent activity 17) have focused on technological innovation. By its nature, technical R&D is less relevant to the service sectors than it is in manufacturing industries and it is likely that traditional measures of innovation are not capturing the full extent of innovative activity taking place in a heavily service sector based economy such as Scotland's.
The development of new products and services in service sectors and creative industries often arises from a variety of non-scientific sources, with the role of design, the creative application of existing technologies, experience and feedback from customers are increasingly driving change in the economy. However, indicators to measure these kinds of innovations are not as developed as that for more tangible technological innovation.
However, how we monitor and evaluate progress is crucial to how we behave in achieving our vision and objectives. In the ( DIUS) White Paper Innovation Nation, NESTA was tasked with developing an innovation index that would capture the new, emerging view of innovation that goes beyond the science, R&D-push, linear model. This extremely complex task takes in the problem of defining innovation in this new context and producing measures which capture the complexity of the system but still manage to produce outputs that are relatively easily understood, that allow for international benchmarking and for measuring trends in progress. NESTA have been asked to produce an interim report in 2009 and a final version in 2010. The Scottish Government is closely involved in this process.
As already outlined above, however, the GES has pioneered a radically different approach by providing a single clear outcome for the whole of Government: increasing sustainable economic growth. In this Framework, therefore, we concentrate on innovation that has an economic impact. That is why the definition we use mentions the "successful exploitation" of new ideas. All of our efforts are aligned to that end and will be judged on that outcome. The way that we judge the success of those efforts is the National Performance Framework, Scotland Performs. It adopts the Virginia model of monitoring and evaluation to measure progress towards outcomes rather than a series of micro-level targets which would have the potential to result in perverse behaviours intended to meet the target rather than contribute to the overall outcome. The National Performance Framework ( NPF) is how our efforts on innovation will be judged. This new, outcome-focussed approach will allow Government and its partners flexibility and autonomy in deciding how they achieve the outcomes, freeing up their potential to be innovative themselves and encourage innovation in others.
In terms of the NPF, it is likely that innovation will have the closest links with to following purpose targets:
- Sustainability. Innovation will be key to developing renewable energy technology and improving energy efficiency to meet targets on greenhouse gas emission reductions through, for example, the Saltire Prize £10m challenge for innovative advances in marine renewable energy.
- GDP growth. As discussed above, innovation is generally held to be central to raising productivity levels and generating economic growth.
- Population growth and healthy life expectancy. While there are a number of other factors involved, including benefits from developments in global research, advances in medicine and treatment of illnesses and disease through research and development may contribute to this target.