Guide to Research and Innovation Strategies for Smart Specialisation

Step 6 - Integration of monitoring and evaluation mechanisms
Monitoring and evaluation
The establishment of monitoring indicators and evaluation plans is an important element of
the RIS3 design process both at the level of strategy and Action Plan. Integration of robust
evaluation into the RIS has been seen as an essential area of progress in the history of RIS,
and presents a key challenge for the RIS3.
Monitoring differs from evaluation in two main respects. Monitoring aims to verify that the
activities are planned, funds are correctly used and spent on delivering planned outputs and
that result indicators evolve in the desired direction. Evaluation, however aims to assess
effects (i.e., the contribution of the interventions to changes in the result indicator) and to
understand why and how the effects are being achieved. It should take account of unintended
results as well as the underlying mechanisms.
Monitoring is normally carried out by actors responsible for implementation, while evaluation
should be carried out by independent experts, but guided closely by those responsible for the
policy. Monitoring and evaluation complement each other. Monitoring provides part of the
empirical basis for evaluation, while evaluation may raise the need for improved monitoring
indicators (to capture new elements, e.g. the depth of partnerships involved in collective
actions).
Both monitoring and evaluation need to be anchored in a clear intervention logic: the
programme should clearly articulate what is the intended change, and how the expected
outputs will lead to the intended results. A frequent problem in evaluation occurs when
strategies and programmes have been established without a clear statement on intended
results.
Targets for result indicators (both in terms of short-term or medium/long-term results) need
to be incorporated into each action and the strategy as a whole from the very beginning. These
targets can be qualitative or quantitative but they must be clear. The indicators form the basis
for the evaluation: when they are missing, this hampers the implementation of effective
evaluations. When applied in conjunction with a performance-based funding discipline,
evaluation provides a powerful way to improve effectiveness of public action to promote
regional innovation.
There is no single standardised approach for developing a monitoring and evaluation system
for a RIS3, since it needs to be tailored to a specific region. In general, result indicators for a
RIS3 should measure change and evolution of the regional productive structure towards
activities that (a) are globally competitive and (b) have a greater potential for value-added.
When results are long term, the programmer may use intermediate result indicators to
measure the progress towards the objective. For example, a programme supporting research
activities may use the number of patents as an intermediate indicator, while the programme
objective is, for example, to increase the production of innovative products.
These indicators should be linked to the programme activities. Their sources could be the
monitoring system, official data bases, ad hoc surveys, peer reviews, more rarely regional or
60