The Evaluation of 'Behavioural Additionality' - IWT

CHAPTER 3 > Conceptual and Empirical Challenges of Evaluating the Effectiveness of
Innovation Policies with ‘Behavioural Additionality’ .
But the official adoption of the ‘3% target’
will be a leverage to build more ‘strategic
intelligence’ for policy making, starting from
monitoring and evaluation. The coordination
of efforts to promote R&D with the federal
government, that still is competent for fiscal
instruments to stimulate R&D, is another
incentive to assess more systematically the
composition of the policy mix and of the
innovation system from a strategic viewpoint.
In particular strategic research has to orient
towards the kind of knowledge economy
that has to be constructed for maintaining
the welfare position. Guidance for strategic
evaluation on choices to be made is not
explicitly formulated yet, but without this,
implicit preferences and priorities shape the
agenda. Therefore the question of evaluation
has to be linked with explicit analysis of the
innovation system and the role of government
in improving performance.
Evaluation practices in Flanders at this
moment are rather piecemeal and pragmatic,
e.g. evaluation of the public research
institutes or the evaluation of instruments
within their own organisational boundaries.
They have a fragmented, organisationally
narrow perspective and lack a firm theoretical
basis. The bottom-up character of policies
does not necessarily cause this, because
also project evaluation can be put in a
broader framework. But the stated policy
objectives (promoting innovation for SMEs;
promoting cooperation between actors and
technology transfer from research institutes
in particular; promoting environmental
technology) are topical and not positioned
on a global evaluation scale of the performance
of the innovation system and the strategy
of government. Therefore the formal
selection criteria that try to embody these policy
objectives in stimulation programmes and
projects can only partly cover the issues of
innovation performance on system level. A
broader evaluation approach is needed.
2.2. THE NEW PERSPECTIVE OF GOVER-
NMENT ON INNOVATION
Because of the limitations of actual evaluation
practices it is of great importance to
have a broader view on the behaviour of
innovation actors in the perspective of the
innovation system, for the evaluation of the
success of policy instruments, be it on project,
programme or overall policy levels.
First, just because there is more than what is
captured by the stated policy objectives.
More in particular, the effects of policies have
to be considered as the result of an interaction
between public strategies on the one
hand and private strategies on the other;
companies responding from their own point
of view to the incentives of public instruments.
Different instruments also combine
mutually through specific interactions (policy
mix). Therefore an ‘ex post’ evaluation of the
global impact of government stimulation is
needed that goes beyond a narrow (one
organisation) incentive (one instrument) to
effect (one objective) perspective. Policy
instruments have to be evaluated as one
among many variables that explain the performance
of the targeted actors, and the
same goes for the policy mix at the level of
the system performance as a whole. Does policy
‘make a difference’ How to evaluate
results of these interactions
Second, the constrained nature of evaluation
at project and actor level of a set of private
intentions or results and related public
objectives can obscure the broader range of
social returns. These are not fully implied in
the actual policy set-up. Innovation seldom
is an individual activity but the result of
complex interactive processes. Policy development
is just beginning to discover and
explore the huge ‘reserves’ of productivity
gains that are linked to the better operation
of the linkages in the innovation system.
Innovation policy has to manage the innovation
system from the point of view of the
social return; Flemish innovation policy in
particular has to devise a strategy to
enhance the performance of the national
innovation system to secure welfare creation
for its citizens. This role of government
as provider of new kinds of (soft) infrastructure
for innovation actors has to be matched
with appropriate evaluation models.
The ambitions of evaluation policy are
therefore closely linked with the apprecia-
60