The Evaluation of 'Behavioural Additionality' - IWT

CHAPTER 3 > Conceptual and Empirical Challenges of Evaluating the Effectiveness of
Innovation Policies with ‘Behavioural Additionality’ .
Table 2 >
Behavioural Additionality in many dimensions
Organisational level Project Company / Strategy System
Process level
Input Scale R&D-budget
Time
Knowledge spillovers
Risk
Portfolio composition
Throughput
- Internal Project management Innovation management
IPR behaviour
Synergy in project family
- External Cooperation Alliances Networking / clustering
Strategic autonomy
Relations with VC / financiers
Quality label
Localisation
Output Procesinnovation Strengthening core or Improved environmental impact
Productinnovation differentiation
Training Specialised knowledge / know-how Human capital
>
strategies, are also a potential strategic area
for making a difference, if the necessary
conditions are present (horizontal innovation
policy).
4. ABOUT THE MEASUREMENT OF
BEHAVIOURAL ADDITIONALITY
The broadening of the additionality concept
makes it more complex and multiplies the
evaluation and measurement problems. The
most known problems that have been identified
are the problem of ‘counterfactual’
evaluation and the problem of ‘attribution’
of cause and effect. The first problem is
inevitable because time is irreversible and
conditions are never completely comparable
between a situation with and without a subsidy.
Control groups of firms without subsidies
to measure the impact by comparison
are very difficult to compose; the reasons for
applying (or for denying) subsidies are
linked with other characteristics that already
differentiate these firms. Moreover the control
group may benefit from spillovers! The
second problem is more general the problem
of separating one impact – a subsidy - in
a multi-causal phenomenon.
These problems are not to be solved with
some sophisticated measurement technique,
but the strategy can be to use complexity
itself as a solution method, using the complementarity
between different forms of
partial solution. Measurement is objectivation
without mystification!
The evaluation of the effectiveness of modern
innovation policies imply two kinds of
specific evaluation challenges: the assessment
of the behavioural additionality of
instruments on the interactions in the system,
and the assessment of the behavioural
additionality on social return. This is because
we consider the mutual reference of knowledge
production (creation) and knowledge
consumption (diffusion and use) as being
the main engine of cumulative growth of
the knowledge economy
The first item – interaction additionality - concerns
the opening of the ‘black box’ of the
interaction between agency and firms, in particular
the effects of instruments on the decision
processes and private strategies about
R&D (one-time and enduring effects). The different
types of behavioural effects of the
instrument need to be identified and
assessed for their learning leverage. This will
preferably be in quantitative form in order to
be related to innovation performance.
But this evaluation finds also an end in itself,
supporting policy design to improve the
additionality enhancing qualities of the
questioned instrument and the policy mix.
How this can be done is by institutional
learning in the form of real life ‘experiments’
and studies on the basis of hypothesis
testing (thought experiments in policy
debate or econometric tests). The use of the
control group and of appropriate ways of
‘revealing’ the ‘true’ behaviour are important
items for this assessment.
66