roadMAP [PDF, 1.9 MB] - fteval

• There is always a goal conflict between structural goals and the limited time horizon of
funding: Multi Actor systems are created; they develop their own lives and a deep interest to
ensure a long-term existence which is a MAP goal. MAP funding on the other hand stops
after some years and it is often not possible or wise to plan self-sustaining initiatives: A
common MAP specific goal is to create self sustaining networks but this is often a naive idea:
ambitious networks / structures / organisations will not survive without some public money
and without a kind of “MAP governance”, namely if real scientific research shall be performed
(see chapter 8 on governance).
• As behavioural and institutional change is important for MAPs, quantitative goals are only to
be applied in a limited form, namely impact on firms and some output indicators. Hence,
qualitative goals are always necessary elements of MAPs (see chapter 7 on evaluation and
monitoring).
• As most MAPs show a large range of different aims, conflicting goals may occur, for
example: the maintenance of scientific excellence, thus following the rules of basic science
with a large amount of freedom, and an accelerated time to market; or as an other example:
long term research and SME inclusion. Such conflicting goals should and can be dealt with in
advance, often being easy to solve at this stage: With proper rules science can flourish and
be relevant.
4.4.2 Formulation of goals and evaluation design as one interlinked process
A crucial point with regard to monitoring and evaluation procedures is the synchronisation of
goal formulation and evaluation procedures before starting the MAP, e.g. in the design phase it
is necessary to formulate goals that are suited for evaluation. Generally spoken, evaluations are
more difficult than in other programmes (see chapter 7 on evaluation and monitoring): As
mentioned above, MAPs pursue a lot of aims, quantitative and qualitative, many of them with a
structural character which means that they are very difficult to measure – at least in the short
run. In the design process attention should also be given to the definition of relevant indicators.
As MAPs are complex, goals and criteria and also indicators are multi-dimensional. However,
summative performance measurement rapidly reaches its limits. For this reason formative,
learning-oriented evaluation approaches can and should be used – partly in competition, partly
complementary to the summative ones. On the basis of multiple evaluation results on the
relationship between programme targets, measures and impacts, spectra of alternative paths of
further development of a programme can be shown, alternatively regarding adjusted targets and
differing starting points, addressees and time and objective range. Such information can aid the
actors to play their strategic games more transparently and "more effectively" in the general
interest.
roadMAP 25