roadMAP [PDF, 1.9 MB] - fteval

enefit/cost analysis” where the data collection turned out to be frustrating in particular in
companies as they very reluctant to present figures, as e.g. on sales. Through interviews and
the structure of the NCE-projects it was evaluated if real or fake collaboration took place. The
third evaluation in 2002 on the rationale of the program summarised that the NCE program is
seen “still as a valuable and unique part of the Canadian research landscape” and gave a
strong recommendation to continue the program. The evaluation methodologies included
International literature review, a review of the NCE performance data and other reports, Survey
of all network partners through an e-mail based survey, interviews with the Network Scientific
Leaders and Network Managers and web-based survey of Network Researchers, finally
interviews with members of the Expert Panels and NCE Selection Committee.
As shown above the StarMAP countries have gained much more experience in particular on
programme evaluation which is evident as most of these programmes run for more then a
decade meanwhile.
7.5 Lessons from the MAP-TN and StarMAP
Coming to an end we want to go back to the very start of our work, were we tried to figure out
which concrete characteristics of MAPs (as defined by the MAP-TN) have which consequence
on the evaluation. Most important seems to be the proper integration of formative evaluation
approaches based on a judgement of the network members.
The following picture can give an outlook on the further challenges in the field of evaluation of
some of the MAP-TN programmes: The common understanding that evaluation has to be a
strategic exercise, which needs a lot of professional know-how as well as a professional and
competent management for the sound implementation of evaluation results. One more point is
that a lot of MAPs struggle with the problem of vested interest: There is an certain need for
more independency, transparency and publicity of evaluation results.
Exhibit 25: Conclusions and outlook
100
More
Strategic
More
Independent
More
Public
More
competence
More forward
looking
HU: Evaluation strategy, coherent and consistent
implementation
Training for programme Managers
SLO: More structured and professional
management
DE: More strict approach against different
attempts to influence evaluation
UK: Evaluation should be an independent and
Public check on performance
ES: Evaluation should be structured in a
public way with clear impacts on
posterior versions of programmes
UK: more forward looking
roadMAP