Gaining health : analysis of policy development in European ...

Chapter 5
242
way through the implementation process. Considerable efforts
were made by expert groups and task forces, and the
results were published in detailed interim reports. Reports
on the results of the monitoring and valuation process then
become a valuable resource for future policy revision.
There is a general lack of policy analysis to help inform the
policy development process. One exception is Sweden,
where it has been decided that there should be a regular
and systemic review of the fulfilment of the objectives of its
public health policy (26).
Difficulties encountered
There are inherent difficulties in monitoring and evaluating
the type of broad policies necessary to promote health,
prevent disease and meet the needs of those suffering
from NCD. As seen above, such policies attempt to bring
about changes in social structures by influencing the social
and economic determinants of health, changes in lifestyles
and behaviour, changes in access to health care and other
services, and changes in the way in which such services perform
in the areas of health promotion, disease prevention,
treatment and rehabilitation. It is difficult to disentangle
the positive and negative impacts of such broad policies,
and there is as yet no widely recognized scientific tool
for distinguishing the impact of one or other programme
element in an implementation process. It is also difficult to
disentangle the impact of general socioeconomic changes
that would have happened anyway from those targeted by
NCD policies.
Particularly in the case of the intersectoral policies necessary
to combat NCD, it is important that the policy document
itself defines how monitoring and evaluation are to be
carried out, who will do it and how they will report back,
and what resources will be made available,.
As has been stated elsewhere (27):
Health promotion action requires multiple approaches, relies
on interdisciplinary inputs and operates at several levels over
long periods of time. Despite this complexity, health promotion
programmes are often forced to be evaluated with methods and
approaches that, although quite acceptable within medical care
and prevention, are totally unsuitable for this field.
It is clear that in this complex field, multiple methods of
evaluation are necessary, involving various partners and using
both process and outcome information. It has been suggested
that a minimum of 10% of the total financial resources
for a health promotion initiative should be allocated for its
evaluation (27). Hungary tried to implement the 10% formula
but this was cut as soon as there was a budget crisis.
It has been suggested (28) that certain areas in the evaluation
of policies, such as those with which we are concerned,
particularly need attention.
• A clearer definition of objectives and targets, whether
related to outcomes or to processes, would avoid
inappropriate expectations concerning evaluation and
accountability.
• This requires more systematic use and development of
reliable indicators of progress.
• An appropriate level of intensity in monitoring and
evaluating should be adopted, modest attention being
paid to monitoring tried and tested interventions and
closer scrutiny given to newer, innovative or potentially
costly interventions.
• Appropriate methods of evaluation should be designed.
Certain experimental methods, including randomized
controlled trials, are simply not appropriate for evaluating
complex policies. The advantages of different research
methodologies, both quantitative and qualitative,
including a wide range of data and information, need to
be combined in an approach that allows the issues to be
seen from the point of view of different disciplines.
Reflections on experiences