Health Information Management: Integrating Information Technology ...

STRATEGY, IMPLEMENTATION AND EVALUATION 199
knowledge to the treatment of control patients as well as intervention ones.
Clustered RCTs may not provide comparable control and intervention groups,
given that no two professionals, departments or hospitals are likely to be the
same. In both types of trial it is not possible to ‘blind’ the participants from the
presence of the health care PCIS.
In addition, generating these kind of ‘objective’ circumstances is often also
undesirable. The specific routines, work processes, leadership patterns, cultures
of professionals and departments within and between hospitals differ—and such
issues are often key in understanding why systems may fail in one situation and
succeed in another. While RCTs will give ‘hard data’ (in the form of establishing
relations between) on a very limited set of pre-set parameters, it cannot answer
the why or how questions that are often the most relevant when one wants to
understand PCIS implementation. Nor can it grasp all the unanticipated
consequences, like sabotage of a system because of user resistance, that are often
crucial to the fate of PCISs. RCT researchers themselves often stress that their
designs are of limited ‘real-world’ use due to the artificial, laboratory
circumstances (e.g. simulation patients, inexperienced subjects) in which the data
are produced. Knowing more about these issues requires methods that are able to
understand the way information systems work in daily practices and the
interaction between the system and the user. In much recent literature on ICT
evaluation, qualitative methods like interviews, participant observations and
focus group meetings are seen as much more suitable for this goal (see also
further).
Finally, social scientists have argued that ‘scientificness’ comes in many guises,
of which hypothesis testing is only one. Building a theory explaining a specific
social phenomenon, for example, is an equally scientific endeavour, which may
or may not be amenable to quantification, or to ‘testing’ through an RCT.
FORMATIVE EVALUATION OF PATIENT CARE
INFORMATION SYSTEMS
Partly due to this historical entwinement with the RCT as ‘gold standard’, the PCIS
evaluation literature still sharply divides ‘objectivist’ and ‘subjectivist’
approaches to PCIS evaluation. The objectivist position starts with the
assumption that the merits and worth of an information system can and should be
quantifiable. ‘To measure is to know’, it is often stated, and observations should
be as objective as possible. In this way, it is argued, two observations of the same
phenomenon will yield the same results, without being affected by how the
resource under study functions or is used. Much attention, therefore, is given to
the avoidance of measurement error, and the prevention of biased observation.
Such issues, indeed, are crucial for any definite summative evaluation. For
formative evaluations, we will argue however, an overly stringent focus on
objective measurement is beside the point.