Health Information Management: Integrating Information Technology ...

202 EVALUATION OF PCISS
evaluations, integrating quantitative and qualitative research methods in this way
requires that these quantitative designs should be quite ‘modest’ in nature: before
—after designs are useful, but more strictly experimental designs (like RCTs)
exceed their aim. Their design is too rigorous to be of help, often directly hinders
the organizational change process that is occurring, and they yield very few
results in the light of their costs.
In the next paragraph we will elaborate on the different steps that come to play
when designing and conducting an evaluation.
STEPS IN DESIGNING AN EVALUATION
In this paragraph we will describe the design of an evaluation as following
several, successive steps (as shown in Figure 11.1) (Heathfield et al. 2001). We
will focus primarily, but not exclusively, on formative evaluation. In reality, the
individual steps are often difficult to separate, and usually impossible to manage
as a simple and steady sequence. The process of designing an evaluation is one
of adjustment and compromise, where choices concerning study design,
evaluation criteria and data collection methods must be offset against the
constraints of conflicting stake-holders’ aims, deadlines, resources available, the
intrusiveness of the evaluation and ethical considerations. The design of an
evaluation is as much a social and political process as that of IT procurement and
implementation itself. By using case material (from the UK and the
Netherlands), we will show that taking these steps can be quite complicated.
Every next step is determined by the situation at hand. Though the cases differ in
technology, set up and success, both can be seen as a realistic example of the
implementation of a PCIS in health care. The more general ‘Issues’ and ‘Lessons
learned’ at the end of each step are based on material from these two sites as
well as other case material.
CASE STUDY
The first case is based upon a study in which the UK NHS Executive,
through its Information Management Group, initiated two programmes of
work which sought to improve patient records using information
technology: the Electronic Patient Record (EPR) project and The Clinical
Workstation (ICWS) project. The EPR project was a three-year research
and development programme to assist clinicians in acute hospitals to
provide better care to patients through the use of electronic patient record
systems. The ICWS project was a two-year project concerned
with providing interface facilities to an EPR. A number of sites with
established Hospital Information Systems and sites in the early stages of
procurement and implementation but with an organizational commitment
to an EPR were selected. External evaluation was seen as essential to both
the EPR and ICWS projects and the evaluation covered all five sites.