Health Information Management: Integrating Information Technology ...

More documents

Recommendations

Info

STRATEGY, IMPLEMENTATION AND EVALUATION 197 patient data management systems, physician order entry systems and decision support systems. Many benefits are claimed: such systems will ‘enhance the quality and efficiency of care’, ‘empower the patient’, ‘provide an answer to labour shortages’, ‘help reduce medical errors’ and so forth. Yet such claims can only be validated through evaluation of the performances and the effects of (using) these systems. In addition, the introduction and maintenance of PCISs consumes large amounts of resources and implementation failure is a very traumatic event for an organization. Decision-makers and those who are responsible for the procurement or development of IT are expected to demonstrate that resources spent on IT provide benefits in clinical outcomes, cost savings, and/or to the health care process. Furthermore, evaluation is increasingly important since there is a need to understand the effects of PCIS on the social, professional and organizational context in which they are used. Those responsible for the design and implementation of evaluation studies, in the meantime, are faced with a bewildering and often conflicting array of choices and dilemmas concerning evaluation criteria, study designs, data collection methods and analysis techniques. In this chapter, we discuss some of the choices and dilemmas in PCIS evaluation, and provide the reader with practical guidance. While evaluation can be aimed at many audiences, we here focus primarily on PCIS evaluation aimed at informing professionals and organization management in their decision-making about the development, implementation and use of the PCIS. This chapter covers the following topics: ■ ■ ■ the randomized controlled trial debate; evaluation from a sociotechnical point of view; steps to designing an evaluation: theory and practice. THE RANDOMIZED CONTROLLED TRIAL DEBATE In the field of evaluation of PCIS, two kinds of evaluation are distinguished: formative and summative. Formative evaluations focus on the continuous improvement of the system. Research is done throughout the lifecycle of a system, aimed at facilitating the organizational learning that is imperative for successful system design and implementation. Summative evaluations, on the other hand, are done to account post hoc for the promised or expected benefits, such as financial savings or the effectiveness of information systems in terms of clinical outcomes. Where formative evaluations are internally oriented, providing insights for the work groups, the project and steering group alike, summative evaluations are externally oriented, towards those paying or politically responsible for the PCIS. From the literature on evaluation of PCIS in health care it is clear that (literature on) summative evaluations are predominant in the field of PCIS evaluation. More specifically, since the field of PCIS evaluation is strongly
198 EVALUATION OF PCISS influenced by the health care context in which it finds itself, the golden standard for evaluating new technologies—as in medicine—is the Randomized Controlled Trial (RCT). An RCT used for testing a new drug, for example, means that two randomly selected groups of patient are created: one group that is given the intervention (the drug) and a control group that is not exposed to the intervention (by being given a placebo). The doctor and the patients are both ‘blinded’ they do not know whether the treatment consists of the drug or the placebo. As a consequence, neither doctors nor patients know whether the drug causes the effect of the treatment or not. Randomization and blinding are means to eradicate confounding variables and biased interpretation. RCTs are regarded as optimally suited to prove whether a new medical technology is effective or not—or whether the effectiveness lies in the eye of the beholder, in the placebo effect and so forth. However, given the need for a control group, to control for unwanted variables, and the need for a strict definition of and protocol for the measurements targeted for improvement, RCTs take up much work and are expensive. Given the fact that IT is also a technology, it is not so strange that health care professionals, medical journals and, in their wake, many health care managers are primarily focused on the RCT as the model for evaluation. Hence, the history of PCIS evaluation is replete with RCT-like studies proving the effectiveness in terms of clinical decisions of, for example, alert and reminder systems (see Kaplan 2001). Yet over the past two decades, more and more authors have voiced their concern about this dominance of the RCT model for PCIS evaluation. RCTs are criticized because of different reasons. First of all, the fact that conceptualizing (parts of) PCIS systems as ‘the intervention’ that a RCT focuses on, supposes that it makes sense and is possible to isolate the functioning of this ‘hardware’ from the social processes that surround it and are integrated with its current functioning. Yet as we have seen time and time again in this book, this assumption is dubious —to say the least. In practice, it is often impossible to disentangle the ‘effect’ caused by the PCIS itself from the ‘effect’ caused by the changes in the work practices induced by the PCIS implementation. (In fact, as we argued in the last few chapters, PCIS systems should be introduced as part and parcel of an organizational development process, which renders a search for the ‘unique effects’ of the information technology wholly elusive). Second, a properly executed RCT is labour intensive and expensive. Conducting an RCT means that randomization has to be accomplished: by patient, or groups of professionals. In the former, the same doctor may use a decision support system on randomly allocated patients, for example. In the latter (also known as clustered randomized trials), individual professionals, departments or whole hospitals are randomized to use the Decision Support System or not to use it. However, generating these kind of ‘objective’ circumstances is often impossible. For example, patient-randomized trials have a strong risk of the ‘carry-over’ effect, i.e. doctors learn from the system and apply this new
Page 2 and 3:
Health Information Management In al
Page 4 and 5:
Health Information Management Integ
Page 6 and 7:
Contents List of illustrations vii
Page 8 and 9:
Illustrations FIGURES 3.1 Medicatio
Page 10 and 11:
Contributors Marc Berg is a physici
Page 12 and 13:
Series preface In almost all Wester
Page 14 and 15:
Acknowledgements I would like to th
Page 16 and 17:
Chapter 1 Introduction In all Weste
Page 18 and 19:
INTRODUCTION 3 touch upon the deplo
Page 20 and 21:
INTRODUCTION 5 can draw upon to mai
Page 22 and 23:
INTRODUCTION 7 Physician Order Entr
Page 24 and 25:
Part I Starting points Understandin
Page 26 and 27:
WAITING FOR GODOT 11 KEY TERMS ■
Page 28 and 29:
WAITING FOR GODOT 13 Netherlands an
Page 30 and 31:
WAITING FOR GODOT 15 facts to show
Page 32 and 33:
WAITING FOR GODOT 17 (Lewinski-Corw
Page 34 and 35:
WAITING FOR GODOT 19 In most cases,
Page 36 and 37:
WAITING FOR GODOT 21 (Anonymous 191
Page 38 and 39:
WAITING FOR GODOT 23 Because of the
Page 40 and 41:
WAITING FOR GODOT 25 they simply sa
Page 42 and 43:
WAITING FOR GODOT 27 expected. Stil
Page 44 and 45:
WAITING FOR GODOT 29 organize their
Page 46 and 47:
WAITING FOR GODOT 31 performance in
Page 48 and 49:
WAITING FOR GODOT 33 different HIS
Page 50 and 51:
WAITING FOR GODOT 35 CONCLUSION: TH
Page 52 and 53:
WAITING FOR GODOT 37 assumption tha
Page 54 and 55:
WAITING FOR GODOT 39 Anonymous (191
Page 56 and 57:
WAITING FOR GODOT 41 Berg, M. (1997
Page 58 and 59:
Chapter 3 Health care work and pati
Page 60 and 61:
HEALTH CARE WORK AND INFORMATION SY
Page 62 and 63:
Page 64 and 65:
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
Page 74 and 75:
Page 76 and 77:
Page 78 and 79:
THE CONTEXTUAL NATURE OF INFORMATIO
Page 80 and 81:
Page 82 and 83:
Page 84 and 85:
Page 86 and 87:
Page 88 and 89:
Page 90 and 91:
Page 92 and 93:
Page 94 and 95:
Chapter 5 Meeting the challenge Int
Page 96 and 97:
STARTING POINTS 81 ‘gender, race,
Page 98 and 99:
STARTING POINTS 83 Now, imagine an
Page 100 and 101:
STARTING POINTS 85 organization,
Page 102 and 103:
STARTING POINTS 87 Figure 5.2 Incre
Page 104 and 105:
STARTING POINTS 89 already overflow
Page 106 and 107:
STARTING POINTS 91 responsibilities
Page 108 and 109:
STARTING POINTS 93 answer the quest
Page 110 and 111:
STARTING POINTS 95 Kohn, L.T., J.M.
Page 112 and 113:
Chapter 6 Designing interactions En
Page 114 and 115:
DESIGNING INTERACTIONS 99 the way t
Page 116 and 117:
DESIGNING INTERACTIONS 101 1 Identi
Page 118 and 119:
DESIGNING INTERACTIONS 103 the deve
Page 120 and 121:
DESIGNING INTERACTIONS 105 communic
Page 122 and 123:
DESIGNING INTERACTIONS 107 Figure 6
Page 124 and 125:
DESIGNING INTERACTIONS 109 ■ Figu
Page 126 and 127:
DESIGNING INTERACTIONS 111 measurem
Page 128 and 129:
Page 130 and 131:
Page 132 and 133:
DESIGNING INTERACTIONS 117 Covell D
Page 134 and 135:
Part II Information strategy, imple
Page 136 and 137:
INFORMATION STRATEGY: AN INTRODUCTI
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Chapter 8 Developing the informatio
Page 154 and 155:
STRATEGY, IMPLEMENTATION AND EVALUA
Page 156 and 157:
Page 158 and 159:
Page 160 and 161:
Page 162 and 163: STRATEGY, IMPLEMENTATION AND EVALUA
Page 172 and 173: Guidelines for information system d
Page 180 and 181: Chapter 9 Implementing information
Page 182 and 183: IMPLEMENTING INFORMATION SYSTEMS 16
Page 198 and 199: Chapter 10 Project management of in
Page 236 and 237: Index accumulation 53-4, 61, 73-4,
Page 238 and 239: INDEX 223 contingent and emergent c
Page 240 and 241: INDEX 225 satisfaction 176 patient
Page 242: INDEX 227 support 164 technology de
show all

Health Information Management: Integrating Information Technology ...

Create successful ePaper yourself

Delete template?

Save as template?