Pediatric Informatics: Computer Applications in Child Health (Health ...

172 M.J. Feldman
system or etiology. 38 Diagnoses are linked to original material from pediatric texts.
Limitations of the system include:
Lack of recognition of negative findings and of certain abbreviations (Ex: PTT,
PPD).
Omission of clinical findings if textbook descriptions are incomplete or if
there is a mismatch between the terminology used by the author and the user.
This limitation is minimized by the use of synonyms and by indexing larger
numbers of textbooks. Initially, the developers did not rank order the diagnostic
lists (to reinforce Isabel’s function as a reminder system 35 ), but have since added
a “degree of match view” to provide a goodness of fit measure between each
matched diagnosis and the entered terms.
Several studies have evaluated Isabel’s performance. In one study of inpatients,
91% of expected diagnoses (in 90 of 99 hypothetical cases) were listed by Isabel. In
the second stage of this study, the final diagnosis from the discharge summary was
listed by Isabel in 95% (83 of 87) of real cases for which the Isabel KB contained
the diagnosis. 35 A second study using the same dataset with two pediatricians working
together to determine a gold standard (one to four diagnoses per case), showed
Isabel listing all gold standard diagnoses in 73% of cases, and in an additional 15%
of cases, Isabel listed at least half of gold standard diagnoses. 38 A retrospective
evaluation of records from 200 pediatric intensive care unit patients in an Indian
hospital showed Isabel listing the final diagnosis (from discharge summary, death
certificate, or autopsy report) 80.5% of the time. 39 In a prospective study analyzing
161 patients admitted to five pediatric intensive care units in the United Kingdom
and the United States where the diagnosis was not known at admission, and where
a discharge diagnosis was available, the admitting team’s differential contained
the ultimate discharge diagnosis in 89.4% of cases whereas Isabel listed the final
diagnosis in 92.5%. 40
12.6 Challenges to Developers and Users of MDDSS
12.6.1 Evaluating MDDSS
In none of these evaluations was the program used as it would likely be used in
actual clinical practice. In all cases, MDDSS output was compared directly with
the discharge diagnosis (and the gold standard for discharge diagnosis often was
not specified, e.g. pathology etc.), or a list of diagnoses developed by clinicians
independent of the system. In actual practice, providers using the MDDSS would
more likely filter advice by considering diagnostic reminders, and may or may
not alter working differential diagnoses. One study 41 of medical students, residents,
and faculty listing differential diagnoses for cases based on actual patients
before and after using MDDSS showed a small but significant increase in both the
number of correct diagnoses and the plausibility of the other diagnoses listed