Challenges and Opportunities for Innovation in the Public Works ...

Arthur Baskin and Stephen C-Y Lu
3.2 Information Systems Should be Grounded in Current Practice
Even when using new information systems to promote cultural change, we have found it wise
to conduct empirical studies of human problem solving in order to better match our tools to
patterns of human information sharing and problem solving (4,5). New tools which conform to
existing patterns of human cognition and problem solving will be more readily assimilated than
tools which require users to adapt to inefficient patterns.
As the design rationale example above indicates, it is possible to evaluate tools by developing
them and metering their effectiveness with actual users. Unfortunately, this generate and test
approach is terribly inefficient and is part of the reason why only a small fraction of developed
software systems are actually put into widespread use. In our work with engineering design, we
have used a different approach. We have conducted empirical studies of architectural design and
aircraft design in which we have analyzed the actions of interactive design teams. We have
looked both at expert and novice behavior. In these situations, we have concentrated on routine
design problems and identified the types of problem solving behavior found. Surprisingly, even
in routine design where one might expect that use of previous patterns might dominate, we have
seen a preponderance of conflict detection and resolution behavior. Thus, even in cooperative
(i.e. non-adversarial) groups, communication of mutually interacting constraints on the solution
plays a dominant role. This rapid folding together of constraints on the solution from different
perspectives leads to better decisions in shorter times.
As part of our development effort, we analyzed transcripts of problem solving activity and
formed a basic theory conflict management for cooperative conflict resolution. Figure 2 shows
a portion of the conflict hierarchy which we developed. In our complete conflict hierarchy, we
were able to identify types of conflicts associated with resource management design and routine
design. Figure 3 shows a schematic diagram of a computer-based system which we developed
to provide conflict management advice to a group of cooperating experts (human and/or
machine). These two figures, taken together, illustrate the pattern of deriving functionality
requirement from studying human performance and then building tools that match the human
performance requirements.
As another example of the importance of grounding new systems in a knowledge of current
practice, consider the development of branching questionnaires to collect medical histories.
These systems were developed in the early 1970s to meet a perceived need for collecting more
complete histories than is possible in a normal office visit. Apart from some managed care
systems in California which used the systems as screening devices, the systems were a failure.
In an effort to be comprehensive, these systems generated so many false positive responses that
their reports were too lengthy to be digested by a physician before an office visit. For this
reason, the systems have not been incorporated into medical practice despite studies that clearly
demonstrate their effectiveness at information gathering. They simply do not fit current practice
and do not provide a sufficiently great benefit to offset their perceived cost.
50