Usability Questionnaire Results - Institute of Aviation - University of ...

ABSTRACTTraditional power system displays are limited in their ability to allow power systemoperators to rapidly assess faults in a power grid. New visualization techniques for displayingpower systems data have been developed to aid operator performance for detecting anddiagnosing faults in a process control simulation. The integrated one-line diagram (part mimicdisplay and close spatial proximity display) and a color contour enhancement were implementedas visualization techniques in this experiment. Participants were required to detect andacknowledge a power grid system failure and diagnose contingencies that would occur in theform of voltage value fluctuations. The voltage value had to be below a threshold limit to beconsidered a violation. Participants were then required to identify and acknowledge all voltageviolations and then go on to correct and solve the violations by switching in capacitors that areelectrically closest until all violations are solved. Twenty-eight participants were randomlyassigned to a tabular and separate static map board display condition (tabular), an integrated onelinediagram display condition (one-line), or an integrated one-line with color contours displaycondition (contour). The proximity compatibility principle predicted that the one-line andcontour display would better support the divided attention task of integrating information forsolving, while visual search models predicted that the tabular display would better support thefocused attention task of detecting. Results generally supported these predictions. Implicationsfor the design of power system displays are discussed.1

1. INTRODUCTIONThe interconnected electric power grid in North America is one of the largest and mostcomplex man-made objects ever created (Overbye, 2000). This grid, which encompasses justabout the entire continent, consists of four large 60 Hz AC synchronous subsystems. Thesesubsystems are 1) the Eastern Interconnect, which supplies electric power to most users east ofthe Rockies, 2) the Western Interconnect, which supplies power to most users west of theRockies and portions of Northern Mexico, 3) the Texas Interconnect, which supplies most ofTexas, and 4) the Quebec Interconnect, which supplies Quebec. These four subsystems are inturn interconnected with each other by DC transmission lines. Altogether, the grid consists ofbillions of individual components, tens of millions of miles of wire, and thousands of individualgenerators.When analyzing power systems data, operators are consequently confronted with a largeamount of multivariate data (Overbye, Klump, & Weber, 1999). For example, in a simple powerflow, data of interest could include a potentially large set of independent and dependentvariables, such as bus voltage magnitudes, transmission line loadings, generator real and reactivereserves, transformer tap and phase positions, scheduled and actual flows between areas, andinterface loadings. In more advanced applications, such as the optimal power flow (OPF),contingency analysis, and available transmission capacity (ATC) calculations, this list ofvariables grows even longer. For example, contingency analysis requires power systemsoperators to plan and react to potential failures within a power grid system, such as whenlightning strikes a part of a power grid causing voltage problems among the system.For the most part, traditional methods for presenting and displaying power system data tooperators are extremely limited. The visualization tools encountered by many power systemoperators and engineers have evolved little beyond tabular displays and the static map boarddiagrams (Overbye, Wiegmann, Rich, & Sun, 2001). In a typical utility control center, systemquantities such as power flows and voltages are usually represented either as analog fields on astatic map board consisting of all the utility’s substations or as entries in tabular list displays. Anoverview of the system is usually only available on a static map board with the only dynamicdata shown using different colored lights.These traditional types of presentation methods do not always allow power systemoperators to rapidly assess the status of the system as a whole, or to accurately assess theinterrelationships among the multiple subsystems throughout the network. For example,performance problems of electric utility operators to voltage disturbances have been associatedwith the severity of power outages. An instance of this occurred when Delmarva Power andLight (DPL) operators experienced a near collapse of their system following a loss of the IndianRiver 2 generator on July 6 th , 1999. One of the issues that contributed to this problem was thedifficulty operators had in simultaneously assessing and understanding the massive amount ofinformation that was presented to them. Consequently, one area clearly in need of furtherresearch and development is the area of information visualization to aid operators in interpretingmultivariate power system data.2

1.2 Evaluation of Integrated One-Line DiagramsVirtually no research has been done to empirically evaluate the effectiveness of theintegrated one-line diagrams and the various feature enhancements that are afforded fordisplaying power systems data. Consequently, it is unclear whether the displays are any betterthan the traditional tabular displays and static map boards that are currently used in powersystem control rooms. Therefore the purpose of the present study was to directly compare theeffects of the integrated one-line diagram with that of the tabular and static map board display onoperator performance. In addition, the potential benefits of color contouring were also examinedas a method for enhancing integrated one-line diagrams. Motion and three-dimensionality werenot examined in this study.2. BACKGROUND AND LITERATURE REVIEWThe expected benefits and costs of the integrated one-line diagram and color contourenhancements were derived from research in different domains. Three lines of research that areof particular relevance to the integrated one-line diagram and color contouring is the work thathas been done in the field of integrated displays and the use of color in displays. The third lineof research addresses issues regarding visual search. However, before these lines of research canbe addressed, it is necessary to discuss process control since power systems are a type of processcontrol. A review of the literature concerning these issues will be presented to better understandthe domain and the design issues involved in the visualization of power systems data.2.1 Process ControlA process control plant is any type of processing plant where a product is produced by aphysical or chemical transformation (Moray, 1997). Moray describes how process control plantsprovide some of the most challenging problems in human factors design of man-machinesystems due to their characterization of having enormous complexity, large physical andconceptual size, the presence of risk and high levels of hazard, high levels of operator workload,complex real-time dynamics, and a need to integrate the activities of many people in a plant.Process control systems have four general characteristics (Wickens & Hollands, 2000).Process variables that are controlled and regulated are slow. An operator who makes a change toa system may not see any indication of a change for seconds or even minutes. A secondcharacteristic is that variables being controlled are essentially analog, continuous processes.Changes normally occur gradually, except for the discrete adjustment of controls by operators.A third characteristic is that a large portion of variables in a process control system areinterrelated. Changes in one variable will often affect another variable. Finally, process controlsystems involve high risk. In the process control of nuclear power, for example, an incident canlead to catastrophic consequences such as a meltdown.The power systems domain described in this thesis is a good example of a processcontrol. As described earlier, a power systems operator is responsible for electricity that hasbeen generated and is being transferred along electrical lines through the power grid. Dependingon an operator’s specific job, he or she may be responsible for ensuring optimal power flows ormonitoring for contingencies. These tasks exemplify how operators must deal with the6

complexity of the power system and its real time dynamics while dealing with high levels ofworkload.Moray (1997) sums up many of the human factors challenges that occur in processcontrol. Some of these include designing control rooms to support safe and economic operation,shifting from physical skills to cognitive skills due to increased use of automation (supportingdecision making), the design and evaluation of advanced interfaces, and the identification of agood balance between prescribed activities and creative intervention.One important role of operators in a process control system is to monitor the state of thesystem. Operators have a particular interest in detecting faults within a system and being able todiagnose the underlying cause of the failures. These two important tasks will be examined in thecontext of process control.2.1.1 DetectionThe detection of abnormal and fault conditions in a process control setting such asnuclear power or power systems is often a complex and difficult task. Detection of faults inprocess control is guided by research in signal detection (Green & Swets, 1966; Swets & Pickett,1982; Wickens & Hollands, 2000) and vigilance (Mackworth, 1948; Davies & Parasuraman,1982; Wickens & Hollands, 2000). Signal detection in its most basic terms consists of twostages: Collecting evidence as to whether a signal is present or absent, and then making adecision about whether the evidence indicates a signal or not. Four outcomes result from signaldetection. A hit indicates that a signal was correctly indicated, a miss means that a signal wasnot indicated, a false alarm indicates a signal was detected when in fact there was none, and acorrect rejection means a signal was not indicated when there was not a signal. Factors affectingsignal detection include sensitivity (the level of discriminability of the signal from noise) and theresponse bias (conservative vs. risky strategy). An example of response bias can be seen bylooking at a nuclear power plant operator. An operator monitoring a nuclear power plant wouldwant to catch all faults that occur due to the severe consequences of missing a failure. Therefore,the operator may have a more liberal response bias and detect more false alarms while detectingmost failures.In the control of power systems, operators may have different response biases dependingon their tasks or instructions. If an operator who monitors a power system has been told by asupervisor not to shut down portions of a grid unnecessarily since it can be quite costly, theoperator may be more conservative when detecting power grid faults and therefore not catch asmany faults. The basis of the research on vigilance has found that the ability to detect signalsdecreases over time, and the steady-state level of detecting signals in vigilance tasks is less thandesirable.Key differences exist from the detection in simulated environments of the classic researchexperiments and process control environments (Wickens & Hollands, 2000). Operators inprocess control do not wait passively for failures but instead are engaged in other monitoring andcontrol tasks (Moray & Rotenberg, 1989). This means that arousal can be maintained and thisreduces the vigilance decrement. Second, failures in process control are normally accompaniedby alarms. This provides a salient feature that draws operator’s attention and aids detection.7

Third, false alarms are much more common in process control environments due to alarmssometimes being too sensitive or abnormal states that signal alarms even though they are inappropriate contexts. Therefore, performance for detecting failures in process control taskswould be better than the performance predicted by classic research experiments due to additionaland redundant sources of information. The likelihood of increased false alarms in processcontrol tasks may lead to some reduced performance for detecting failures since operator’s trustin the reliability of the alarms may be altered.2.1.2 DiagnosisOnce a fault or failure has been detected in process control systems, the next step isnormally to diagnose the fault and take corrective action, but there is a hierarchy of criteria thatneed to be met before diagnosis can commence. Wickens and Hollands (2000) describe threecriteria that are important for the operator to try and meet after failure detection. The mostimportant criteria is to perform actions that will ensure plant or system safety. The nextimportant criteria is to perform actions that will not jeopardize system economy and efficiency.Finally, diagnostic actions should be performed to localize and correct the fault. Diagnosis ofteninvolves the need to integrate information from multiple sources. Failure states may berecognized by looking at the values of multiple parameters. These three criteria may conflictwith one another, especially in the case of ensuring plant safety and not jeopardizing systemeconomy and efficiency. A task that ensures plant safety may be extremely costly in terms ofmoney and time.An important aspect in the ability of operators to correctly diagnose failures in processcontrol systems is for workers to make use of mental models of the systems (Moray, 1997).Operators are able to gain knowledge of the dynamics, physical appearance and layout, andcausal relationships of complex systems after working with them for long periods of time. Thisbecomes the basis for an operator’s mental model. This mental model allows operators to predictfuture states and events and the ability to diagnose failures and unexpected events because oftheir greater understanding of the working system.2.2 Redesign of Displays in Process ControlThe complexity of process control and the importance of supporting operators has led tomuch research on the ability of operators to detect and diagnose problems. An operator’s abilityto interact with the plant revolves around the interface. Most of the information that operatorsacquire about process control operations is from the displays. The difficulties involved withdetecting failures and diagnosing them with current displays in process control systems has led toresearch in display design to find remedial solutions for these tasks. These displays can begrouped into several different categories based on the underlying common features. The resultshave shown creative design solutions that have promise. A few examples will be illustrated.The first type of display to be covered will be mimic displays. Then more integrated displayssuch as the close spatial proximity display, object display, and configural display will bediscussed in detail.8

2.2.1 Mimic DisplaysMimic displays present information at the level of physical function of a system (Hollan,Hutchins, McCandless, Rosenstein, & Weitzman, 1987; Bennett, 1993; Bennett & Malek, 2000).A mimic display provides representations of important components, systems, or subsystems asthey appear in the physical world. For example, a mimic display of an automobile’s fuel systemwould include a schematic outline of the vehicle with a fuel tank and gas lines and valvessuperimposed. The level of fuel would be indicated in the gas tank representation.A particularly good feature of mimic displays is their ability to show physicalconnections and limiting problems of the important components or systems. These provisionslead to better support for detection and diagnosing of faults in a system (Bennett & Malek,2000). Animation in mimic displays has also been achieved to provide additional informationabout the flow of information and resources within a system (Bennett, 1993; Bennett & Malek,2000). Part of the benefit of mimic displays comes from their pictorial realism. This refers to adisplay representation being a pictorial representation of the entity it is representing (Wickens &Hollands, 2000).Moray and Rotenberg (1989) used a type of a mimic display to display the operation of asimulated thermal hydraulic system. Participants in the experiment saw a physical representationof the thermal hydraulic system. Temperatures and the level of fluid in the tanks were directlydisplayed to aid the operator’s mental model of the current state of the system. This wasintended to aid the operator in both detection of faults in the process control and diagnose theproblems.Mimic displays are one useful way to represent information about a system. However,mimic displays do not address the issue of integrating information. The discussion will now turnto integrated displays to discuss the integration issue.2.2.2 Integrated DisplaysThe trend for displays over recent years has been to integrate various sources ofinformation into one display referred to as integrated displays. Integrated displays can bedefined as any type of display that integrates multiple sources of information into one format ordisplay. One component of integrated displays is a distinction made by Garner (1970) betweenwhat was called integral dimensions and separable dimensions. Integral dimensions weredescribed as occurring when the level on one dimension cannot be specified without identifyingthe level on another dimension. An example of this would include a point on an x-y plane. Thepoints place in space can only be determined by acknowledging both its position along the x axisand the y axis. Separable dimensions can be specified independently such as with separate lineson a line graph.Different types of integrated displays have been designed over the years. Each type ofdisplay has its own form of benefit. However, each display has potential for costs inperformance as well. Four types of integration for displays will be discussed.9

2.2.2.1 Close Spatial Proximity DisplaysThe basic feature of a close spatial proximity display is to create a “psychologicalcloseness” between elements in a display through physical manipulations (Wickens & Carswell,1995). Perhaps the most common manipulation to increase the psychological closeness is toplace elements physically close to each other in the display. An example of this occurs in thecockpit of airplanes. The directional heading and altitude reading are important informationsources for navigation and are placed close together in the cockpit. Since these informationsources are mentally linked, the spatial closeness is beneficial. The integrated one-line diagramdescribed earlier has collocated buses and voltage values in an attempt to create a psychologicalcloseness as well.Other types of proximity manipulation described by Wickens and Carswell (1995)include connections, source similarity, code homogeneity, object integration, and configuration.Connections between elements can be made through the use of lines or enclosures. An exampleof source similarity is making separate elements the same color or placing them along the sameaxis. Code homogeneity in a display will also increase proximity. As example of codehomogeneity is exemplified by using two bar graphs to represent two temperatures instead ofusing a bar graph and a round needle indicator. Object integration is defined as arranginginformation sources so that they appear to the user to be part of a single object. Finally,configuration refers to perceptual proximity created by a combination of homogeneous features,close spatial proximity, and the arrangement of sources that leads to the emergence of a newpattern.Woods, O’Brien, and Hanes (1987) examined process control in the nuclear powerdomain. They provide an example of a close spatial proximity display. The control panels innuclear power plants tend to follow the single-sensor single-indicator style of design. Woods etal. showed how simple enhancements such as boxing and lines dramatically helped a user toperceive groupings of controls and indicators. This helped to define the functional relationshipof different controls and indicators which in turn is expected to improve the ability to diagnoseproblems.2.2.2.2 Object DisplaysAn object display is defined as a graphical technique that uses several dimensions of asingle perceptual object to present multiple sources of information (Carswell & Wickens, 1987;Andre & Wickens, 1989). A characteristic of object displays is that elements are linked by linesand/or contours. For example, a square could represent one value along its dimension of widthand another value along its dimension of height. A clear benefit of object displays is that thevarious attributes of a single object are bound together in perceptual processing which leads toparallel processing (Kahneman & Henik, 1981).A particular benefit of object displays is that they often, but not always, produce anemergent feature. The theory of emergent features was first described in work by Pomerantz(Pomerantz, Sager, & Stoever, 1977; Pomerantz, 1981). One way to describe an emergentfeature is as a combination of smaller or simpler elements that produce a new relational orinteractive property (Sanderson, Flach, Buttigieg, & Casey, 1989). Object displays are more10

likely to have emergent features than separated displays. Emergent features allow for enhancedperformance because they have the ability to exchange perceptual cues for cognitive processing(Sanderson, Flach, Buttigieg, & Casey, 1989; Buttigieg & Sanderson, 1991; Bennett & Flach,1992; Vicente & Rasmussen, 1992). Instead of having to integrate information mentally aboutvarious sources of information, the emergent feature in essence does the cognitive processingperceptually and therefore more automatically. It is no longer necessary to perform morecomplex mental operations after examining multiple information sources. Instead, theperception of an emergent feature is all that needed since it represents the conclusion of all thecomplex mental operations that would have been performed.A classic example of an emergent feature can be seen with bar graphs. If three bar graphsare placed next to each other and have the same height when a “normal” state is occurring, animaginary line can be made across the top of the bar graphs. If some type of system changeoccurs and one bar graph’s height increases, the imaginary line connecting the tops of the bargraphs is no longer straight (Sanderson et al., 1989). This can be considered an emergent featuresince a person monitoring the graphs would be looking for any change in the imaginary line todetect problems.One example of an object displays depicts how emergent features can be made torepresent certain states of a system. Woods, Wise, and Hanes (1981) studied a display for anuclear power reactor that represented eight important parameters whose measures wereindicated by the length of spokes extending from the center of the display. The ends of eachspoke were connected to the next closest, thereby forming a polygon. This display providedemergent features in the shape of the polygon. In fact, specific system states and problems couldbe determined just by looking at the shape of the polygon. This supports both the detection offaults and especially diagnosing problems. Certain shapes of the polygon could be indicative ofspecific underlying problems. This understanding could be made through perception of theshape instead of through cognitive reasoning.Emergent features provide an additional useful source of information that is able to “popout” at the user. This is commonly called the salience of an emergent feature. The ability tohave such high salience accounts for part of the benefits in performance attributed to emergentfeatures that substitute perceptual processing for cognitive processing (Wickens & Carswell,1995). This can be quite beneficial when a task or process can be creatively mapped to a displayand utilizes an emergent feature.2.2.2.3 Configural DisplaysConfigural displays are a display format that involves the mapping of several individualvariables into a single geometrical that changes shape dynamically as a function of changes inthe individual variables (Bennett, Payne, Calcaterra, & Nittoli, 2000). Configural displays aredesigned to represent high-level constraints of a system through the relationships that existamong the low-level data that define the constraint (Bennett, Toms, & Woods, 1993).Configural displays are similar to object displays, but a key difference between the twodisplay formats is that configural displays are spatially laid out so that they will always producean emergent feature whereas object displays may or may not have emergent features.11

Furthermore, configural displays need not be object displays. In configural displays, high-levelconstraints are represented as an emergent feature of the structure and behavior of the low-leveldata. This replaces the need of directly detecting the value of the high-level constraint. Theemergent feature is based on the relationship of the low-level graphical elements or theconfiguration of these elements (Bennett, Toms, & Woods, 1993).Bennett, Toms, and Woods (1993) have created a configural display to support theprocess control task of manual control of water level in a boiler during the start-up of a powerplant. The configural display consisted of a square grid where the indicated steam generatorslevel and compensated steam generators level were plotted in the vertical axis. The steam andfeed flow variables were plotted in the horizontal axis. A rectangle is formed where the linesrepresenting the four variables meets, and this creates the ability for emergent features.Empirical results found that this configural display could support both high-level constraints ifcritical data relationships are reflected and low-level data if graphical elements are madeperceptually more salient. This means that this configural display allows for the low-levelconstraints involved in detection to be supported while at the same time supporting the high-leveltask of fault diagnosis.Two more examples of configural displays are the rankine cycle display (Vicente, Moray,Lee, Rasmussen, Jones, Brock, & Djemil, 1996) and a configural display developed by Reisingand Sanderson (1998) that helps operators control a milk pasteurizing plant. The rankine cycledisplay is a display that embeds the domain constraints of the thermodynamic process of energymassconversion known as the rankine cycle. The cycle includes water being converted to steam,which then releases mechanical energy, which then is converted back to water to start the processagain. The display represents the degree of subcooling and other thermodynamic plant functionsin a configural form intended to enhance information extraction. Reising and Sanderson’sdisplay for the milk pasteurizing plant was designed to accurately represent the thermodynamicequations that underlie the heat exchange process. This representation was achieved throughadapting an energy balance display and through inventing a heat exchange display. The displaywas intended to aid the operator in the process control environment, but was not evaluated in thiscontext.2.2.2.4 Common Traits of Integrated DisplaysThe three integrated displays discussed (close spatial proximity displays, object displays,and configural displays) all have similar traits as a result of their ability to integrate multiplesources of information. Consequently, the effects on operator performance for each displayformat will likely be a function of the relationship between task proximity and display proximity.Task proximity refers to the degree in which multiple sources of information must beintegrated in order to perform a task or understand a process (Wickens & Andre, 1990; Wickens& Carswell, 1995). A task with high task proximity requires integration of separate sources ofinformation while a task with low task proximity requires focused attention upon or independentprocessing of those sources. In the case of power systems, an example of a task with low taskproximity would be the detection of a single voltage value that is out of a tolerance range. Thisinvolves a focused retrieval of information from one source. An example of a task with high task12

proximity would be diagnosing which capacitor will correct a voltage value problem since thisrequires integrating information across voltage values, buses, and location of capacitors.As was seen in the discussion of integrated displays, display proximity can be defined interms of either physical metrics or objectness (Wickens & Andre, 1990). The close spatialproximity display creates high display proximity by physically locating separate sources ofinformation together as was seen in the example of the integrated one-line diagram where busesand their voltage values are placed next to each other. The object display capitalizes on thefeatures of an object to create high display proximity. As explained in the section on objectdisplays, objectness refers to the combination of two or more distinct dimensions into a singleobject that represents the two dimensions in an integral format (Barnett & Wickens, 1988;Carswell & Wickens, 1987).2.2.2.5 Proximity Compatibility PrincipleIn the designing of integrated displays, it needs to be determined where display elementsshould be located given their relatedness to other displays. The proximity compatibility principle(PCP) is one guideline that serves to glue all integrated displays together. The proximitycompatibility principle (PCP) has been closely examined and researched over the years (Barnett& Wickens, 1988; Bennett & Flach, 1992; Wickens & Andre, 1990; Wickens & Carswell, 1995).The principle states a relationship between task proximity and display proximity. The PCPasserts two basic display guidelines. If a task requires high processing proximity (taskproximity), then there should be high display proximity. Conversely, if a task requires lowprocessing proximity (task proximity), then there should be low display proximity. The moreintegrated information sources must be to perform a task, the more beneficial it will be to presentthose sources close together. This implies that integrated displays better support dividedattention tasks such as trend analysis or any other task where multiple sources of informationneed to be used.The predictions of the proximity compatibility principle have been investigated andsupported in varying contexts and have shown specific benefits for integrated displays. Theproximity compatibility principle has generally been well supported and the benefitsreproducible. In other research, Carswell and Wickens (1988) reviewed the literature oncomparative graphics. Of particular interest in this review were studies that compared differenttypes of integrated graphic formats such as line graphs and bar graphs. The review stated thatthe proximity compatibility principle generally applies well to graphs as is predicted by theprinciple. Since graphical displays integrate sources of information, they do support integratedtasks better than focused attention tasks.In another graph reading study, Goettl, Wickens, and Kramer (1991) have shown that themerits and costs of integral displays compared to separable displays depends on the task beingperformed. Their experiment required subjects to integrate information on speed and accuracydata for hypothetical experiments. Performance for this inference task was better when the datawere presented as two points or objects in a two-dimensional speed-accuracy space than as fourseparate bar graphs or objects presenting speed and accuracy for each condition separately. Incomparison, bar graph displays showed superiority when subjects were asked to perform a task13

calling for precise judgment of one specific value. Other studies have found similar results(Gillie & Berry, 1994; Hollands & Spence, 1992).Studies in process control have shown support for the proximity compatibility principleas well. Woods, O’Brien, and Hanes (1987) have observed that increasing display proximitythrough the use of lines on nuclear power control boards allows groups of indicators to beprocessed together. Another example was configural display created by Vicente, Moray, Lee,Rasmussen, Jones, Brock, and Djemil (1996) that supported better detection and diagnosis ofproblems in a nuclear power plant due to its integrated design and features in a rankine cycledisplay. In an experiment by Jones, Wickens, and Deutsch (1990) that simulated the processcontrol in a chemical processing plant, evidence was found for the proximity compatibilityprinciple in that integral displays had a consistent performance advantage over separable displaysfor tasks that required complete integration of information.The predictions of the proximity compatibility principle are not all positive for integrateddisplays. One of the greatest costs predicted by the proximity compatibility principle is theproblem of getting exact values or readings from integrated displays (Carswell & Wickens, 1987;Goettl et al., 1991). Integrated displays are generally recognized for their ability to help identifytrends in data, but they are often less well suited when a specific value or piece of data is needed.This trouble with focused attention within integrated displays has specific implications for howto design these displays. If it is important to know precise values or specific pieces ofinformation, separable displays would most likely be more suitable. However, Bennett, Toms,and Woods (1993) suggested that the costs in focused attention attributed to displays with closespatial proximity could be offset through careful design of displays. One technique to supportfocused attention in integrated displays is to make low-level features more salient perceptuallythrough emphasis of scale, spatial separation, and color-coding. Wickens and Andre (1990)demonstrated this technique through the use of color borders in an object display. The use of anobject display improved information integration but disrupted focused attention. The addition ofcolor borders to the object display restored focused attention accuracy with only a slight cost toresponse time. Only by closely examining and evaluating a task in terms of what types ofinformation are most important can a designer know when to use separable displays and whenmore integrated displays are necessary.In summary, the proximity compatibility principle makes predictions as to which taskswould be best supported by integrated displays. Integrated displays are better suited for taskswith high task proximity. The task of diagnosing faults in process control domains would havehigh task proximity, therefore this task would best be supported by an integrated display. Faultdetection in process control domains has a low task proximity. Since focused attention isrequired for this task, more separable displays would better support this task.2.3 Color CodingColor coding has great aesthetic appeal when designing displays. The use of colors indisplays is something that can add benefits to performance when used properly, or incur costs ifnot used carefully. Different display conditions produce varying amounts of benefits. Thisimplies that the individual needs and requirements that a display must satisfy may be served14

differently by the use of color. Careful planning is necessary to optimally employ color in adisplay.2.3.1 Benefits of ColorChrist (1975) found strong benefits of color after reviewing much of the literature oncolor coding and performing a meta-analysis. One of the most overall apparent findings was thatif color of a target is unique and known in advance for that target, then it would aid the processof identification and searching. This means that color is quite beneficial for rapid localizationand Christ suggests that the effect of color is superior to that of size, shape or brightness oftargets to be identified or searched for.A similar finding was revealed in work by Fisher and Tan (1989). Their work exploredthe use of color as a highlighting mechanism. The intent of highlighting is to attract attention toan area to aid in search and identification of targets. Fisher and Tan were able to showempirically that color was superior to boxing, blinking, and reverse video as a means ofhighlighting as long as the color is highlighting an important target.Another aspect of color that has been seen as a great benefit is that some colors tend tohave well established population stereotypes (Wickens & Hollands, 2000). Within a givenpopulation, colors have come to represent information and meanings. For example, westerncultures tend to recognize the color red as symbolizing warnings, danger, or stopping. The colorgreen is typically used to symbolize safety or to “go”. A person who uses a display for the firsttime would likely associate the color red or green to have one of these meanings without beingexplicitly told so. Road signs and stop lights capitalize on these color stereotypes to provideredundant information. Stop signs and stop lights use the color red. These population stereotypesprovide a type of guideline to use when implementing color in a display.As was seen earlier in the discussion about the proximity compatibility principle, colorwas one way to manipulate display proximity. Two or more display elements can be linkedtogether spatially by using a common color scheme. For example, Wickens and Andre (1990)found that identical colors for two objects in a display can create a display proximity effect thatbenefits tasks that require integration of information. This feature was included as a physicalmetric of display proximity, an alternative to the better known attribute of physical distancebetween display elements that is more commonly associated with the proximity compatibilityprinciple. If multiple bar graphs are included in a display but only two of them need to becompared to each other or integrated, portraying the two bar graphs in the same color wouldserve as a means to group them and integrate them without changing their position in space.Color has also been shown to provide a benefit in terms of redundant coding.Redundancy in this case refers to more information than is minimally needed to identify a targetis provided (Christ, 1975). Color can be used quite effectively as a redundant feature to shape.If in a display circles are always green, squares are always red, and triangles are always yellow,then color is serving as a redundant attribute. Color alone would be enough to give cues as towhether an object is a circle, square or triangle. It may be necessary to inform users on how theredundancy plays an effect so that they can take advantage of it. Backs and Walrath (1995)15

found that subjects performing a visual search task were faster using displays with relevantredundancy if they were told of the redundancy.2.3.2 Costs of ColorCosts and limitations with the use of color have also been found in research. Thefrivolous and careless use of color for purely aesthetic reasons can often produce detrimentaleffects on the tasks that displays are meant to enhance. Following are a list of known costsassociated with the use of color coding in displays.The relative effects of absolute limits of judgment place a limit on the number of colorsthat should be used in a display. Absolute judgment refers to the ability of an observer to place astimulus in one of multiple categories along some type of sensory dimension (Wickens &Hollands, 2000). A task of absolute judgment for color would be to place five different hues ofthe color in order from darkest to lightest. Absolute judgment experiments have shown typicallythat errors begin to be made in discriminating when five or six different stimuli exist. Carter andCahill (1979) found that this applied specifically to color as well when colors represented a valueor meaning. This provides a general display guideline that no more than five or six colors shouldbe used to represent values that must be categorized or else costs may be incurred. Other factorsthat specifically affect absolute judgment of color include glare and low illumination (Stokes,Wickens, & Kite, 1990). Occurrences of either of these factors decreases a user’s ability todiscriminate between colors and essentially lowers the overall absolute judgment limits.A further cost associated with color is that no natural continuum exists. If users are askedwhether the color blue is less or more than the color yellow, there would probably not be aunified consensus (Wickens & Hollands, 2000). This feature has implications of what types ofinformation may be best suited for color coding.Although it has been noted that color can be used to create a display proximity effect,limitations may exist. One potential problem that was found using color to create close displayproximity is that the shared identity of color may inhibit the ability to focus attention on onesource of information while ignoring the other (Wickens & Andre, 1990). This relates to theproblem of focused attention in integrated displays discussed in the previous section.It was mentioned earlier that population stereotypes can create benefits by associatingcolors with common meanings. This can also be viewed as a cost. Erroneous use of colors maycreate conflicting meanings. Take the power systems domain for example. A display could bemade to represent voltages that are below a certain voltage threshold by surrounding the lowvoltage area with a red contour. One possible meaning of the color red is to indicate heat andwarmth. However, power systems operators understand that as voltage values decrease, the heatgiven off also decreases. There now exists a conflict in meaning. The voltages in a displaycould drop and cause a red contour to form which may be associated with heat instead ofcorrectly indicating that the heat given off has been reduced. Another power systems example isthe red and green colors used for circuits. A red switch indicates that a circuit is closed while agreen switch indicates that a circuit is open. However, red may be associated with abnormal andgreen as meaning normal. It could be interpreted that a red switch for a closed circuit is an16

abnormal problem. It is important to understand the natural associations and the context of thetask when deciding on the use of color.2.4 Display Arrangement and Visual SearchThe final human performance issue that needs to be addressed is visual search within thetask of process control. Characteristics of visual search have been identified in the literature andrelevant issues will be discussed here in terms of their implications for the design of displays andhuman performance on focused and divided attention tasks.Visual search is an important process for the detection of faults in process control. Faultdetection involves both selective and focused attention on an appropriate stimulus source. Visualsearch in displays normally occurs in a serial sequence where each item is inspected in turn(Neisser, Novick, & Lazar, 1964; Drury & Clement, 1978; Treisman & Gelade, 1980).Therefore, as the number of elements in a display increase, the search time increases as well.The importance to process control here is that a fault will take longer to detect when there aremore items to be scanned.However, there is often a tradeoff between the number of items to scan versus thedistance between the items to scan (Drury & Clement, 1978; Teichner & Mocharnuk, 1979).Items that are spaced out at a wider distance do slightly increase search times. However, if thedistance between items is small and there are multiple items, a clutter effect is observed. Thescanning time increases slightly for items placed too close together. What this means fordetection of faults in process control displays is that a designer should not place items to bedetected to far apart, but also needs to beware of placing them too close together and creatingclutter.Target searching is driven in part by the expectation of where a target will be located(Wickens & Hollands, 2000). The areas where targets are most likely to be found are fixatedfirst and most frequently. Within process control displays, the expectations of where faults willbe displayed can be used to decrease search times if the faults appear in a localized area. If afault is expected, visual search may begin in the localized area first since that is where mostfaults are located and this will reduce visual search time.Visual attention is drawn to items that have high salience. Salience refers to the ability ofan item to attract visual attention. Two important types of salience are color and highlighting.Fisher and Tan (1989) found that color can be used to increase the salience of an item throughhighlighting. Other forms of highlighting such as boxing, blinking, and reverse video could beused to increase salience, but were less superior to color. Highlighting techniques, such as theuse of color, can be used in process control displays to draw attention to important items. In thetask of fault detection, highlighting fault items should enhance detection time by drawing theuser’s attention to the item.Perceptual organization also affects visual search. Basic Gestalt principles such asproximity, similarity, common fate, good continuation, and closure help to preattentively groupitems together (Palmer, 1992). Due to high redundancy, displays using these gestalt principlesallow an accurate guess of the location of other items once one display item is located (Garner,17

1974). This reduces visual search time. A general tendency that also is worth noting is thatusers tend to search from top to bottom of menus and displays (Somberg, 1987). Theseperceptual organization guidelines can be employed in process control displays to reduce visualsearch time for a fault detection task as well. Organizing items into related groups helps the userto detect problems.Some aspects of divided attention require visual search. When divided attention isnecessary, parallel processing occurs (Wickens & Hollands, 2000). Divided attention andparallel processing are important in high demand environments such as in process control. Inparticular, fault diagnosis is a process that usually requires information integration from two ormore indicators. This integration can be supported by divided attention. Emergent features, forexample, allow for the parallel processing of attributes, such as is the case when three bar graphsare lined up next to each other. The tops of the three bar graphs can be processed in parallel bylooking at the imaginary line connecting the tops of each one. However, the integration in adiagnosis task may also involve attentional switching and visual scanning between indicators. Inthis case, both spatial proximity and good perceptual organization are beneficial for searchingand finding relevant pieces of information.One problem of particular relevance is the tradeoff of divided and focused attention(Wickens & Hollands, 2000). The failure of focused attention in successful divided attention isan example of this tradeoff. The perceptual organization issues described as part of the sectionon selective attention also pertains to divided attention and the task of fault diagnosis in processcontrol as well.2.5 SummaryRevisiting the traditional displays and the new proposed displays to be used in powersystems operations reveals some new insights. The static map board currently used by powersystem operators is a type of mimic display. The physical representation of a power grid systemis mapped onto the static map board. The actual state of the power system is contained in asingle tabular display.In contrast, the newer integrated one-line diagram is a combination of these two types ofdisplays. It is part mimic display, in the fact that the one-line diagram is a physical representationof a power grid system, similar to the static map board of the traditional display. However, theintegrated one-line diagram display also integrates the tabular information into the mimicdisplay. Buses and their relevant voltage information are located spatially close in this displayand connections between buses are visualized with electrical lines. Therefore, integrated onelinediagrams are also close spatial proximity displays.The question now is how to best design the new displays, and to determine whether ornot the new displays are in fact better than the old. All of the human factors issues listed abovehave important implications for how power systems data should be displayed to improve andenhance performance. However, in attempting to formally assess visualization effectiveness atleast two issues must be considered. The first issue is determination of the task. That is, what isthe task that the user needs to accomplish. Of course in the area of power system operations andanalysis there are a wide variety of different tasks. The tasks of the system planner are18

substantially different from the tasks of the system operator, which are in turn quite differentfrom those of the marketer or the engineer making presentations to regulators. In this experimentthe task focused on was the traditional process control task that required both detection (focusedattention) and diagnosis (integration) of system failures.The second issue is how closely the experimental setup matches what actually occurs inpractice. The goal in the setup of this experiment was to provide results that would help todesign the next generation displays for use by either electric utility operators or operationalplanning engineers. Therefore in the contextual setup of the experiment the goal was to at leastroughly approximate a utility control environment in which an operator responds to a systemvoltage disturbance, such as the one faced by the Delmarva Power and Light (DPL) operators onJuly 6 th 1999 during which their system experienced a near voltage collapse following the loss ofthe Indian River 2 generator (Interim Report of the U.S. Department of Energy’s Power OutageStudy Team, 2000).To completely replicate such a situation, and then test the response of the operators tovarious system visualizations would be quite difficult because of the complexity of powersystems control. In the initial stages of the research project reported here the experimental setupwas admittedly of lower fidelity than that in a control center environment, in which experiencedoperators or engineers monitor and control a system of which they are quite familiar usingdisplays they have used for years. Nevertheless, the experimental setup used in this experimentshould be viable and a good platform for evaluating future enhancements proposed forimplementation into the operation setting.Given these considerations, the question of whether the new display formats show thebenefits predicted by the human factors research were investigated by conducting an experimentwhere participants had to use one of three display formats to solve problems in power systemgrids. The first display simulated the tabular information and static map board that real lifepower system operators currently use. The second display (similar to Figure 1-1) utilized anintegrated one-line diagram display format and the third display (Figure 1-2) utilized anintegrated one-line diagram display format with a color contours enhancement. The first taskthat participants had to do was detect and acknowledge voltage problems in a power grid systemwhen they occurred. This task had low task proximity since focused attention was required toread individual voltage values. The second task that participants performed was to correct orsolve the voltage problems by opening and closing capacitors. This task had high task proximitysince information needed to be integrated, and hence ideally attention was divided betweencapacitor location information and bus voltage value information. These sources of informationrequired integration to perform the solving task.The proximity compatibility principle predicts that the integrated displays should bebetter at the high mental proximity task of solving problems in a power system, but they may beworse in the focused attention task of finding all voltage value violations. It also predicts that thetabular display and static map board should lead to worse performance for subjects performingthe high mental proximity task of solving. The tabular display and static map board shouldbetter support detecting/finding all voltage problems in a power grid since the perceptualorganization of buses and voltages into a single column is predicted to decrease visual searchtime. The location of violations displayed in the integrated one-line diagram display are not19

organized as well perceptually, and visual search time will likely be slowed. Color contoursmight mediate the effect on search times by highlighting specific buses with problematic voltagevalues. This highlighting may serve to facilitate focusing attention on voltage values andimproving performance on the low task proximity task of detecting and acknowledging.3. METHOD3.1 ParticipantsTwenty-eight participants, 23 men and 5 women, were recruited from Power Systemsclasses in the College of Electrical and Computer Engineering at the University of Illinois,Urbana-Champaign. All participants were required to have completed, or at least be currentlyenrolled in, one class in Power Systems. The median number of Power Systems classes takencurrently or in the past by the participants was 1.5 classes, with a range between 1 and 10classes. The median age of participants was 22, with a range from age 19 to age 31. Participantswere paid $8.00 an hour for their voluntary participation.3.2 Computer Software and ApparatusA modified version of the PowerWorld® Simulator software was used for theexperiment. The software allowed for buses, capacitors, per unit voltages, and electrical lines tobe configured and represented within the power system simulation used in this study. Thesoftware was run on a Dell Optiflex 800-Mhz computer with 128 megs of RAM and a 20-inchTrinitron monitor. A mouse was used to input and progress through the experiment.3.3 Power System Simulation and TaskThe simulation represented a 30-bus power system network (see Figure 3-2). During thesimulation, the initial voltage at each bus was programmed to be within acceptable limits of 0.95PU or above. Periodically, however, “contingencies” or voltage violations would occur at one ormore buses in the system, causing the PUs to drop below acceptable levels. The low voltage textvaluesthen turned from black to red and an audible alarm sounded, which was a repeatedbeeping from the computer.Participants were required to perform two consecutive tasks after a contingency hadoccurred. First, they were required to identify the particular locations of the buses experiencingvoltage violations. The voltage violations were identified or acknowledged by clicking directlyon the red voltage value text, which then turned to green. Upon the acknowledgement of allviolations, the audible alarm was terminated and the second task began. The second task was tocorrect the problem causing the contingency and thus restoring the voltages to acceptable values.The voltages were corrected by switching one or more capacitors to either their open or closedpositions. Capacitors were numbered and could be either activated or deactivated by clicking themouse directly on the capacitor information. The capacitor that is electrically closest to thevoltage violations needs to be activated to correctly solve a contingency. The identifying andacknowledging task requires focused attention of voltage values while the solving task requiresdivided attention between capacitor location and bus voltage values to integrate the information.20

3.4 Display ConditionsParticipants in the present study completed the experimental task using one of threedisplay conditions. These were (1) a separate tabular and one-line diagram (Figure 3-1), (2) anintegrated one-line display (Figure 3-2), and (3) an integrated one-line display with colorcontouring (Figure 1-2).3.4.1 Separate Tabular and One-Line DiagramInformation was displayed across two separate formats. The first was a static paper copyof a one-line diagram representing the 30 bus system, similar to the static system board used bymost operators (see Figure 3-1). The one-line diagram graphically represents the physical layoutof a power grid and includes the location of components such as buses and capacitors. The otherdisplay was a tabular list of numeric information about each of the system buses and capacitorsthat was presented on the computer screen. In particular, the tabular display presented threepieces of information (see Figure 3-1). On the left of the display, bus numbers were listed fromtop to bottom in ascending order. Next to each bus number, the PU voltage value was listed. Tothe right of this information, was a separate column listing all the numbers of the capacitors inthe system, which were listed from top to bottom in ascending order. A status indicator waslocated next to each capacitor number and reflected the status of the capacitor as either “open” or“closed”. The column located on the far right of the display was the line outage indicator. If apower line was not working, this was indicated by listing the two buses between which the linethat was out. For example, if the power line between bus 26 and bus 22 was out, the line outageindicator would list bus number 26 under the “From Number” column and list bus number 22under the “To Number” column.Voltage violations were acknowledged using this display by clicking directly on the redvoltage values until the cells of all voltage violations turned green. The voltage violations werethen resolved by identifying the location of the violations on the static paper one-line diagramand then using information in the tabular display to toggle between the open and closed status ofa capacitor until PUs were reestablished within acceptable parameters.3.4.2 Integrated One-Line DisplayThe integrated one-line display presented the one-line diagram of the 30-bus system onthe computer and incorporated the tabular information into the display. The bus voltage valueswere listed beside the associated buses and their voltage values changed dynamically as thesystem status changed. In addition, when an electrical line between two buses went out, the linebecame dashed on the display and the values turned to red.As can be seen in Figure 3-2, close spatial proximity is achieved between the bus and itsrelated voltage value by placing the two next to each other. This was intended to reduce visualsearch time for retrieving voltage values for buses. The close spatial proximity also aided inlocating the nearest capacitor to a bus since the electrical lines acted as a connection between thetwo elements.21

Figure 3-1. Tabular Display280.97 pu50.98 pu21.00 pu1151.01 pu180.96 pu 0.97 pu70.96 pu140.96 pu40.96 pu30.97 pu190.98 pu80.96 pu60.97 pu12130.97 pu1.00 pu90.97 pu 110.96 pu160.96 pu170.96 pu260.99 pu100.98 pu251.01 pu221.00 pu210.98 pu240.99 pu200.98 pu231.00 pu2729 301.00 pu 0.99 pu0.98 puFigure 3-2. Integrated One-Line Diagram Display22

Voltage violations were acknowledged by clicking directly on the red voltage value ofthe bus until all voltage violation values turned green. To solve the voltage violations, capacitorscould be opened or closed by clicking directly on a capacitor circuit breaker. The capacitorcircuit breaker was a small square button located on the line nearest the capacitor that it mostdirectly affected. The square button was green when the capacitor was open and red when thecapacitor was closed.3.4.3 Integrated One-Line Display with ContoursThis display was similar to the integrated one-line display; however, it also had contourssuperimposed on the one-line diagram. Specifically, when a voltage limit violation occurred, theregion surrounding the bus experiencing the low voltage became shaded using a contour patternthat varied from dark blue to light blue to yellow in a discrete manner (see Figure 1-2 in theIntroduction). The size and the color of the contour was depicted according to the severity of thevoltage and its impact on nearby buses. The color key for the meaning of the color and contourswas shown on the bottom left-hand side of the display. Acknowledging and solving voltageviolations was identical to the procedure for the integrated one-line diagram condition.3.5 NASA-TLXFollowing the experiment, participants were given the NASA-TLX workload assessmentto evaluate the workload associated with the three displays. The NASA-TLX workloadassessment test generates workload value between 0 and 100 for six dimensions often associatedwith workload. The six dimensions are mental demand, physical demand, temporal demand,performance, effort, and frustration level. A rating sheet is provided that lists the six dimensionswith a Likert scale next to each of them. Participants rate the individual dimensions by circlingone of the 21 tick marks along the Likert scale. Each tick mark represents a multiple of 5starting from 0 to 100, where 0 is a low rating and 100 is a high rating for each dimension.3.6 Procedure3.6.1 Group Assignment and InstructionsParticipants began the experiment by completing the consent form and providing basicdemographic and educational information. They were then randomly assigned to a displaycondition. The three conditions were the separate one-line and tabular display condition (Tabulargroup, n = 9), the integrated one-line condition (One-line group, n = 10), or the integrated onelinewith contours condition (Contour group, n = 9). Participants in each group were thenprovided with specific instructions about the display and tasks that they were to complete as partof the experiment. Participants were asked to read through the entire instructions and inform theexperimenter when they were ready to proceed or ask questions if any came up. Theexperimenter reinforced the written instructions by verbally reviewing the main points of theinstructions. Points of emphasis included reminding the participants that voltage violations werebest solved by clicking on the capacitor that is electrically closest, and that each trial could besolved by closing one capacitor although two or more could be used. In addition they wereasked to be as quick and as accurate as possible when doing the tasks.23

3.6.2 Phase 2: TrialsParticipants in each condition were presented with the power system information usingthe display format associated with their group assignment. However, for each of the participantsacross all conditions, participants were administered six practice and seventeen experimentaltrials and the same power system grid was used for the three conditions, which did not changefrom trial to trial. One or more low voltage violations occurred on each trial. After a randomlychosen interval between 5 and 15 seconds, a contingency occurred, which was signaled by thealarm and the changing of values of the affected buses from black to red. The same busesincurred voltage violations across the three display conditions during each trial, but the busesthat incurred voltage violations changed from trial to trial. Participants were allowed as muchtime as needed to acknowledge and solve the voltage violations. After all the voltage violationsfor a trial have been successfully corrected, a pop-up screen informed the participant of thesuccessful solution. Participants then proceed to the next trial by clicking “OK”. Participantscompleted a total of 13 testing trials.3.6.3 Phase 3: Post-Experimental Questionnaire and DebriefingAt the conclusion of the three conditions, participants were administered a post-experimentalquestionnaire which included the NASA-TLX workload assessment evaluation. Aftercompletion of the NASA-TLX, participants were given a debrief sheet explaining the generalpurpose of the experiment. This concluded the experiment.4. RESULTS4.1 Dependent MeasuresSeveral dependant measures were recorded in this experiment. The time required forparticipants to acknowledge all violations was recorded for each trial. In addition, the order inwhich violations were acknowledged was recorded for purposes of identifying any possiblestrategies that participants may have employed while performing this task. The time to removethe contingency, or resolve the voltage violations after the violations had been acknowledgedwas also recorded. The efficiency of using the various displays in resolving the violation wasassessed by capturing the number of capacitors that were switched on or off during the solutionphase of each trial. All times are presented in seconds. Task-order errors were also ascertainedby recording the number of times participants had attempted to solve the voltage violationsbefore they had properly acknowledged all of the violations. Finally, scores on the NASA-TLXwere obtained to examine the effects of display format, if any, on subjective workload during theexperiment.4.2 Acknowledgement Task4.2.1 Reaction TimesAs stated earlier, acknowledging is a task that requires focused attention. Since bus andvoltage information are perceptually organized in a column, visual search models predict that thetabular display should better support the acknowledgment task than either the integrated one-linediagram displays. The integrated displays do not present all violations in a fixed location. The24

color contours were expected to possibly aid the user in the acknowledgment task byhighlighting where the violations are occurring.Acknowledgement times were examined as a function of the number of voltage violationsthat occurred during the trial. A distinction was made between the 7 trials that had only onevoltage violation and the 6 trials that had more than one voltage violation. This examinedwhether the predictions of visual search models held true for different levels of task complexityfor the acknowledging task. In particular, response times were analyzed using a 3 (Group:Tabular vs. One-line vs. Contour) X 2 (Violation: Single vs. Multiple) Analysis of Variance(ANOVA). Scheffe post-hoc tests were conducted to explore significant differences acrossgroups.The results of the analysis revealed a significant main effect of display group,F (2,25) =9.70, p

with multiple violations (M=3.316, SD=.133). However, this main effect was also qualified bythe significant interaction. Specifically, the time to acknowledge violations increased more forthe multiple violations in the contour group than it did in the other display conditions.Nonetheless, the differences between trials with a single violation and trials with multipleviolations were significant for each display format; the tabular group (t (8) =-12.27, p

evealed a significant main effect for display condition (F (2,25) =297.0, p

Median # of Capacitor Closures per TrialMedian # of Closures1.51.41.31.21.110.9Tabular One-Line ContourSingle Violation TrialsMultiple Violation TrialsFigure 4-3. Median Number of Capacitor Closures per Trial4.5 Task-Order ErrorsTask-order errors occurred when participants attempted to solve the voltage violationsbefore acknowledging all of the violations. A task-order error could occur only once per trial, ifone occurred at all. The occurrence of a task-order error was rare, and the mean number ofoccurrences per subject and the variability was generally very small across all groups (see Figure4-4). Therefore, a non-parametric median test was used again in lieu of an ANOVA to examinethe number of task-order errors across display groups. The results of this analysis revealed thatthe number of task-order errors that occurred total in a condition differed between displayconditions, χ 2 (2) = 6.6, p=.037. More task-order errors generally occurred in the contour group(Md= 1, SD=.83) than in either the tabular group (Md=0 , SD=.33) or the one-line group (Md= 0,SD=.32).Order Violations Across GroupsMean # of Trials with OrderViolations10.80.60.40.20Tabular One-Line ContourFigure 4-4. Order Violations Across Groups28

4.6 NASA-TLX Workload ScoresThe average overall workload rating was 33.4 (SD=19.7) for the tabular display, 21.9(SD=21.9) for the one-line display, and 23.5 (SD=23.5) for the contour display. Figure 4-5shows TLX scores for the six dimensions that comprise the overall workload measure for each ofthe display formats. For every dimension except frustration level, the average workload scoresfor the tabular group were higher than either the one-line or contour group. The greatestdisparity between the tabular group and the one-line and contour group was for the temporaldemand dimension where the score was nearly twice the amount for the tabular group. A 3(Group: Tabular vs. One-line vs. Contour) x 6 (TLX Dimension: Mental vs. Physical vs.Temporal vs. Performance vs. Effort vs. Frustration Level) ANOVA was performed to look fordifferences in the amount of workload for the three different display types. However, nosignificant effects were observed.NASA-TLX Dimension ScoresTLX Score706050403020100TabularOne-LineContourMental DemandPhysical DemandFigure 4-5. Workload ScoresTemporal DemandPerformanceEffortFrustration Level5. DISCUSSIONThe current study was conducted to evaluate new visualization techniques for powersystems data. The integrated one-line diagram display was designed using some of these newvisualization techniques that resulted in a display that was both a mimic display and a closespatial proximity display. Color contouring was also examined as an enhancement technique.The performance effects of the integrated one-line diagram display were evaluated by comparingit to a separate tabular display and static one-line diagram in the context of two process controltasks. The first task, detection, required participants to locate voltage value faults in a 30 buspower system. The second task, diagnosis, required participants to integrate information aboutbuses and capacitors to solve the faults. Predictions about participants’ performance of thesetasks with either the separate tabular display and one-line diagram or the integrated one-line29

diagram display were derived from the human factors literature. The results will first bediscussed in terms of the detection task and then in terms of the diagnosing task.5.1 DetectionThe tabular display was predicted to improve performance for the detection taskcompared to the integrated one-line diagram display. This prediction was supported. Thetabular group showed that they were able to acknowledge violations quicker by about 1-2seconds. The speed advantage of acknowledging using the tabular format is best explained onthe basis of the visual search literature. The tabular display depicted bus and voltage valueinformation in column format and hence capitalizes on certain features of perceptualorganization. Since target searching is driven in part by the expectation of where a target will belocated (Wickens & Hollands, 2000), participants were most likely scanning the column wherethe voltage violations would appear. Due to the grouping of the buses and voltage values into asingle column, high redundancy occurred that allowed participants to accurately guess thelocation of other voltage violations once the first voltage violation was found (Garner, 1974). Incontrast, the integrated one-line diagram display lists the voltage values at buses placedsporadically on the screen which are meant in practice to spatially match the physicalrepresentation of the system. These buses of the integrated display were spaced out at widerdistances compared to the column format of the tabular display. The need to search across largergaps between buses served to increase the search times slightly in this type of display (Neisser,Novick, & Lazar, 1964; Drury & Clement, 1978; Teichner & Mocharnuk, 1979). As a resultvisual search time was reduced for the tabular display relative to the integrated display.The column format of the tabular display also led the participants to develop a dominantstrategy for detecting and acknowledging the voltage value faults. A top to bottomacknowledging strategy was used when voltage violations occurred, where participants begandetecting and acknowledging at the top of the column first and then scanned down through thecolumn. This strategy makes intuitive sense and can be explained within a visual searchparadigm. Visual search in displays normally occurs in a serial sequence where each item isinspected in turn (Drury & Clement, 1978; Treisman & Gelade, 1980). When searching throughmenus and vertical columns of data, the serial search often occurs from top to bottom (Somberg,1987). Since all the buses in the presented tabular display were perceptually organized by beingaligned in a vertical column, the easiest strategy for acknowledging violations would have beenstarting at the top and going down the list and clicking all the voltage values in red.The color contouring enhancements for the integrated one-line diagram display wereexpected to offset some of the cost in time for detecting and acknowledging associated with theintegrated display due to the contouring acting as a type of highlighting feature. This prediction,however, was not supported. In fact, the use of the color contouring led to worse detectionperformance on trials with multiple violations.Seeing multiple color contours when faults occurred might have been distracting to theparticipants and interfered with quickly detecting and acknowledging violations. Indeed,contours also did have some effect in terms of strategies used by the operators when detectingand acknowledging. The voltage violation with the largest contour was acknowledged first moreoften in the contour display than for the equivalent voltage violations in the tabular and one-line30

displays. This suggests that the larger contour did have some ability to attract the user’sattention to it first. The decreased performance on the detection and acknowledging task in thecontour group may be due to participants employing a strategy to find the worst violation first.Instead of acknowledging the first violation detected, they may have been waiting until the worstviolation had been detected and acknowledged that violation first.One explanation for the cause of this strategy of acknowledging the largest contour firstis the use of highlighting. The intent of highlighting is to attract attention to an area to aid insearch and identification of targets, and color can be used to do this (Fisher & Tan, 1989).Although this highlighting feature did not appear to offset the cost in time seen between thetabular display and the integrated display for detecting and acknowledging, the color contoursdid appear to draw and focus attention on problems in the power grid.Color contouring led to a surprising finding when examining the task-order violations.As described in the results, a task-order violation occurred when a participant attempted to solvefor voltage violations before acknowledging them all first. This equates to a failure in thedetection task. The occurrence of a task-order error was virtually non-existent for the tabular andone-line groups, but occurred almost once per subject for the experiment in the contour group.This observed effect may be due to attentional tunneling by participants. Mentionedpreviously, some participants in the contour condition did detect and acknowledge the worstviolation first. Participants using this strategy may have been too eager to acknowledge theworst violation first. This may be due to the fact that the worst violation for each trial had thelargest contour, and more attention was drawn to it due to its high salience. This type ofattentional tunneling may prevent users from recognizing that other voltage violations exist andthey instead go on to solve for the worst contour. Moray and Rotenberg (1989) found a similarbehavior in an experiment that simulated a process control task in a thermal hydraulic system.Their results suggested that operators prefer to work on only one fault at a time and that this‘cognitive lock-up’ hinders recognition of further faults. This could also explain why colorcontours were found to incur a cost in time only when acknowledging for multiple trials. Ifparticipants are indeed sometimes trying to solve before acknowledging because, as Moray andRotenberg suggested, they prefer to work on only one fault at a time, then the acknowledgingtime would increase as well, which was observed in this experiment. The acknowledging timewould increase because the simulation would not allow the solving task to commence until allviolations were acknowledged. A step would be added where it would be necessary to recognizethe error committed, and then go back to the acknowledging task.Color contours, however, could prove to be more useful in larger power systems. The30-bus system used in this experiment is relatively small and bus information management isfairly simple. Power systems with hundreds of buses such as real life power grids would bemuch more difficult in terms of information management. Contours in these larger powersystems may help to pinpoint the most serious problem areas for power system engineers toexamine more closely. Contours could be useful if the task for a user is to find the worstviolation first since it was found that the largest contour did have some ability to attract theuser’s attention to it first. Another possibility of how contours could enhance performance is indisplays that are more dynamically changing. The faults in this experiment were meant tosimulate problems that occur when lightning strikes a power grid. This created an instantaneous31

effect on the bus voltages that was then displayed. This means that one second a bus voltage waswithin prescribed limits, and the next it may not have been. Contours may provide additionalbenefits if the occurrences of faults are more gradual. For instance, a generator could slowly losepower output over time. This would cause bus voltages to slowly decrease. If contours werebeing used, the display may show a growing contour that darkens in color over time. This slowchange may cue the user in as to a potential problem occurring. Future testing with larger powersystems and more gradual occurrences of faults needs to be carried out in order to examine thesepredictions.5.2 DiagnosisIt was predicted that the integrated one-line diagram displays would lead to betterperformance for the high mental proximity task of diagnosing and solving problems in a powersystem. The tabular display and static map board was expected to lead to worse performance forthe diagnosing and solving task. These predictions were supported. The participants using theintegrated one-line diagram displays were on average 8-9 seconds quicker to solve than those inthe tabular group. The integrated one-line diagram display also supported greater efficiency inresolving the violations than the tabular display.As was predicted by the proximity compatibility principle (Barnett & Wickens, 1988;Wickens & Andre, 1990; Wickens & Carswell, 1995), the integrated displays of the one-line andcontour conditions did increase the performance of participants in the more integrated task ofsolving. As mentioned previously, the proximity compatibility states a relationship between taskproximity and display proximity. If a task requires high task proximity, then it should be bettersupported by high display proximity. If a task requires low task proximity, then it should bebetter supported by low display proximity. Task proximity refers to the amount of integration ofinformation necessary to do a task. Display proximity refers to the “closeness” of the sources ofinformation presented.The integrated one-line diagram display used in the one-line and contour condition is anexample of a close spatial proximity display because the bus and voltage values are located nextto each other and electrical lines show the capacitors that are electrically closest to each bus.This created a high display proximity that led to the improved performance for the integratedtask of solving by reducing the search and visual scanning between the relevant sources ofinformation for problem solving. These results also support the findings of other studies thathave also found that tasks that require more integration of information are more compatible withintegrated displays (Goettl, Kramer, & Wickens, 1986; Barnett & Wickens, 1988; Jones,Wickens, & Deutsch, 1990; Wickens & Andre, 1990; Bennett, Toms, & Woods, 1993; Gillie &Berry, 1994).The proximity compatibility principle predicts that the benefits of close proximitydisplays relative to distant displays for integration tasks will not be found for focused attentiontasks (Wickens & Carswell, 1995). Indeed these results were obtained and in fact it was foundthat focused attention tasks were better served by the non-integrated display. However, thebenefit was probably not due to more distant proximity costs of focused attention, but rather wasdue to better arrangement of the display that helped the ordered search for the faults.32

e evaluated for contingencies that dynamically occur. Although very few performance benefitswere found for contours in this experiment, perhaps they would provide benefits in a systemwhose state is slowly changing or where the tasks are different.Other steps in this type of research of visualization techniques for power systems data areto explore the use of 3-D displays and motion. These visualization techniques were mentionedpreviously but not examined in this experiment. Many additional human factors issues arisewhen implementing these types of visualization techniques. They need to be empirically testedas to what their effectiveness and limitations may be.ACKNOWLEDGMENTSThis material is based upon work supported by the Power Systems Engineering ResearchCenter (PSERC). Any opinions, findings, and conclusions or recommendations expressed in thispublication are those of the authors and do not necessarily reflect the views of PSERC.REFERENCESAndre, A. D. & Wickens, C. D. (1989). Information processing and perceptual characteristics ofdisplay design: The role of emergent features and objects. University of IllinoisTechnical Report ARL-89-8/AHEL-89-4. Savoy, IL: Aviation Research Laboratory.Backs, R. W. & Walrath, L. C. (1995). Ocular measures of redundancy gain during visual searchof colour symbolic displays. Ergonomics, 38[9], 1831-1840.Barnett, B. J. & Wickens, C. D. (1988). Display proximity in multicue information integration:The benefits of boxes. Human Factors, 30[1], 15-24.Bennett, K. B. (1993). Encoding apparent motion in animated mimic displays. Human Factors,35[4], 673-691.Bennett, K. B. & Flach, J. M. (1992). Graphical displays: Implications for divided attention,focused attention, and problem solving. Human Factors, 34, 513-533.Bennett, K. B. & Malek, D. A. (2000). Evaluation of alternative waveforms for animated mimicdisplays. Human Factors, 42[3], 432-450.Bennett, K. B., Payne, M., Calcaterra, J., & Nittoli, B. (2000). An empirical comparison ofalternative methodologies for the evaluation of configural displays. Human Factors,42[2], 287-298.Bennett, K. B., Toms, M. L., & Woods, D. D. (1993). Emergent features and graphical elements:Designing more effective configural displays. Human Factors, 35, 71-98.Buttigieg, M. A. & Sanderson, P. M. (1991). Emergent features in visual display design for twotypes of failure detection tasks. Human Factors, 33[6], 631-651.34

Carswell, C. M. & Wickens, C. D. (1987). Information integration and the object display: Aninteraction of task demands and display superiority. Ergonomics, 30, 511-527.Carswell, C. M. & Wickens, C. D. (1988). Comparitive graphics: History and applications ofperceptual integrality theory and the proximity compatibility hypothesis. University ofIllinois Technical Report ARL-88-2/AHEL-88-1/AHEL Technical memorandum 8-88 .Savoy, IL: Aviation Research Laboratory.Carter, R. C. & Cahill, M. C. (1979). Regression models of search times for color-codedinformation displays. Human Factors, 21[3], 293-302.Christ, R. E. (1975). Review and analysis of color coding research for visual displays. HumanFactors, 17[6], 542-570.Davies, D. R. & Parasuraman, R. (1982). The psychology of vigilance. London: Academic Press.Drury, C. G. & Clement, M. R. (1978). The effect of area, density, and number of backgroundcharacters on visual search. Human Factors, 20, 597-602.Fisher, D. L. & Tan, K. C. (1989). Visual displays: The highlighting paradox. Human Factors,31[1], 17-30.Garner, W. R. (1970). The stimulus in information processing. American Psychologist, 25, 350-358.Garner, W. R. (1974). The processing of information and structure. Hillsdale, NJ: Erlbaum.Gillie, T. & Berry, D. (1994). Object displays and control of dynamic systems. Ergonomics,37[11], 1885-1903.Goettl, B. P., Kramer, A. F., & Wickens, C. D. (1986). Display format and the perception ofnumerical data. Proceedings of the Human Factors Society 30th Annual Meeting , (pp.450-454). Santa Monica, CA: Human Factors Society.Goettl, B. P., Wickens, C. D., & Kramer, A. F. (1991). Integrated displays and the perception ofgraphical data. Ergonomics, 34[8], 1047-1063.Green, D. M. & Swets, J. A. (1966). Signal detection theory and psychophysics. New York:Wiley.Hollan, J. D., Hutchins, E. L., McCandless, T. P., Rosenstein, M., & Weitzman, L. (1987).Graphical interfaces for simulation. Advances in man-machine systems research, 3, 129-163. Greenwich, CT: JAI.Hollands, J. G. & Spence, I. (1992). Judgments of change and proportion in graphical perception.Human Factors, 34, 313-334.35

Interim Report of the U.S. Department of Energy's Power Outage Study Team. (2000). January,http://tis.eh.doe.gov/post/interim.pdf.Jones, P. M., Wickens, C. D., & Deutsch, S. J. (1990). The display of multivariate information:An experimental study of an information integration task. Human Performance, 3[1], 1-17.Kahneman, D. & Henik, A. (1981). Perceptual organization and attention. In M. Kubovy and J.R. Pomerantz (Eds.) Perceptual organization (pp.181-211). Hillsdale, New Jersey,Erlbaum.Martens, M. H. & Wickens, C. D. (1995). Lowlighting solutions to display clutter. University ofIllinois Institute of Aviation Technical Report (ARL-95-10/NASA-95-4), Savoy, IL:Aviation Research Laboratory.Mackworth, N. H. (1948). The breakdown of vigilance during prolonged visual search.Quarterly Journal of Experimental Psychology, 1, 5-61.Meyer, J., Shinar, D., & Leiser, D. (1997). Multiple factors that determine performance withtables and graphs. Human Factors, 39[2], 268-286.Moray, N. (1997). Human factors in process control. In G. Salvendy (Ed.) Handbook of humanfactors and ergonomics (pp. 1944-1971). New York: Wiley.Moray, N. & Rotenberg, I. (1989). Fault management in process control: eye movements andaction. Ergonomics, 32[11], 1319-1342.Mykityshyn, M. G., Kuchar, J. K., & Hansman, R. J. (1994). Experimental study ofelectronically based instrument approach plates. The International Journal of AviationPsychology, 4[2], 141-166.Neisser, U., Novick, R., & Lazar, R. (1964). Searching for novel targets. Perceptual and MotorSkills, 19, 427-432.Overbye, T. J. (2000). Reengineering the Electric Grid. American Scientist, 88, 220-229.Overbye, T. J., Klump, R. P., & Weber, J. D. (1999). A virtual environment for interactivevisualization of power system economic and security information. Proceedings IEEEPES 1999 Summer Meeting (pp. 682-687). Edmonton, Canada.Overbye, T. J., Wiegmann, D. A., Rich, A. M., & Sun, Y. (2001). Human factors analysis ofpower system visualizations. Proceedings 21st Hawaii International conference onsystem Sciences [6]. Kona, Hawaii.Palmer, S. E. (1992). Common region: A new principle of perceptual grouping. CognitivePsychology, 24, 436-447.36

Pomerantz, J. R. (1981). Perceptual organization in information processing. In M. Kubovy & J.R. Pomerantz (Eds.) Perceptual organization (pp. 141-180). Hillsdale, NJ: Erlbaum.Pomerantz, J. R., Sager, L. C., & Stoever, R. J. (1977). Perception of wholes and of theircomponent parts: some configural superiority effects. Journal of ExperimentalPsychology: Human Perception and Performance, 3, 422-435.Reising, D. V. C. & Sanderson, P. M. (1998). Designing displays under ecological interfacedesign: Towards operationalizing semantic mapping. Proceedings of the Human Factorsand Ergonomics Society 42nd Annual Meeting, 372-376.Sanderson, P. M., Flach, J. M., Buttigieg, M. A., & Casey, E. J. (1989). Object displays do notalways support better integrated task performance. Human Factors, 31[2], 183-198.Somberg, B. L. (1987). A comparison of rule-based and positionally constant arrangements ofcomputer menu items. Proceedings of CHI & GI '87 conference on human factors incomputing systems. New York: Association for Computing Machinery.Stokes, A. L., Wickens, C. D., & Kite, K. (1990). Display technology: human factors concepts.Warrendale, PA: Society of Automotive Engineers.Swets, J. A. & Pickett, R. (1982). Evaluation of diagnostic systems. New York: Academic Press.Teichner, W. H. & Mocharnuk, J. B. (1979). Visual search for complex targets. Human Factors,21, 259-276.Treisman, A. & Gelade, G. (1980). A feature-integration theory of attention. CognitivePsychology, 12, 97-136.Vicente, K., Moray, N., Lee, J. D., Rasmussen, J., Jones, B. G., Brock, R., & Djemil, T. (1996).Evaluation of a rankine cycle display for nuclear power plant monitoring and diagnosis.Human Factors, 38[3], 506-521. 1996.Vicente, K. & Rasmussen, J. (1992). Ecological interface design: Theoretical foundations. IEEETransactions on Systems, Man, Cybernetics, 22, 589-606.Wickens, C. D. & Andre, A. D. (1990). Proximity compatibility and information display: Effectsof color, space, and objectness on information integration. Human Factors, 32[1], 61-77.Wickens, C. D. & Carswell, C. M. (1995). The Proximity compatibility principle: Itspsychological foundation and relevance to display design. Human Factors, 37[3], 473-494.Wickens, C. D. & Hollands, J. G. (2000). Engineering psychology and human performance (3rded.). New York: Prentice Hall Inc.Wickens, C. D., Kroft, P., Yeh, M. (2000). Data base overlay in electronic map design: Testing acomputational model. Proceedings of the IEA 2000/HFES 2000 Congress , 451-454.37

Wilhelmson, R. B., Jewett, B., Shaw, C., Wicker, L., Arrott, M., Bushell, C., Bajuk, M., & Yost,J. (1989). A study of the evolution of a numerically modeled severe storm. InternationalJournal of Super Computer Applications, 4[2], 22-36.Wilson, P. N., Foreman, N., & Tlauka, M. (1997). Transfer of spatial information from a virtualto a real environment. Human Factors, 39, 526-531.Witmer, B. G., Bailey, J. H., Knerr, B. W., & Parsons, K. C. (1996). Virtual spaces and realworld places; Transfer of route knowledge. International Journal of Human-ComputerStudies, 45, 413-428.Woods, D. D., O'Brien, J. F., & Hanes, L. F. (1987). Human factors challenges in processcontrol: The case of nuclear power plants. In G. Salvendy (Ed.), Handbook of humanfactors. New York: Wiley.Woods, D. D., Wise, J., & Hanes, L. F. (1981). An evaluation of nuclear power plant safetyparameter display systems. Proceedings of the 25th Annual Meeting of the HumanFactors Society. Santa Monica, CA: Human Factors.Yeh, M. & Wickens, C. D. (2000). Attention filtering in the design of electronic map displays: Acomparison of color-coding, intensity coding, and decluttering techniques. (TechnicalReport ARL-00-4/FED-LAB-00-2). Savoy, IL: University of Illinois, Aviation ResearchLab.38