In-Vehicle Tasks: Effects of Modality, Driving Relevance, and ...

ABSTRACTThe current study was designed to assess the factors that could moderate the differencesbetween auditory and visual delivery of in-vehicle information, primarily as such informationaffected measures of driving performance, and secondarily as such factors would influence theprocessing of the in-vehicle information itself. In particular, we were interested in threemoderating factors: task relevance, task complexity, and the nature of the driving performancevariable (lane keeping versus hazard response). Finally, we were interested in how the use of aredundant combination of auditory and visual delivery could also moderate these effects.The results revealed that drivers generally protect lane keeping from interference, but donot protect their hazard response, replicating the results from the Horrey and Wickens (2002)study. However, drivers did not protect their hazard response from interference as secondary taskinformation was given priority when the messages were visual and relevant (they respondedfaster to these messages). Drivers exhibited better steering control when the messages wereauditory and relevant, supporting the idea that relevant, auditory messages leave the driver moreattuned to the roadway and thus they are less abrupt with their lane corrections. Increasedmessage complexity increased the auditory benefit to driving performance (steering velocity wasgreatest for visual, complex messages), but led to an auditory cost for in-vehicle task (message)performance. An interaction between relevance and modality suggests that the safety benefits forauditory delivery of in-vehicle information appear to be greater when the message is relevantthan when it is irrelevant. Redundant presentation of messages across both modalities sometimeshelped, relative to the worst of the two single-modality interfaces, but rarely did it appear to offerthe “best of both worlds,” supporting performance that was sometimes as good, but never betterthan either of the single-modality conditions. These results have implications for training driversto use redundant systems.1

Driving Information DisplaysDriving is a multitask environment which requires concurrent processing of information.The primary task of the driver is to simultaneously guide the position of the vehicle, detect andclassify potential hazards, and navigate the route. Secondary tasks compete for resources anddeteriorate the primary driving performance (Lansdown, Brook-Carter, & Kersloot, 2002).Examples of secondary tasks include in-cab viewing, roadside scanning, and auditory, cognitive,and motor tasks (Wickens, Gordon, & Liu, 1998). The presence of secondary tasks intrudes upondriving and diverts a significant amount of attention away from the driving task as to pose asafety hazard. Hughes and Cole (1986) quantify the amount of visual attention that is devoted tosecondary tasks as 30-50% of driver’s visual attention, and Antin et al. propose that driversspend a third of their time looking inside the vehicle (Antin et al., 1990). Appropriate taskprioritization and the influence of display-improving mechanisms do not necessarily help tomitigate the attentional decrements in driving performance that secondary task displays create(Noy, 1990). This realization highlights the need for offsetting the attentional and visualmonitoring costs of secondary task displays. A necessary step towards implementing informationdisplays is to thus explore the display manipulations that will afford significant reduction insecondary display costs to attentional resources. A resource model, proposed by Wickens (1980,1991, 2002) examines the effectiveness of time-sharing multiple tasks along a number ofdimensions; these dimensions have implications for display design.Multiple ResourcesThe multiple resource model predicts the relative benefits of different display types tomultiple task performance (Sarno & Wickens, 1995; Wickens, 2002). The model proposes thattwo tasks are better time-shared if they share separate resources than if they share commonresources. Tasks are categorized according to their demands along four resource dimensions:processing stage, perceptual modality, visual channel, and processing codes.The stage of processing dimension refers to the separate resources used for perceptualcognitiveactivity (perceptual processing) compared to response selection and action.Physiological research supports the separation of perceptual and response selection processes inthe stage distinction of the multiple resource model. Isreal et al. (1980) examined the eventrelatedbrain potential (ERP) measures in assessing task activity and found that perceptual andresponse processes had systematic differences in task workload. The stage dichotomy predictsthat there will be substantial interference between perceptual and cognitive processes involved inworking memory functions (Liu & Wickens, 1992) and in two response processes, such asdialing a cell phone and steering a vehicle (Serafin, Wen, Paelke, & Green, 1993), but lessinterference between two tasks drawing from separate resources across the stage-defineddichotomy. Drivers are able to concurrently scan the roadway (perceptual processing) whilesteering (response execution); however, the driving task is degraded if another manual task isperformed.The codes of processing dimension refers to the distinction between spatial and verbalprocessing resources; these processing codes employ separate resources associated with the twocerebral hemispheres (Polson & Friedman, 1988), and define different resources within each ofthe individual stages of processing. Considering response execution, manual responses (spatial)3

are efficiently time-shared with vocal outputs (verbal). Processing of spatial and verbalinformation employs different cognitive resources (Wickens & Hollands, 2000). The processingcode distinction is derived from Baddeley’s (1995) model of working memory in which thecentral executive component (an attentional control system) coordinates information from thephonological loop (verbal representation of information) and the visuospatial sketchpad (spatialrepresentation of information). The model of working memory identifies spatial and verbalinformation as separate components that activate separate brain structures and that independentlyrely upon the attentional mechanisms. Each of the two systems or codes, given their independentprocessing and separate resources, suffer more when a concurrent task requires the same type ofinformation processed in working memory than when information in the alternative code is beingsimultaneously processed. As the two codes use separate processing resources (Wickens, 1992),in controlled experimental evaluations, it is necessary to carefully design secondary tasks thatmanipulate modality to employ the same code of processing for the manipulation of modality tobe a pure comparison.The visual channel distinction differentiates the form of visual processing between focaland ambient vision. The two types of visual processing employ separate brain structures. Focalvision is used to recognize discrete objects and patterns and primarily involves use of the fovea.Ambient vision is used in continuous motion processing and guidance and involves use ofperipheral vision and to some degree fovea vision (Wickens, & Hollands, 2000). Weinstein andWickens (1992) manipulated display location for multiple flight displays. The results indicatedthat pilots performed more quickly and accurately using displays that employed separate visualresources than the same visual resource, particularly two displays requiring ambient vision.Within the context of the driving domain, drivers are able to guide the vehicle (ambient) whilereading roadway signs (focal); however, the ability to concurrently navigate with an in-vehicledisplay or process other detailed information on such a display, and to detect hazards whiledriving results in a competition for visual resources as both tasks employ focal vision.The processing modalities dichotomy, which will be the focus of the present research,refers to the visual and auditory input channels. Dual-task performance is improved when inputis distributed to the separate modality channels. Cross-modal time-sharing (between visual andauditory input) is more effective than intra-modal time-sharing (two auditory inputs or two visualinputs; Wickens, 2002). Wright, Holloway, and Aldrich (1974) found that participants werebetter able to concurrently perform a tracking task and to process verbal information wheninformation secondary to the tracking task was presented auditorially rather than visually. Whilethe multiple resource theory outlines the structural dichotomies of resources that help to accountfor the differences in time-sharing efficiency, the allocation of the available resources ismodulated by the characteristics of the task(s) and the allocation strategies of individuals.Resource AllocationThe success of time-sharing in the driving task depends on the total amount of attentionalresources available to the driver and the strategy of allocating those resources. The strategies ofresource allocation are affected by task properties and environmental goals. An increase in thedifficulty of a particular task will result in a decrement to the concurrent task unless the operatormaintains an unchanging allocation of resources, so the increasing-difficulty task suffers, orunless the concurrent task is data limited (through automaticity) (Wickens & Hollands, 2000). In4

a driving context, a study by Liu (2001) revealed that an increase in the amount of informationunits presented to a driver during a secondary task results in the subsequent increase in theamount of decrement to primary and secondary task performance. Further, the level ofimportance assigned to a particular task dictates the allocation strategy of attentional resources.An increase in the bandwidth of a particular task can contribute to a shift in task priority; as anexample, the increase in road curvature would potentially cause a shift of attention from an invehicletask to the roadway.The salience and expectancy of the information, the required amount of effort to obtainthe information, and the value of the task influence the allocation strategy of individuals(Wickens, Helleberg, Goh, Xu, & Horrey, 2001). Information that is more salient (conspicuous)carries a greater likelihood of being attended to and processed. Visual displays that are placed atwider eccentricities (and are consequentially less salient) require longer switching times anddegrade performance of tasks that are positioned on those displays. Wickens, Dixon, and Seppelt(2002) examined a paradigm with a primary compensatory tracking task paired with acomprehension side task; the results revealed that more eccentric displacements of the side taskinformation lead to delayed initiation and completion of the side task response, and increasedtracking error during side task response. Increased display separation increases the effortrequired to sample information from the different displays. Information that requires greatereffort to obtain is less likely to receive allocation of attentional resources (Wickens et al., 2002).The expectancy of information defined by the frequency or bandwidth of the informationpresentation also influences the allocation of attention. For instance, drivers may attend moreclosely to their following distance behind a lead vehicle if the braking response of the leadvehicle is frequent and erratic (conveying more information in terms of information theory).Finally, the value assigned to particular tasks modulates the level of attention that is given todisplay a source that supports the particular task; this subjective measure changes as a functionof the perceived cost of failing to attend to a particular task and to process the relevantinformation. For example, in nearly all driving tasks, the value of sampling the outside world isgreater than the value of sampling inside, because the consequences of failures to noticedeviations or hazards in the outside world are quite severe. These characteristics of a particularenvironment (salience, effort, expectancy, value) and the degree to which they are factors in aparticular information processing task dictate the allocation of attention between tasks.Additional resource allocation issues are relevant when considering the processingmodalities dimension (auditory vs. visual) and these include visual resource competition and theauditory preemption effect (Helleberg & Wickens, 2003). The majority of feedback the drivergets from driving is visual (according to Rockwell (1972) 90 percent). It would seem prudent toavoid in-vehicle devices that require use of visual processes, especially in high-workloadsituations. However, visual information is self-paced in that it allows drivers to delay attendingto the information until a time when they can safety remove their eyes from the roadway; thelimitations of working memory are alleviated with visual information. Auditory information incontrast, is generally forced-paced and requires the driver to perceive it immediately. It wouldrequire a repeat function in the equipment in order for the information to be recalled if it isforgotten (Sanders & McCormick, 1993).5

A further challenge to an auditory display is the auditory preemption effect which canreduce the benefit of employing the two modalities. This phenomenon occurs when thepresentation of discrete auditory information disrupts or preempts an ongoing visual task(Wickens & Liu, 1988). The transient nature of the auditory information and the limitation of theworking memory to retain extensive pieces of information, as noted above, may necessitate thatattention is promptly directed to the auditory information upon its presentation (Wickens, Dixon,& Seppelt, 2002); performance on the driving task is consequently sacrificed. Furthermore, theauditory preemption effect is also resultant from the attention-grabbing characteristic of theauditory modality (Sorkin, 1987; Spence & Driver, 1997). This disruptive effect of the auditorymodality appears to be particularly prominent in situations where the auditory presentation of theIVT task is complex and unexpected (Wickens, Dixon, & Seppelt, 2002). Auditory messages thatare more abrupt, less predictable, and longer tend to interrupt a continuous visual task to agreater extent than messages that are expected and short (Helleberg & Wickens, 2003; Wickens,Goh, Helleberg, Horrey, & Talleur, in press).The various attentional mechanisms described modulate the benefits and costs to use ofthe auditory and visual modalities in the driving task. The benefit of one perceptual modalityover another depends upon the design manipulations which may amplify the effect of a particularset of those mechanisms. The differential effect of auditory versus visual presentation of invehiclemessages on driver performance may differ as a function of the relevance of the invehicleinformation, task workload, and display separation.Auditory Presentation of In-Vehicle InformationThe use of the auditory modality helps to reduce the attentional and cognitive demandsthat are inherent in multiple concurrent processing situations. For example, a study by Folds andGerth (1994) examined a multiple process-monitoring situation where a tracking task and amonitoring task were concurrently performed. The results showed that auditory presentation ofwarning information led to faster and more accurate primary and secondary task performancethan visual presentation of the same information. Wickens, Dixon, and Seppelt (2002) examineda dual-task paradigm in which the primary task involved a manual tracking task and thesecondary task involved verbal read-back of digit strings. The experiment revealed similarresults as in the Folds and Gerth study (1994), showing a clear advantage of auditory side taskdelivery relative to visual delivery of side task information at wide eccentricities, for both theprimary and secondary task performance; participants responded more quickly to auditory thanto visual secondary task information, and tracking error decreased with auditory presentation ofthe side task relative to visual presentation.A good deal of research has compared auditory and visual presentation modalities of adiscrete task paired with a continuous visual task (e.g., Wickens, Sandry, & Vidulich, 1983;Braune & Wickens, 1985; Wickens & Goettl, 1984; Wickens, Braune, & Stokes, 1987; Yeh &Wickens, 1988; Isreal, 1980; Wickens, 1980; Tsang & Wickens, 1988; see Wickens & Liu, 1988,and Wickens,1980 for a review). The general conclusions drawn from these studies are thatperformance with the visual modality suffers from increased spatial separation from the primarytask (the continuous visual task); this visual cost gives an advantage to the auditory modality toexhibit better time-sharing than with the visual modality. However, minimal visual separationresults in a more ambiguous differential effect between the auditory and visual modalities. At a6

educed eccentricity, the auditory preemption effect is more likely to account for the differencesbetween the two modalities, penalizing the continuous visual task, but rewarding the discreteauditory task.The effect of task modality on time-sharing in dual-task situations has also beeninvestigated in high-fidelity domains. A number of studies in vehicle driving have examined theeffect of modality on vehicle control, guidance, and navigation performance. In general, thesestudies pair a side task with the primary driving task and examine the influence of side taskinformation modality on primary and secondary task performance. The primarily visual drivingtask is hypothesized to benefit from an aurally-delivered side task when the implications of themultiple resource model are taken into consideration. These studies show mixed results,however, as to whether there is an auditory benefit (better primary and secondary taskperformance using the auditory modality to present information as opposed to the visualmodality) or an auditory cost (better primary and secondary task performance using the visualmodality to present information as opposed to the auditory modality) to auditory presentation ofin-vehicle task information.Following the procedure used by Wickens and Seppelt (2002) we present the followingreview of driving studies in 3 categories: 1) those that showed auditory benefits in neither task,2) those that showed auditory benefits in one task, and 3) those that showed auditory benefits inboth tasks. We then try to infer common properties of the studies that lead to one of theparticular category of results that they show. To represent the collective effects of a set ofstudies, we first represent each study by a simple icon: a horizontal axis with two vertical linesemerging (see Figure I1 below). The degree of auditory benefit (up line) or cost (down line) isindicated by the vertical direction of the lines. The line on the left refers to benefits (or costs) ofauditory display to the primary driving task. That on the right refers to performance on the IVTitself. When there is neither cost nor benefit, it is indicated by a dot on the horizontal axis. Ifworkload was varied, its effects are represented by two vertical lines on either the primary (leftside) or secondary (right side) task. The slope of the vector connecting these lines thus representswhether increasing workload enhances the auditory benefit (or diminishes the auditory cost)sloping upward from left to right, or has the opposite effect, sloping downward. Finally, next toeach icon, we describe two key features that, we will see, appear to modulate the relative patternof effects: 1) whether the display of visual information is in a head-up (HUD) or head-down(HDD) location, and 2) whether the IVT contains driving relevant (Rel) or irrelevant (IRR)information.Example:RelHUDLo HiLoHiFigure I1. Example of icon figure.7

AV better than VV on NEITHER taskAn auditory cost can be seen in a number of studies that examine modality of in-vehicleinformation displays (Matthews, Sparkes, & Bygrave, 1996; Horrey & Wickens, 2002; Lee,Gore, & Campbell, 1999; Dingus et al., 1997). These studies show the visual modality to beequal to if not better than the auditory modality for measures of both primary and secondary taskperformance, hence providing results at odds with the predictions of multiple resource theory.Matthews, Sparkes, and Bygrave (1996) had participants drive a low-fidelity drivingsimulator (on a desktop computer); the roadway consisted of straight and curved sections. Areasoning task (irrelevant to the driving task), requiring subjects to asses whether simplesentences followed by a letter pair were true or false, was presented either visually orauditorially. The auditory items were generated as computerized speech; the visual items wereoverlaid onto the simulated driving scene (e.g., on road signs). Results showed an auditory costfor both primary and secondary task measures in the high driving workload condition. No dualtaskdecrement was observed for the visual display, suggesting that the focal vision required didnot disrupt the ambient driving task (e.g., Summala et al., 1996). The accuracy of the responsesto the IVT task for both modalities deteriorated as more priority was given to the driving task.Horrey and Wickens (2002) employed an experimental paradigm similar to that used inthe current experiment. They examined the effect of display clutter, separation, and modality ondriving and side task performance. Participants drove through a high-fidelity simulated drivingenvironment, consisting of urban, rural straight, and rural curves roads, and concurrentlyperformed a digit callout task (irrelevant to the driving task). Random strings of 4, 7, and 10digits were presented to participants either on an overlaid display (0 degrees below the horizonline), a head-up display (7 degrees below the horizon line), a head-down display (38 degreesbelow the horizon line in the center of the console), or aurally through a set of speakers locatedin the vehicle. Response times and accuracy were recorded for the digit task, and vehicle lanekeeping, speed control, and hazard event detection times were recorded for the driving task.Drivers generally protected the driving task regardless of display manipulation, showing noeffect on vehicle control, and no effect on hazard event detection in any condition except for thehead-down presentation in which the hazard response was slowed. Accuracy and responseduration to the digit task suffered with complex auditory information compared to visualpresentation (HDD and HUD) of the side task information. However, participants reacted morequickly to the side task, supporting a preemptive effect of the auditory modality (Helleberg &Wickens, 2003).Lee, Gore, and Campbell (1999) also used auditory information identical in form to thevisual information (text messages conveying driving status for different driving events). Thevisual display was located in the instrument panel directly in front of the driver, approximately21 degrees below the line of sight. Participants drove simulated routes in which they receivedmessages alerting them to the presence of varying events (relevant to the driving task)throughout the driving environments. Drivers were asked to acknowledge each informationmessage (or roadway warning) by pressing one of two steering wheel buttons in order to measureresponse time to attend to the in-vehicle messages. The messages were presented either with acommand or a notification style to test the effect of conveyed immediacy of the message ondriver response time. The primary driving task was protected across manipulations with vehicle8

control and guidance measures equivalent for auditory and visual displays; this is consistent withthe finding from Horrey and Wickens (2002). The effect of modality on side task (IVT)performance however showed an auditory cost with a smaller percentage of messageacknowledgement compared to visual messages; this auditory decrement was modulated bymessage style with command messages increasing the effect of the decrement.Dingus et al. (1997) examined the effects of modality of collision warning informationdisplays on driving performance in a realistic environment. The visual form of the warninginformation displayed a series of nine colored bars placed in perspective; as the distance betweenthe participant’s vehicle and the lead vehicle became shorter, new bars were displayed one belowthe other, increasing in number and changing in color from green to orange to red shades,depending on the urgency of the headway distance. The visual screen was mounted in thedashboard next to an analog speedometer and was located approximately 23 degrees below theline of sight. The same criteria that determined the urgency of the headway distance wasemployed in the auditory form of the warning information; voice messages signaled a shift in theurgency of the headway distances in the same way that color was used in the visual display. Aredundant condition included both the auditory and visual display formats. No significantdifferences across modality were observed for either primary or secondary task measures. Dualtaskdecrements however were observed for each modality condition compared to their singletask measures of headway-monitoring performance.The results from these studies, which are graphically summarized in Figure I2, suggestthat the benefit to auditory presentation of secondary task information may be mitigated byprocessing mechanisms and display characteristics. Four studies fell into this category ofauditory cost or no auditory advantage. In a distinction that will prove to be important below,two had irrelevant messages (Matthews, Sparkes, & Bygrave, 1996; Horrey & Wickens, 2002)and two had driving-relevant messages (Lee et al., 1999; Dingus et al., 1997). Also these twoirrelevant message studies found no visual cost (relative to auditory) when visual display washead-up. In general however, there appeared to be no clearly defining property that characterizedall four of these studies.9

Class I – Auditory Cost Primary Task Secondary TaskMatthews et al., 1996IRRHUDLo HiHorrey & Wickens, 2002IRRHUDLee et al., 1999RelDownDriving WorkloadLo HiDriving WorkloadRTAccuracyLo HiIVT ComplexityDingus et al., 1997RelDownFigure I2. Icon figure of auditory cost studies. (In Horrey and Wickens (2002), we only depictthe results from the HUD condition here.)AV better than VV on ONE taskBoth an auditory cost and an auditory benefit can be seen in a number of studies thatexamine modality of in-vehicle information displays (Mollenhauer et al., 1994; Labiale, 1990;Parkes & Burnett, 1993; Hurwitz & Wheatley, 2002; Ranney et al., 2002; Srinivasan & Jovanis,1997). These studies show the visual modality to be equal to if not better than the auditorymodality on either the primary or secondary task performance but the auditory modality to besuperior to the visual modality on one of the two tasks (primary or secondary), suggesting atrade-off between modality and the allocation of resources between the two tasks.Mollenhauer, Lee, Cho, Hulse, and Dingus (1994) examined the effects of modality andinformation workload of in-vehicle information systems on driver performance. Drivers wereexpected to process and adhere to road sign information presented throughout the simulateddriving task. The relevant road sign information included the current speed limit and roadnumber, and the information presented on the last displayed sign; this information was presentedeither auditorially or visually on an LCD display located on top of the simulator instrument paneldirectly ahead of the driver (approximately 11 degrees below the line of sight). The secondary10

information task consisted of two levels of task workload; the road sign information waspresented either at a rate that a driver would see signs during a normal drive (high workload) orat half the rate seen during a normal drive (low workload). Auditory presentation impaireddriving performance resulting in greater lane position deviation, steering wheel velocity, roadheadingerror, and speed deviation compared to driving performance with visual displays. Atradeoff in driving performance decrement was seen with an improvement to secondary taskperformance. Drivers recalled approximately 2 more information items with auditorypresentation than with visual presentation. However, independent of modality, higherinformation workload significantly decrease the number of recalled items.Horrey and Wickens (2002), as discussed earlier, found that drivers protected the primarylane-keeping task in driving, regardless of display manipulation. For the primary task measure ofhazard detection however, drivers noticed hazard events more rapidly in the auditory conditioncompared to the head-down visual condition, thus showing a greater level of driving safety forauditory displays. In contrast, auditory presentation of side task information resulted in greaterspeed variation than with the visual head-down display; though, the implications to theclassification of an auditory cost or benefit are inconclusive as speed control was not consideredan indicator of driving safety. Again, results showed an auditory cost for secondary taskmeasures of accuracy and response duration. A significant auditory advantage was evidenthowever for response times to the secondary task.Labiale (1990) had participants engage in real driving that was directed according to thenavigational information received through either the visual or auditory modality in verbal format.An auditory beep signaled the instance of each occurrence of a navigational message in bothmodalities. The length of the messages was varied from 4 information units to18 informationunits to examine the effects of message length on driving performance; the visual informationwas displayed for a limited amount of time, calculated to allow only one read-through in order tocompare to auditory constraints in phonetic memory. The visual display was located on the righthand side of the instrument panel and oriented towards the driver (approximately 23 degreesbelow the line of sight). Results did not reveal a main effect of modality of road informationmessages on vehicle and speed control performance. However, the manipulation of IVTworkload revealed some important interactions; at low IVT workload, the visual and auditorymodalities performed equally in measures of primary and secondary task performance. Theincrease in message length (IVT workload) however modulated the differential effect betweenmodalities for primary task performance with a benefit to using the auditory display seen in thehigh workload conditions, enabling more precise speed and vehicle control than in the highworkloadvisual conditions. This supports the notion that dual-task performance is affected bychanges in workload; the increase in workload of one task is offset with performance decrementsin the other task (Wickens, 2002). However, an auditory cost was seen in the high-workload IVTmessage condition for secondary task performance, with the decrease in recall performance inthe high workload condition relative to the low workload condition of a greater magnitude thanthat in the visual modality.Hurwitz and Wheatley (2002) analyzed driver performance as participants drove in ahigh-fidelity driving simulator on either a predominantly straight road or a predominately curvyroad. The information processing task required participants to monitor a continuous stream ofletters and to enact a button-response when a particular letter was presented; this task was11

Six studies fell into this category of mixed auditory cost and auditory benefit for primaryand secondary tasks (refer to Figure I3). The results from these studies suggest that the benefit toauditory presentation of primary task information may be determined by display location. All ofthe studies that showed a benefit to primary task performance had visual displays located at wideeccentricities from the line of sight; an auditory cost for primary task performance was seen onlyin the one study that presented visual information on a head-up display (Mollenhauer et al.,1994). However, there appeared to be no clear characterization of secondary task performance.As in the first set of studies, which showed an auditory cost, half of the studies (three out of six)had irrelevant messages (Horrey & Wickens, 2002; Hurwitz & Wheatley, 2002; Ranney et al.,2002) and the other three had driving-relevant messages (Mollenhauer et al., 1994; Labiale,1990; Srinivasan & Jovanis, 1997). Another property of the studies shown in Figure I3 is the factthat increasing workload generally increased the auditory advantage.Class II - MixedMollenhauer et al., 1994RelHUDHorrey & Wickens, 2002IRRDownLabiale, 1990RelDownHurwitz & Wheatley, 2002IRRDownRanney et al., 2002IRRDownPrimary Task Secondary TaskHazard;Lane Pos.RTAccuracyLoHiLo HiDriving WorkloadIVT ComplexityLoHiLoHiIVT ComplexityNot ReportedLo HiDriving WorkloadPeriphery DetectionSteeringSrinivasan & Jovanis, 1997RelDownHazard;Lane Pos.LoHiDriving WorkloadFigure I3. Icon figure of mixed auditory cost/benefit studies.13

speed limit, traffic density, and number of intersections. The navigational instructions were ineither visual, auditory, or redundant form. The visual information consisted of either a routeguidance map or text messages combined with icons of vehicle and roadway status, and waspresented on an LCD display mounted centrally above the dashboard (approximately 24 o to theleft of the top of the steering wheel). The auditory information contained roadway-relevantinformation or route guidance information and was presented through a speaker in front of thepassenger seat. The redundant information was a simple combination of the auditory and visualinformation. The three modality conditions utilized the same set of information units (for thenavigational information task) which included geographical entities, road type and position, anddriving instruction. Participants performed the navigation task under both high (visual: 9 – 14information units; auditory: message presented every 5 – 8 seconds) and low (visual: 3 – 5information units; auditory: message presented every 20 + seconds) workloads. An additionalsecondary task required participants to manually respond with a button-press to the vehicleinformation by either categorizing it as road condition or vehicle condition information.The overall effect of modality on primary task performance suggests a benefit to auditorypresentation of IVT information with better steering and speed control for auditory than visualinformation. The benefit to auditory presentation was modulated by information complexity; anincrease in information complexity resulted in an increase of the auditory benefit to drivingperformance. The increase in message complexity additionally resulted in longer reaction timesfor the button-press task with visual, complex messages. The effect of modality on secondarytask performance was contingent upon the driving task load, showing significant effects ofmodality only with the high workload driving condition. The auditory display resulted in betterinformation processing relative to the visual display with faster reaction times to the button-pushtask, fewer navigation-related errors to the navigation task, and a higher percentage of correctturns. The benefit to auditory presentation was somewhat mitigated however by the informationcomplexity; more wrong turns were made in the auditory condition than in the visual conditionwith complex information.Streeter, Vitello, and Wonsiexicz (1985) conducted a study that compared theeffectiveness of two navigational aids that differed from each other in both modality andprocessing code. Participants engaged in a driving task along moderately difficult local roads andsimultaneously navigated their route with a particular in-vehicle navigation display. The visualinformation was a spatial representation with a navigational map (a hand-held map); the auditoryinformation was a verbal representation with a recorded list of street names and direction.Performance on the primary driving task was better using the auditory display with moreaccurate speed control and navigation; this benefit may be attributed to the reported greatersuccess in navigating using textual instructions rather than a spatial representation of theparticular desired path (Sanders & McCormick, 1993). The benefit of a verbal representation ofnavigational information is further seen in secondary task performance with auditory informationdisplays resulting in only one-third of the navigational errors (e.g., wrong turn, missed turn) asthose observed with visual information displays.Walker et al. (1990) examined the effects of auditory and visual in-vehicle navigationdevices on driver performance, as assessed by lateral deviation, speed control, and reaction timeto gauge changes. The visual device presented route electronic and textual route information (ona CRT display located to the right of the driver approximately 20 degrees below the line of15

sight); the auditory device presented aural direction and route specification text messages. TheIVT workload was manipulated for both the auditory and visual displays with three levels ofinformation complexity. The driving task workload was also manipulated with the addition ofcrosswinds, traffic, narrowing lanes, and information processing tasks. The auditory informationdisplays supported better time-sharing performance relative to the visual information displays forboth primary and secondary task measures (vehicle and speed control, and number ofnavigational errors respectively). The effect of primary task workload on driver and IVTperformance was evident only in the preferences of drivers to navigate in a low complexityenvironment. That is, driving workload had no influence on the auditory benefit for drivingperformance. The interaction of workload with secondary task performance however reduced theauditory benefit of the IVT performance (the advantage of auditory over visual performancedecreased), but IVT complexity increased the driving performance advantage of the auditorydisplay.The results from these studies, summarized in Figure I4, suggest that the benefit toauditory presentation of secondary task information may be influenced by display location andmessage relevance. Five studies fell into this category of auditory benefit or auditory advantagefor both primary and secondary task performance. The majority of the studies (four out of five)present the visual information on a head-down display. This would suggest that the auditorypresentation of the navigation information may receive an advantage over the visual presentationdue to the large visual display separation, an observation which will be discussed in a latersection. Furthermore, all five of the studies had driving-relevant messages (Burnett & Joyner,1997; Gish et al., 1999; Liu, 2001; Streeter et al., 1985; Walker et al., 1990), showing a cleartrend for an auditory benefit for both primary and secondary task performance with relevantmessages. The extent to which this benefit will fail to emerge with irrelevant messages will beassessed in the current study.16

Class III – Auditory BenefitPrimary Task Secondary TaskBurnett & Joyner, 1997RelHUDGish et al., 1999RelDown / HUDHazard2 TasksLiu, 2001RelDown(& over)Streeter et al., 1985RelDown(Map)Walker et al., 1990RelDownFigure I4. Icon figure of auditory benefit studies.RT and AccuracyNavigationDriving WorkloadLoHiIVT ComplexityLo Hi Lo HiIVT ComplexityThe literature comparing cross-modal and intra-modal information presentation revealsmixed findings for dual-task performance. If multiple resources are the important mechanism,auditory presentation of side task information is expected to result in improved time-sharingperformance between primary and secondary tasks relative to visual presentation of the sameinformation (Wickens, 2002). The various display manipulations however mitigate the instanceof an auditory advantage and disadvantage within the literature. A review of the driving literaturethat manipulates modality delivery of in-vehicle task information in realistic driving simulationsidentifies the location of the visual display (HUD versus head-down) as one factor thatmodulates performance relative to an auditory display. A second factor, potentially, is therelevance of the message to the driving task as a trend in distinguishing the instances of anauditory advantage (more likely when relevant) from an auditory disadvantage (more likelywhen irrelevant; Wickens & Seppelt, 2002). We consider this relevance factor in more detail inthe following section.17

Relevance of In-Vehicle TasksA majority of vehicle-based cross-modal studies have delivered “related” (ATIS,navigation) information, rather than unrelated in-vehicle task information (i.e., infotainmentinformation, e-mail, cell-phone numbers). Of the reviewed studies, 2 out of the four auditory coststudies, 3 out of the six mixed auditory benefit/cost studies, and all 5 out of the five auditorybenefit studies had driving-relevant messages (Lee et al., 1999; Dingus et al., 1997; Mollenhaueret al., 1994; Labiale, 1990; Srinivasan & Jovanis, 1997; Burnett & Joyner, 1997; Gish et al.,1999; Liu, 2001; Streeter et al., 1985; Walker et al., 1990). These driving studies reveal eitherthat auditory presentation of side task information provides less disruption of driving than visualpresentation of side task information, or that auditory and visual presentation of side taskinformation yields equivalent effects on driving performance. In only one case did researchersfind that visual presentation of side task information resulted in less distraction to drivingperformance than auditory presentation of side task information (Mollenhauer et al., 1994).In contrast to the above, an auditory cost is more likely to be seen in studies that utilizesecondary tasks that carry less direct relevance to the driving task. Matthews, Sparkes, andBygrave (1996), required participants to perform Baddeley’s (1986) Reasoning Test as asecondary task to the primary driving task. The task had no bearing on the nature of the drivingtask. Results yielded a cost of utilizing the auditory modality for presentation of the varyingsentences. Horrey and Wickens (2002) reported a slight cost of presenting unrelated auditoryinformation with the auditory condition exhibiting more variation in speed control relative to thehead-down visual condition.This analysis of the dual-task driving literature suggests a trend, that is partiallysupported by other data, suggesting that irrelevant auditory messages can be distracting fromprimary visual tasks (e.g., Latorella, 1998; Wickens & Liu, 1988). A non-driving study by Tsangand Rothschild (1985) that employed a non-relevant secondary task presented a spatialtransformation task in conjunction with a compensatory tracking task supports this interpretation.The results indicated a cost to presenting the spatial information aurally (achieved throughvariance of tone location) as opposed to presenting the information visually. Participants aredistracted by the information as opposed to being assisting in performing the tracking task.Because irrelevant tasks do not support the task of driving, a primarily visual task, there is nobenefit in offloading cognitive demands to the auditory modality in decreasing the workload ofthe driving task. Drivers are less skilled in being able to integrate (or filter) irrelevant auditoryinformation when they are concurrently performing the visual driving task, and indeed suchauditory delivery has been found to “preempt” ongoing visual tasks in aviation research (e.g.,Helleberg & Wickens, 2003; Latorella, 1998). Wickens and Liu (1988) developed a “signature”of this auditory preemption effect, whereby a discrete secondary task (here represented by the invehicletask) benefits from auditory delivery of information, since that delivery will “callattention” to its message, whereas the ongoing visual task (here, driving) will suffer by auditoryrelative to visual delivery of the secondary task. And while the auditory-visual / related-unrelatedinteraction can be tied with auditory preemption of unrelated tasks, it is important to note that hiscan also be linked to emerging findings from the field of educational technology, namely thecognitive load theory, articulated by Sweller and his colleagues (e.g., Tindall-Ford et al., 1997;Wickens & Hollands, 2000, chapter 6). Proponents of this theory stress that when deliveringrelated verbal and spatial (i.e., pictorial) information for instruction, the former is more18

effectively delivered by speech (auditory) than by text (visual), thereby capitalizing on multipleresources.The theoretical explanations behind the relevant/irrelevant distinction are as follows: forrelevant messages, an auditory display is beneficial because having attention preempted to thesecondary task is useful in that the information pertains to the driving task and is intended toameliorate the driving task, to support the driving task. This is more beneficial than a visualdisplay because the visual display is more easily ignored and the information could be missed;additionally, taking the eyes from the roadway to gather information to the driving task isunnatural and is distracting to a task that requires one to look at the roadway. A driver couldbetter be able to make use of multiple resources and to integrate the information they are hearingif it pertains to the driving task than if it does not have any significance to the primary task. It iseasier to disengage from the IVT if it is relevant to the driving task because drivers do not needto re-orient their mental scripts or processes if they are processing the same type of information.However, for an irrelevant display it is harder to disengage from the IVT task if it is an auditorytask because they have to make the mental switch, especially as the auditory informationpreempts their attention and redirects it to other cognitively-different processes. For the visualdisplay, the operator need only to look away for them to be able to disengage themselves formthe IVT task; it is a way to self-monitor the intake of information in cognitively-taxing situations.For irrelevant messages, drivers need to be able to ignore the secondary task; this is whya visual message is appropriate because it gives the driver the ability to look away from thedisplay when (s)he needs to concentrate their attentional resources on the driving task(particularly if it is complex). Having an auditory display is potentially detrimental because, aswe have noted, it can preempt the driver’s attention and cause them to direct their attention awayfrom the driving task to the secondary task to information that is irrelevant to the primary taskthey are conducting.Spatial Separation of In-Vehicle TaskThe literature comparing auditory and visual modalities of discrete tasks coupled with acontinuous visual task (Wickens & Seppelt, 2002), supports the conclusion that the larger thedisplay separation, the greater the cost to using a visual display over an auditory display. Schonsand Wickens (1993) found that spatial separation between displays greater than 7.5 degreesdegraded performance significantly. Wickens (1992) defines regions of visual scanning in termsof degrees of separation for pairs of displays. The outlined model of information access effortreports that display separation greater than the no-scan region (approximately 7.5 degrees)requires increased time and effort to acquire information in scanning. At greater eccentricities,drivers must divide their focal visual resources between relevant information in the drivingenvironment and in-vehicle, and thus they are less likely to detect unexpected events in thedriving scene (Gish et al., 1999; Horrey & Wickens, 2002).Of the studies presented in the icon figures, two out of the four auditory cost studies, fiveout of the six mixed auditory benefit/cost studies, and four out of the five auditory benefit studieshad head-down displays; these studies showed either an auditory benefit or equivalence betweenmodality for primary task performance. Of these eleven studies with head-down displays with19

greater than 20 degrees separation, all but three (Horrey & Wickens, 2002; Hurwitz & Wheatley,2002; Ranney et al., 2002) had driving-relevant messages.The four remained studies in the review had a separation angle of less than 11 degrees(HUDs). Of these, one revealed an advantage of auditory over visual display use (Burnett &Joyner, 1997), one revealed equivalent results with the auditory and visual display use (Horrey &Wickens, 2002 - with the HUD), while two actually found an advantage of using a visual displayover an auditory display (Mollenhauer et al., 1994; Matthews et al., 1996). Note that Matthews etal. (1996) was a study in which the IVT display was unrelated to driving. Thus it appears thatwhen there is substantial visual display separation (>15 degrees), there is usually a cost to usingthe visual display over the auditory display. When there is not a large display separation, theresults are equivocal, showing either no difference, or a visual advantage.WorkloadPrimary Task WorkloadA relatively small number of dual-task driving studies look systematically at how anydifference in auditory/visual information displays are modified by driving workload or taskcomplexity (i.e., in terms of a modality x complexity interaction in some measure of drivingperformance; Liu, 2001; Walker et al., 1990; Hurwitz & Wheatley, 2002; Srinivasan & Jovanis,1997; Horrey & Wickens, 2002; Matthews, Sparkes, & Bygrave, 1996). These workload effectsare reflected by the slope of the lines in Figures I2 – I4. Liu (2001) reported that under highdriving load condition, participants tended to more accurately modulate speed when using theauditory display than when using the visual display or the multimodality display. Additionally,as driving load increased, driving control (as measured by mean absolute velocity deviation) withuse of the multimodality display increased relative to the auditory and visual displays. Hurwitzand Wheatley (2002) found that steering wheel variations and lane deviations were greater oncurved (versus straight) tracks that utilized visual secondary tasks than on trips that utilizedauditory tasks. The benefit of auditory displays in primary control performance increased as thedriving load increased. Srinivasan and Jovanis (1997) found that the auditory benefit increasedwith the more complex driving situations. Horrey and Wickens (2002) found no interactionbetween increased driving load and display modality. Only one study showed an interaction suchthat there was greater harm in the auditory than visual condition, as driving workload increased(Matthews, Sparkes, & Bygrave, 1996). Matthews et al. (1996) found that dual-task interferencewas evident only in curve driving in the auditory condition; a finding that did not occur instraight driving, inferring that auditory and visual displays render equivalent performance in lowdriving loads.The manipulation of driving load further carries implications for the in-vehicle side taskperformance. Liu (2001) found that an increase in driving load leads to improved navigationusing an auditory display compared to using a visual display (with the highest percentage ofcorrect turns associated with the auditory display); at the low driving load, significant effects didnot arise between the modality manipulation, however, at high driving load, the auditory displayperformed significantly better than the visual display.20

Thus, in general, most of the reviewed studies show an increased benefit to using anauditory display with the increase in driving workload. Increased complexity of the driving taskincreases the bandwidth of the guidance, navigation, and hazard detection tasks; thus, driversmust devote more of their visual resources to attend to the driving environment and consequentlyhave fewer visual resources to allocate to an in-vehicle task. The auditory modality helps tooffload the visual requirements and provide better time-sharing for both the driving task and thein-vehicle task.Secondary Task WorkloadAn increase in the complexity of the side task in dual-task literature is shown to have animpact on driving and secondary task performance (Dingus et al., 1994; Liu, 2001; Walker et al.,1990; Ranney et al., 2002; Srinivasan & Jovanis, 1997; Labiale, 1990; Mollenhauer et al., 1994;Horrey & Wickens, 2002). A number of studies find that an increase in IVT complexity leads toan increased auditory benefit for the primary driving task (Liu, 2001; Walker et al., 1990; Dinguset al., 1994; Labiale, 1990). Liu (2001) reported that compared to equivalent performance in lowIVT workload conditions, complex (high IVT workload) visual information decreased driver’svehicle control and led to more frequent major lane deviations than when such information waspresented by auditory and multimodality information. Under high load conditions, the auditorymodality required less driver attention and led to safer driving behavior than did the visualmodality. Both Walker et al. (1990) and Labiale (1990) report results that suggest that increasingcomplexity either produced a visual cost, or amplified a pre-existing visual cost (in using a visualinformation display). Walker et al. (1990) reported that a low IVT workload was met withequivalent performance whereas a high IVT workload resulted in an auditory benefit with greaterlane maintenance and speed control found with use of the complex auditory display over thecomplex visual display. Labiale (1990) reported that at low workloads, visual and auditorypresentations of IVT information equally affected driving performance; however, at high IVTcomplexity, auditory presentation resulted in more effective speed and vehicle control than didvisual presentation of IVT information. Dingus et al. (1994) found that the addition of voicereduced the visual attention demand requirements; this effect was more pronounced with anincrease in display complexity (turn-by-turn versus route map). The combined results of thesestudies would suggest that the predictions of MRT are upheld in that subjects were found toexhibit better driving performance when able to offload the secondary task to the auditorymodality.The conclusions drawn from research that has examined the effects of increased side taskworkload on side task performance on the other hand are mixed. Liu (2001) found an auditorybenefit to the IVT load increase, showing that under the complex secondary task condition, agreater decrement in response time and error rate was evident for the visual modality comparedto the auditory modality and multimodality; the IVT performance was equivalent for the visualand auditory modalities under low IVT workload conditions. In contrast, Labiale (1990) foundthat under high workload conditions, visual presentation of IVT information results in betterrecall performance than auditory presentation of IVT performance; IVT performance isequivalent at low workload conditions. However, the degree of auditory cost found in the Labiale(1990) study is mitigate by the finding that auditory presentation is better than visualpresentation of IVT information with increased message complexity as measured by the numberand duration of visual explorations. This finding is arguable a more direct measure of driving21

safety than recall performance and thus a more relevant determinant of AV cost/benefit. Finally,Horrey and Wickens (2002) noted that very long auditory messages (10-digit phone numbers)led to a decrease in accuracy as the capacity of working memory was exceeded, a decrease thatwas not shown in the visually delivered phone numbers nor, as noted above, in the primarydriving task.Redundant Presentation of InformationIn-vehicle information may be presented using visual displays, auditory displays, or both.The majority of the information needed to perform the driving task is obtained visually, and thusit would follow that auditory presentation of secondary task information would detract less fromthe driving task. Further, intuition would support the idea that redundant presentation of thesecondary task information would lead to better performance, as the individual modalities in bothmodalities would be reinforced by the additional modality present. Helleberg and Wickens(2003) however present contrasting findings to that of intuition. The study compared theeffectiveness of auditory, visual, and auditory-visual data link displays (in the context of anaviation domain). The redundant display condition resulted in inferior performance relative toboth single modality conditions; the cost in performance was attributed to the distracting natureof the auditory stimulus when initiated during an ongoing visual task. Stokes, Wickens, and Kite(1990) reinforce the idea of audio messages being intrusive (or preemptive, as discussed earlier).The results of the Helleberg and Wickens study (2003) suggest that redundant presentation ofinformation may not lead to superior performance because of the difficulty in uncoupling the twomodalities in such a way as to take advantage of each modality individually. Essentially, the twomodalities limit each other in mutually competing for attention and in forcing the completion ofthe task to hinge upon the least effective (i.e., longest reaction time, longest processing time)modality. The findings on redundancy in its effect on dual-task performance are mixed.In order to better understand the dual-task effectiveness of redundancy, Wickens andGosney (2003) constructed a five-category system to classify the effects of redundantpresentation of secondary task information relative to single-modality presentation. In the casethat the redundant display produces better dual task performance than either single-modalityalone, the effect is classified as “Gestalt”, following “the whole is greater than the sum of itsparts” definition. In the best of both worlds (BOBW) pattern, the redundant display producesperformance that is equal to the better of the two single-modality conditions. In the case that theredundant display has dual-task performance that is midway between the performance of the twosingle-modality conditions, the redundant display is classified as Average. In the worst of bothworlds (WOBW) pattern, the redundant display produces performance that is equal to the poorestof the two single-modality conditions. And finally, in the case that the redundant displayproduces worse dual task performance than either single-modality alone, the effect is classifiedas Anti-Gestalt (a summary of these five categories can be seen in Figure I5 below). Naturallyone of these 5-level categories can characterize performance on each of the primary andsecondary tasks, in each set of experimental results.22

Gestalt ☺ABest of BothWorldsAverageWorst of BothWorldsAnti-Gestalt AV > A, VAV = Best (A, V)AV = (A+V) / 2AV = Worst (A,V)AV < A, VA V AVA V AVA V AVA V AVA V AVFigure I5. Classes of redundancy effects, from Wickens and Gosney (2003). The graphs on theright represent hypothetical measures of performance, with higher performance shown by higherlevels.Within the context of the 5-level categorization, Wickens and Gosney (2003) examinedthe effects of modality, redundancy, display location, and task priority of a discrete IVT task (adigit callout task) on interference with a continuous visual tracking task. Results showed that theredundant display of the secondary information resulted in an average effect of redundancy atnear angles of separation and a WOBW effect of redundancy at far angles for measures ofprimary and secondary task performance. And while instructions in the use of the redundantdisplays did improve their performance, the single-modality displays still resulted in betterperformance as the redundant condition was overall the average of the two single-modalityconditions.In the context of the driving domain, the results of four studies show a greater advantagein using a redundant display over a single-modality display whether for visual (Dingus et al.,1994; Srinivasan et al., 1994; Parkes & Burnett, 1993) or over both single-modalities (Liu,2001). Dingus et al. (1994) presented four configurations of a navigational system toparticipants. Each participant navigated a route with a visually displayed turn-by-turn systemeither with voice or without voice, or a route map with voice or without (an auditory aloneconditions was not included). The visual form of the navigational information presented either afull, forward-directional route map (located to the right of the driver on the horizontal center ofthe dashboard approximately 15 degrees below horizontal), graphical representation of staticturn-by-turn information, a textual paper direction list, or a conventional paper map. Theauditory form of the navigational information presented the visual directional list and theforward-directional route map with redundant aural messages. The format of the routeinformation (turn-by-turn information or a spatial map representation) varied the workload withincreased visual attentional required with the spatial map presentation. Mean velocity and23

Redundant modality presentation (auditory and visual) of secondary task information isexpected to improve driving and side task performance in providing the benefits (strengths) ofboth modal channels; however, redundant presentation has not always lead to better performancethan that of the two single-modality display conditions (Helleberg & Wickens, 2003); and of thedriving studies reviewed, only one evaluated both single-task conditions, to allow the results tobe placed in the context of the scheme shown in Figure I5.The different manipulations and methodologies of studies that investigate deliverymodality of in-vehicle task information in relatively realistic driving simulation revealinconsistencies in the conclusions regarding the effects of secondary task complexity,redundancy, and modality presentation on primary and secondary task performance. Our reviewindicates that relevance of the secondary task information to the primary driving task maymodulate the differential effect of auditory versus visual presentation of in-vehicle messages ondriver performance (see also Wickens & Seppelt, 2002). However this contrast between relevantand irrelevant in-vehicle messages has never been examined within an experiment, thatcontrolled for the length or complexity of the two classes of messages; hence, this study willinclude the manipulation of the two message types in the design. A second factor that is expectedto mediate the relative difference between the two modalities of information presentation is thecomplexity of the message set, which this experiment will also manipulate. Finally, in addition toauditory and visual delivery, this experiment will add a third delivery medium involving theredundant presentation of auditory and visual information, for which the literature, as notedabove, reveals an inconsistent pattern of benefits.All conditions in this experiment involve the same general procedure adopted by Horreyand Wickens (2002), in which in-vehicle messages were delivered while the driver engaged inroutine vehicle control and unexpected response to roadway hazards in a high-fidelity simulator.The difficulty of the driving task was controlled with a consistent high (workload) task demandinvolving numerous curves and elevation changes. Numerous curves were included in the roaddesign as the primary driving task was intended to significantly tax the driver such that theirattentional resources were limited. Measures of lane deviation, steering control, and hazard eventdetection were taken to assess primary driving performance. At random intervals throughout thedriving task, participants were presented with in-vehicle messages delivered either auditorially,visually (in a head-down console), or redundantly; half of the participants received messagesrelevant to the driving task, the other half of the participants received messages irrelevant to thedriving task. The complexity of the information messages was varied with either a low-workloadmessage (single sentence/idea, maximum of 5 words) or a high-workload message (twosentences/ideas, maximum of 11 words). Every unexpected hazard occurred just following thedelivery of a complex message. A number of hypotheses, for this experiment, were formulatedbased on a comprehensive review of the literature (Wickens & Seppelt, 2002) examining thefactors which modulate the differential effect in costs and benefits of modality presentation toprimary driving performance and secondary (IVT) task performance.1) Cross-modal (auditory-visual) information presentation is expected to result in betterprimary task (safety, e.g., hazard detection) performance and secondary task performancethan intra-modal (visual – visual) information presentation, given the head-down locationof the latter.26

narrow drop shoulder curve, 2 lane rural fork, and 2 lane rural t-intersection with left hand turnlane. The residential sections contained the following tiles: 2 lane residential 90 degree turn, and2 lane straight residential with center turning lane. Each driving environment was comprised of 2rural sections and 2 residential sections, with appropriate transition tiles between the different tilesections. Ambient traffic moved throughout the driving environment at a minimal rate ofapproximately 3-5 cars per minute (or 3-5 cars per kilometer). This moderate amount of trafficreflected a reasonable level of traffic in a rural setting. An example of one driving course isshown in Figure M1b. The environments were portrayed from the drivers’ viewpoint in a mannerconsistent with those seen in Horrey and Wickens (2002) (see Figure M1a).a) b)Figure M1: View of the driving simulator apparatus adopted from Horrey and Wickens (2002)(1a). Overhead view of typical driving environment with curved rural and residential roadsections (1b).EventsCoupled with the task of driving, subjects were required to process messages that carriedeither direct relevance to the driving task or that carried no direct relevance to the driving task.The messages were delivered at approximately 2-minute intervals for a total of 10 messages perdriving route (based on the total driving time of 20-25 minutes); subjects encountered a total of30 messages across conditions. The 10 task messages per route were randomly selected from atotal of 14 possible events. Message complexity was varied for the 10 messages within eachdriving environment (comprised of 5 simple messages and 5 complex messages per drive).During the periods of baseline measurement, no messages were presented to subjects in order toassess driving task performance alone.Relevant task messages warned of driving events that occurred downstream; therefore,the relevant driving task messages carried significance for the primary driving task, and thedriver’s compliance with the message could be inferred from his or her subsequent steering ordecelerating behavior. Irrelevant task messages carried no significance to the primary driving29

task or to the downstream implications to the driving environment. In order to ensure that bothrelevant or irrelevant messages were heard or read, operators were asked to repeat them out-load.For relevant conditions, the braking and steering responses were examined to determine if theimplications of the messages were understood; an early braking or steering response indicated ahigher degree of understanding and compliance to the message. For irrelevant conditions,operators were asked to categorize the messages semantically to ensure that the implications ofthe messages were understood (see Appendix C for coding forms for relevant and irrelevantmessages). The messages (relevant and irrelevant) were delivered to the subjects approximately8 - 10 seconds before the relevant driving conditions were encountered downstream and wereplaced in the driving environment at approximately 2-minute intervals. The irrelevant“infotainment” messages were equated for complexity (equal word length) with the relevantmessages and occurred at the same location in the drive (see Appendix D for an example), as thecorresponding relevant messages. The irrelevant messages fell into 4 categories including: 1) cellphone/e-mail, 2) national news, 3) radio, and 4) weather. The four categories of irrelevantmessages were equally represented such that 2 of each type of messages (for a total of 8) and 2randomly selected comprised the 10 messages per scenario. Examples of the relevant events,ATIS messages (relevant and irrelevant), and downstream implications are presented in TableM1.30

Table M1: Examples of ATIS relevant and irrelevant messages.Event Relevant Messages Irrelevant Messages Implications Downstream1. Accident in lane Accident in road ahead. (simple);Accident in road ahead. Slow to 45 MPH.Mostly sunny and warm. Highs in the 70s.(complex; Weather);Cones, collided cars,emergency vehicles(complex)Smith Job Talk tomorrow. (simple; CellPhone);Ex-Serb leader faces tribunal. (simple;National News)2. Curve Sharp curve ahead. Slow to 35 MPH.(complex)Artic chill deepens. Drop to -10 degrees.(complex; Weather)90-degree turn (residential)3. Crosswalk Pedestrian crossing ahead. (simple) Alternative Rock 107.1. (simple; Radio);Light showers today. (simple; Weather)Pedestrian crossing withpedestrians present onsidewalk/roadway4. Emergency vehicle in lane Approaching emergency vehicle (simple);Emergency vehicle in lane ahead. Blindspots in curved road ahead. (complex)5. Merging traffic Merging traffic from the right/left. (simple);Heavy traffic ahead. Merging traffic fromthe right/left. (complex)Afternoon flurries are expected today.Temperatures rising in the early evening.(complex; Weather);Volleyball practice tonight. (simple; CellPhone)Smooth Jazz V98.7. Home of the Trip-A-Day Give-Away. (complex; Radio);Girl missing after plane crash. (simple;National News);Contact Linda Carr. Office hours from 3-5daily. (complex; Cell Phone);Low temperatures today. Cold front fromthe North. (complex; Weather)Oncoming emergency vehicle(traveling down center of lane)Employ merging road tile, lineof cars moving onto mainroadway from side road6. Merging vehicle Merging vehicle ahead. (simple);Merging vehicle ahead. Beware of blinddriveways. (complex)Information request received. Send receiptto Accounting. (complex; Cell Phone);National Census taken. Latinos largestU.S. minority. (complex; National News)Obscured driveway with carpulling out and joining trafficflow7. Object in road Object in road ahead. (simple);Object in road ahead. Use caution.(complex)San Francisco bans Segway. (simple;National News);Drew and Mike Show. (simple; Radio);Highs in the 60s. Sunny tomorrow.(complex; Weather)8. Railroad crossing Railroad crossing ahead. (simple) Holiday party tonight. (simple; Cell Phone);Library item available. (simple; Cell Phone)9. Reduction of lead vehiclespeedSlow vehicle ahead. (simple);Slow vehicle ahead. Curve in road.(complex)Terrorist leader arrested. (simple; NationalNews);Tower of Power - What Is Hip? (complex;Radio);Send project fax. (simple; Cell Phone)Bicyclist in road ahead indriver's laneEmploy a railroad tileRapid deceleration of lead car;or car merging onto roadwayfrom left side of road followinga curve10. Road construction Construction ahead. (simple);Construction ahead, Detour to the left/right.(complex)LiteRock 98.7. (simple; Radio);Partly cloudy. (simple; Weather);94.7 WCSX. Rock to the Hits. (complex;Radio)Cones, barrels, constructionvehicle, arrow trailer; roadveering off to the right/left thatloops back onto main road11. School zone School zone ahead. Watch for children.(complex)Light snow showers. Wind to increase.(complex; Weather);Smooth Jazz 104.3. Tunes to Groove.(complex; Radio)12. Slippery roadway Slippery road ahead. (simple) War protestors arrested. (simple; NationalNews)13. Speed reduction Speed reduction ahead. (simple);Speed reduction ahead. Speed limit 35MPH (complex)Research position available. Applicationdeadline June 5th. (complex; Cell Phone)Place school building andcrosswalkChange in steering wheelcontrol variablesTransition tile from residentialto rural following curved ruraltile14. Traffic congestion Traffic congestion ahead. (simple);Traffic congestion ahead. Detour to theright/left. (complex)Chemical warheads found. Inspectors to Line of cars; road veering off tocontinue search. (complex; National News) the right/left that loops backonto main road31

The complex messages were twice the length of the simple messages both in word countand in syllable count (the summary statistics of the messages are presented in Table M2).Table M2. Summary statistics of word count and syllable count of relevant and irrelevantmessages.Category Number of Words Number of SyllablesSimple Complex Simple ComplexRelevant Mean 3.27 7.2 Mean 6.6 12.2Stdev 0.8 1.32 Stdev 1.3 2.65Irrelevant Mean 3.27 7.2 Mean 6.67 13.87Stdev 0.8 1.32 Stdev 1.88 4.09All text messages were presented to the driver in the head down display positioned on thedashboard next to the instrument panel. Auditory messages were presented through the vehicle’s4-speaker surround sound system. Redundant messages presented both modalitiessimultaneously, with the initial phoneme of the auditory message coinciding with the onset of thevisual message.Critical hazard eventsIn addition to the events pertaining to the task messages, subjects were presented withtwo unexpected events per driving environment. The critical events were intended to measuredriving performance with respect to reaction time to unexpected events (an indirect measure ofhazard awareness). The types of critical events are described below (see Figure M2):Pedestrian/Bicycle IncursionsVehicle IncursionsParked Car PulloutWide Right TurnOncoming Lane DriftOncoming Lane Drift (Curve)32

a) b)c) d)e) f)Figure M2. Unexpected hazard events used in the study, adopted from Horrey and Wickens(2002): a) Lane drift on a curve, b) Lane drift on a hill; c) Car incursion; d) Bicycle incursion, e)Parked car pullout, f) Wide right turn.Driving performance was assessed according to the subject’s response time to theunexpected events. For the incursion events, the response time was measured from the time thatthe simulator vehicle triggered the event (via a location or time trigger located on the roadway)to the time when the subject first responded to the event (measured responses: braking, steeringmaneuvers). The events that involved a lane drift or pullout were measured from the moment thatthe intruding vehicle began to move towards the simulator vehicle’s path to the first response ofthe subject (events and measures adopted from Horrey & Wickens, 2002). Two unique hazardevents per driving environment were presented for a total of 6 events per subject.33

Throughout the experimental drives, participants were presented with IVT messages.Participants were instructed to promptly attend to the messages and to respond to theirpresentation with a manual button press as quickly as possible; the response buttons were locatedon the steering wheel, at either side, for ease of response for the driver. The calculation of theresponse time to the button press began at the instance of visual presentation (or for the auditoryconditions, at the start of the initial phoneme) of the IVT message. Following the button press,participants were to read-back the displayed (or vocalized) message. Participants were notinstructed to press the button at the end of the read-back due to the varied number of words inboth the simple and complex conditions. Additionally, in the irrelevant message condition,participants were instructed to verbally classify the particular message according to the 4category types. This categorization task was intended to guarantee some semantic processing ofthe message, as was assured to be the case with the relevant messages (in processing andintegrating the information for response to the downstream events). The inter-stimulus intervalbetween each message presentation was approximately 2 minutes. Half of the subjects receivedthe relevant messages that previewed them to upcoming roadway and hazard conditions whichcarried implications downstream on the roadway. The second half of the subjects receivedmessages that carried no significance (irrelevant) to the driving task at hand but that required theverbal response and categorization in addition to the manual key press in order to ensure mentalprocessing of the message information. Participants were instructed to attend to the IVTmessages as promptly as possible and to properly navigate for safe driving.During each experimental drive, participants were presented with two randomly-selectedcritical hazard events. Each hazard event was coupled with a complex IVT message. The coupledpresentations occurred at random placements within the driving environment to ensure theunexpected nature of their presentation.Following completion of the three experimental drives, participants were administered abrief post-experimental questionnaire on basic driving habits (see Appendix J). Thequestionnaire was given to allow participants a moment to overcome any residual feelings ofunease from driving in the simulator before leaving; and thus, the data from the questionnairewas not analyzed nor reported in this study. Participants were then thanked and reimbursed fortheir participation.Experimental DesignThe experimental design was a mixed factorial design (3 x 2 x 2 design) with 2 withinsubjectsvariables and 1 between-subjects variable. The experiment considered 3 levels of IVISdesign: (a) message modality (visual, auditory, combined visual-auditory), which was a withinsubjectsvariable; (b) message complexity (simple, complex), which was a within-subjectsvariable; and (c) message type (relevant, non-relevant), which was a between-subjects variable.The order of message modality and of the three driving worlds was counterbalanced in a Latinsquare design. Message complexity was randomly selected within each driving environment.Participants were assigned to the two levels of relevance in an alternating fashion.In the “relevant” condition, (half of the subjects), each of the three roadway environmentswas scripted with its own unique set of relevant messages, downstream events, and hazardevents. In the “irrelevant” condition, (half of the subjects), a (unique) message set consisting of35

the 4 category types was created in a way that matched the syllable and word count of the simpleand complex relevant messages. One third of the subjects drove through each environment withone of the three display configurations.RESULTSThe results will be described according to primary (driving) and secondary (messageprocessing) task performance. The first section addresses the effects of message modality,relevance, and information complexity on measures of primary task performance including lanedeviation (tracking error), steering velocity, and reaction time to hazard events. The secondsection addresses the effects of the same independent variables on secondary task performanceincluding reaction time to message presentation and accuracy of message read-back. Measures ofcompliance are presented as support for the effective design and participant use of the secondarytask messages. The analyses were performed using the Systat 10 software. Standard error barsare used on the graphs and show 2 SEs for each data point. Each of the graphs is presented withthe x-axis containing message modality ordered with the following labels: Auditory, Redundant,and Visual. The redundant modality is placed in-between the single modalities to better comparethe effects of redundancy to the single-modal conditions.Data from ten subjects (those mentioned in the Methods section) was not included in theanalyses for the primary or secondary task performance due to (substantially) incomplete datasets resulting from simulator sickness and equipment failures. However, two of the thirty-sixsubjects included in the analyses had minimal missing data; the missing cells were removed fromthe data analysis for these subjects.Due to the high volume of driving control data collected (60 Hz), the raw data wastransformed into readable outputs and condensed into 15-second windows of time surroundingthe 10-second message events. The remainder of the time was integrated and used for controlmeasures. Data for the 15-second windows of time were separated into 4 data bins that weregrouped according to meaningful flags in the data across time. The data flags corresponded to thetimeline of events presented earlier in the METHODS section; namely, message presentation(MP), the instance of a button press (BP), and the downstream event (DSE). The three datamarkers categorized the data across time and served as the zero points for data analyses. A visualrepresentation of the 15-second window of time and the relevant data markers is presented inFigure R1.36

Figure R1. A typical 15-second message window. The graph shows the continuous vehiclecontrol data over a 15-second window of time as a function of time with normalized y-axisvalues (for the purposes of plotting all of the dependent primary task measures on one graph).The yellow arrows represent the different data bins within the 15-second window of time. Thefigure depicts a trial in which a hazard became visible around the time of message presentation.Here it is possible to see both the abrupt steering response (green line downward) and brakingresponse (yellow dashed-line upward) that occurs at the time of the button press.From the third chronological data marker, downstream event (DSE), two additional dataflags were derived and additionally used to categorize the data: 5 seconds prior to thedownstream event (-5 DSE), and 5 seconds following the downstream event (+5 DSE). Thedownstream events (referred to in Figure R1) were identical across message relevance (relevant,irrelevant messages), a between-subjects manipulation; only the messages themselves varied as afunction of modality and complexity within each of the relevance conditions. For both theprimary and secondary task measures, the results for messages that were coupled with hazardevents immediately following the message presentation were analyzed separately from those thatwere not coupled.37

Primary Task PerformanceThe main performance measures for the primary (driving) task were collected from theperiod of time from the message presentation (MP) to 5 seconds prior to the downstream event(-5 DSE). The logic behind the use of the truncated timeline rests on the notion that participantswill have effectively processed the visual and auditory messages 5 seconds prior to thedownstream event, and therefore this time period should reflect the greatest impact of thedemands and interference of processing the message.The time period from 5 seconds prior to the downstream event to 5 seconds following thedownstream event more addresses the compliance of the driver to the relevant messagesthemselves. Compliance measures will be addressed in the secondary task performance section.Driving performance during the latter period of time identifies whether the operator trulyunderstood the meaning and significance of the message.Lane KeepingAn omnibus ANOVA comparing message modality, message relevance, and messagecomplexity was examined for the dependent measures of lane position. Lane position wasmeasured by the absolute distance deviation (in meters) of the vehicle relative to the center of thelane.Figure R2 plots lane deviations as a joint function of modality and message relevance forthe nonhazard data (those eight out of ten messages per driving environment that did not includea hazard event). The analysis of these data revealed that modality of message delivery had noeffect on lane keeping (F(2,68) = .43; p = .65), although there was a small, but significantincrease in lane deviations during and after the delivery of all message types, relative to the valueof 0.43 measured during baseline driving when no message was present (t(107) = 3.52; p < .01).There was a non significant trend for relevant messages to disrupt lane keeping more thanirrelevant messages (F(1,34) = 2.60, p = .12), a trend that appears (in Figure R2) to be enhancedwith the auditory presentation, although the modality x relevance interaction did not approachsignificance (F(2,68) = 1.731; p = .185).38

Tracking ErrorModality x Relevance0.6Tracking Error (LanePos.)0.50.40.30.20.10Auditory Redundant VisualRelevantIrrelevantModalityFigure R2. Tracking error (lane position) in meters by relevance and modality.Neither the main effect of complexity, nor its interaction with the other variables (seen inFigure R3), approached conventional levels of significance (F(1,34) = 1.06; p = .31; F(2,68) =1.06; p = .35). Thus, in general, and replicating Horrey and Wickens (2002), drivers did a goodjob of protecting driving lane keeping performance from differential interference caused by thedifferent aspects of the message task, although messages of all types and modalities equallycaused some disruption. The redundant information compared to single-modal information didnot show a significant advantage; for all conditions, the redundant display was the average of thetwo single-modal displays for tracking performance.Tracking ErrorModality x Complexity0.6Tracking Error (LanePos.)0.50.40.30.20.10Auditory Redundant VisualSimpleComplexModalityFigure R3. Tracking error (lane position) by modality and complexity.39

An omnibus ANOVA comparing message modality and message relevance for trackingerror during the Hazard events (those two out of ten messages per driving environment thatincluded a hazard event), additionally showed no significant effects for the main effects ofmodality, relevance, nor the interaction between modality and relevance (F(2,68) = 1.6; p = .21;F(1,34) = .55; p = .46; F(2,68) = 1.5; p = .23). The effect of message property on the delay inresponding to the hazard will be discussed below.Steering VelocityAn omnibus ANOVA comparing message modality, message relevance, and messagecomplexity was examined for the dependent measure of RMS steering velocity. Steering velocitywas measured by the root mean squared error of the vehicle’s steering wheel velocity (the rate ofchange of the position of the steering wheel over time). In the absence of differences in lanekeeping error, we generally assume that conditions triggering greater velocity are moredisruptive to driving performance.Figure R4 plots steering wheel velocity as a joint function of modality and complexityduring non-hazard trials; a measure of baseline steering wheel velocity performance is alsoincluded as a comparison between single and dual task steering control performance. The figurepresents the RMS velocity during the time period from the message presentation to 5 secondsprior to the downstream event. For the nonhazard data (these are the eight messages that are notcoupled with unexpected events), there are main effects of message modality and complexity onsteering wheel velocity (respectively F(2,68) = 4.54; p = .01; F(1.34) = 5.56; p = .02). Thesteering velocity is lower with displays that have an auditory component; thus, drivers are able topay more attention to the roadway and be less affected in their steering control when they arepresented with auditory rather than visual messages (or when their eyes are on the roadwayinstead of on a visual display). More complex messages also result in greater steering velocity,and the significant interaction between complexity and modality (F(2,68) = 2.84; p = .06)indicates that this complexity effect increases as the presence of the auditory component isreduced.40

RMS Velocity30282624222018NONHAZARD Steering Wheel VelocityModality x ComplexityAuditory Redundant VisualComplexSimpleBaselineFigure R4. Steering wheel velocity by modality and complexity, with a comparison of baselinesteering velocity control, for non-hazard events.Neither the main effect of relevance, nor its interaction with the other variables(modality; complexity) approached significance (relevance: F(1,34) = .356, p = .555; modality xrelevance: F(2,68) = .982, p = .38; complexity x relevance: F(1,34) = 2.31, p = .138). Overall, theeffect of redundancy compared to the single-modality information displays is averaged betweenthe two single modalities. Finally, there was a significant increase in steering wheel velocityduring the instances of dual-task relative to single-task driving processing (t(107) = 5.7, p < .01).Considering the effects on steering behavior, in conjunction with those of the lanekeeping results, show that though drivers protect the primary tracking task lane keeping fromvisual distraction, their steering behavior is more disrupted with the presentation of a messagewhen they need to look down at a visual display and when the information presented in thedisplay is complex (of greater length). Drivers exhibit more abrupt steering maneuvers in thesecircumstances which would indicate that they are correcting for their removal of attention fromthe roadway to long (complex) messages on the visual display.Figure R5 plots steering wheel velocity as a joint function of modality and relevanceduring hazard events (two out of the ten messages on each drive); with baseline steering controlagain included as a comparison between single and dual task steering control performance. Notethe much greater steering activity in response to the hazard, compared to the non-hazard data inFigure R4. The baseline performance is statistically equivalent across relevance (F(1,34) =2.309; p = .14) and therefore plots the average value of the two relevance conditions. For thehazard data, the only significant effect is a main effect of relevance on steering wheel velocity(F(1,34) = 5.67, p = .02), with relevant messages having less steering velocity than irrelevantmessages; drivers are more aggressive in steering control when they encounter a hazard eventwhen attempting to process irrelevant messages than when processing a relevant message. Thisdifference may reflect the fact that relevant messages kept the driver’s attention relatively moreattuned to the roadway (independent of delivering modality) and hence less disrupted when the41

hazard occurred. The velocities seen in the irrelevant conditions show an overcompensation of asteering maneuver (when compared with the baseline condition, t(107) = 15.6, p < .01), whichwould cause the driver to veer dangerously into the oncoming lane when avoiding a hazard.HAZARD Steering Wheel VelocityModality x RelevanceRMS Velocity706050403020100Auditory Redundant VisualIrrelevantRelevantBaselineFigure R5. Steering wheel velocity by modality and relevance, with a comparison of baselinesteering velocity control, for hazard events.Hazard ResponseEach driver encountered 2 events per driving environment for a total of 6 hazard events.The following 6 unique hazard events were presented to each participant: 1) car incursion; 2)bicycle incursion; 3) right turn across path (wide right turn); 4) lane drift (curve); 5) lane drift(hill); 6) car pullout. The 6 events were pooled for data analysis in order to increase the statisticalpower of the analyses and to prevent the occurrence of a Type II error. The hazard response wasdetermined from the 15-second timeline of continuous vehicle control (see Figure R1). Thehazard response time is defined by the time period from the initiation of hazard movement to thefirst instance of a response to the hazard.To increase the interpretability and reliability of hazard response data, a hazard responsewas classified and analyzed according to two different criteria: 1) the first occurrence of either azero acceleration value (removing foot from accelerator), a braking response, or a steeringresponse; 2) the occurrence of either a braking response or a steering response. For the firstcriteria, a removal of the foot from the accelerator was considered a cautionary movement and aprecursor to a braking response; the accelerator value was only considered as a response measurewhen it was subsequently followed by a braking response. The second analysis was run onbraking and steering response alone as these maneuvers are considered more intentional andmore certainly indicative of a safety response than an accelerator release, although these mayoverestimate the actual response time. An accelerator release could potentially be confoundedwith speed adjustments, driver inattention, or the automatic response of drivers to the message inthe “relevant” condition.42

In contrast to lane keeping, hazard response was more disrupted by differences in themessage interface. Figure R6 presents the time till the initial hazard response was recorded viameasure 1 (either accelerator release, braking, or swerving) as a joint function of modality andmessage relevance. A between-subjects ANOVA comparing message modality and relevancewas conducted in order to minimize the number of deleted cells (due to instances of nonresponses to the hazard events) in the data analysis. It will be recalled that hazards were onlypresented during the complex messages. The main effect of modality (F(2,96) = 3.48; p =.035),reflecting the generally higher costs of visual display, can best be interpreted within the contextof the marginally significant relevance x modality interaction (F(2,96) = 2.62: p=.08). The datashow that for relevant messages, there was a strong tendency for a cost to visual presentation,and a benefit for auditory presentation, with the redundant combination falling in between. Forirrelevant messages, modality had little effect. The main effect of message relevance was alsonot significant. A factorial ANOVA comparing modality for relevant messages alone confirmedthat the interaction was driven by the relevant condition showing a significant effect of modalityequal to that seen in the between-subjects ANOVA (F(2,28) = 3.8, p = .035). A one-wayANOVA on the irrelevant condition showed no significant effect of modality (F(2,30) = .538, p= .59). As Figure R6 shows, there is an approximately half-second slowing of hazard responseassociated with the visual presentation of relevant messages, a time during which a vehicletraveling at 55 mph could cover 50 feet. The redundant condition was the average of the twosingle-modal conditions across relevance for hazard response.2Hazard Reaction TimeModality x RelevanceRT1.81.61.41.2RelevantIrrelevant1Auditory Redundant VisualModalityFigure R6. Reaction time to hazard events by relevance and modality.The analysis of hazard response with criteria 2 confirms the effect of display type onhazard response found with the first analysis (which included accelerator response as adependent measure). Figure R7 presents the time until the initial hazard response was recorded(either braking or swerving) as a joint function of modality and relevance. A between-subjectsANOVA was again conducted to minimize deleted cells; the analysis again revealed a maineffect of modality (F(2,97) = 4.33; p = .016). Though the relevance x modality interaction did43

not reveal a significant effect, a t-test between the auditory relevant condition and the auditoryirrelevant condition show a marginally significant benefit to the auditory presentation of relevantinformation (t(33) = 1.83, p = .076). The second analysis supports the trends found in the firstanalysis for hazard response times.Hazard RT - Brake/Steering ONLYModality x RelevanceRT21.81.61.41.21Auditory Redundant VisualModalityRelevantIrrelevantFigure R7. Reaction time to hazard events by modality and relevance.The number of non-responses to hazards was also examined (i.e., instances when analysisof the continuous vehicle control data failed to reveal a braking or swerving event), and thisrevealed a pattern similar to that found for hazard response time. The number of such eventsacross the auditory, redundant and visual conditions respectively were 2 (2 relevant, 0irrelevant), 12 (4 relevant, 8 irrelevant), and 18 (9 relevant, 9 irrelevant), (Χ 2 (2) = 12.25, p < .01).Drivers were less likely to see the occurrence of the particular hazard events when they wereviewing a visual display than when they were using an auditory-component display; an effectthat was relatively uninfluenced by display relevance. This follows intuitively that taking theeyes of the road, even for a split second, can lead to disastrous consequences, such as notnoticing a bicycle pulling out from between two cars. As with other measures, performance inthe redundant conditions appears to lie in-between the two pure modality conditions.Secondary task message responseThe time to initiate a response to the message presentation was assessed from theinitialization of the first phoneme (in the auditory and redundant conditions) until the first pressof the response button (located on either side of the steering wheel). For the visual conditions,the time to initiate a response was measured from the initial presentation of the message on theIVT display until the first press of the response button. An omnibus ANOVA comparingmessage modality, complexity, and relevance was conducted for both non-hazard and hazardmessage sets.44

Figure R8 plots the time to initiate a response to the secondary message task, as afunction of modality and message complexity, for messages delivered in the absence of a hazard.There was no effect of relevance in response time nor interaction of relevance with modality.These data indicate a main effect of modality (F(2,68) = 34.27, p = .01), suggesting a slowing ofresponse to all messages with an auditory component (auditory and redundant), relative to thepure visual message. As the figure suggests, responses to complex messages were initiated moreslowly than to simple messages (F(1,34) = 66.83; p

RELEVANT Message (IVT) Reaction TimeModality x Relevance x ComplexityIRRELEVANT Message (IVT) Reaction TimeModality x Relevance x Complexity5544RT32SimpleComplexRT32SimpleComplex110Auditory Redundant Visual0Auditory Redundant VisualModalityModalityFigure R9. Three-way interaction of reaction time of button press to message presentation byrelevance (relevant – left side; irrelevant – right side), complexity and modality.One explanation for the slowed response for the auditory modality displays is due to theobservation that most participants waited to press the response button (instructed to be depressedat the beginning of the vocalization) until the auditory message was completely vocalized; this islikely due to participants attempting to avoid crosstalk, which is the when the response(s)relevant for one task (in this case the verbal read-back) overlaps with the processing for adifferent task (in this case the auditory vocalization; Fracker & Wickens, 1989). Though theinstructions attempted to eliminate this effect, most participants still fell into the habit of waitinguntil the vocalization finished instead of pressing the button as soon as they heard the auditorymessage. For the redundant conditions, the auditory modality dictated the response time of theparticipants as they again waited until the completion of the vocalization of the message beforethey responded. The redundant modality exhibited the worst of both worlds relative to the singlemodalities.The analysis of response time to messages that occurred just prior to the occasionalhazard event (or in conjunction with the occurrence of hazard events) revealed tradeoffs toperformance between the primary and secondary tasks. These data, shown in Figure R10,indicated, again, a main effect of modality (F(2,68) = 5.7; p < .01), with visual responses fasterthan auditory, and a main effect of relevance (F(1,34) = 6.05, p = .02) with relevant messagesresponded to more rapidly than irrelevant ones. The interaction between relevance and modalitywas not significant (F(2,68) = 1.71, p >.10). We note that the mean RT in Figure R10 (mean =2.80) was a half-second longer than the mean RT in Figure R8 (mean = 2.32) which indicatesthat our timing of events was generally successful in getting drivers to perceive and respond tothe message prior to dealing with the hazard. These data are consistent with the view that thefaster responses to relevant and visual messages compromised the responses to hazard events(Figure R6). Based on the analysis of response time to hazard events, visual relevant messages(as opposed to auditory relevant messages) are particularly at risk of diverting the driver’sattention away from the road during a crucial instance requiring a quick response to situations inthe road ahead.46

HAZARD Message (IVT) Reaction TimeModality x RelevanceRT43.532.521.510.50Auditory Redundant VisualModalityRelevantIrrelevantFigure R10. Reaction time of button press to message presentation during hazard events bycomplexity and modality.Finally, we examined the accuracy of the read-backs of the task messages as a function ofmodality, complexity and relevance. These read-backs took place after the participant hadpressed the button indicating a recognition of the message presentation; the read-back wasdesigned to ensure that the participant had perceived the message. This analysis revealed that inalmost all conditions, accuracy was high, (around 93%) and did not vary. The only exceptionwas the low (65%) accuracy of messages that were irrelevant, complex, and delivered without avisual component (auditory only), a similar observation to that observed by Helleberg andWickens (2003).Sometimes, appropriately, drivers allowed their perception of the hazard to preempt andprolong their response to the IVT, despite the fact that the IVT appeared first. Interestingly, whenhazard and non-hazard IVT responses were combined in a single ANOVA, this revealed, inaddition to the significant half-second slowing caused by the hazard (F(1,34) = 27.0, p < .01), ahazard x relevance interaction (F(1,34) = 5.72, p = .022), which indicates that there was greaterhazard-induced slowing of the IVT RT for irrelevant than for relevant messages. This is turnsuggests that drivers were more able to “disengage” for the IVT task when it was irrelevant. Thiseffect was not modulated by modality.Comprehension AnalysisIn an effort to ensure that the semantic content of messages was effectively processed bythe participants, comprehension measures were analyzed for both relevant (downstreamcompliance) and irrelevant (category accuracy) messages. The comprehension measures(relevant and irrelevant) were analyzed as a function of message modality and complexity. Thenon-hazard and hazard data were analyzed separately.47

Relevant messagesThe time period from 5 seconds prior to the downstream event (-5 DSE) to 5 secondsfollowing the downstream event (+5 DSE) was analyzed to assess the comprehension, asoperationally defined by the downstream compliance, with the relevant messages. Drivingbehavior downstream determined whether the driver comprehended (or complied to) the relevantmessage. Each message’s 15-second timeline was examined graphically to identify compliancemeasures (brake, steering, acceleration) in the -5 DSE to +5 DSE period.A factorial ANOVA comparing message modality and message complexity wasexamined for the relevant downstream compliance measures for non-hazard events. A one-wayANOVA compared the effect of modality for the relevant downstream compliance measures forhazard events. A compliance response was classified and analyzed according to the followingcriteria: 1) the occurrence of a braking response, 2) the occurrence of a steering response; or 3)the occurrence of a zero accelerator value. Any combination of these variable, in addition to thesingular values, was considered a definitive response.Three levels of compliance were used to code the data. A steering or braking response (orthe combination of the two) was considered a complete compliance response and was coded withthe value of 1. A acceleration response was considered a partial compliance response and wascoded with a value of 0.5. The lack of either a steering, braking, or acceleration response wasconsidered a null compliance response and was coded with a value of 0.Across conditions, participants were 70.5% compliant to the relevant messages in theabsence of a hazard. Figure R11 plots the compliance measures for the relevant messages as afunction of modality and complexity, for messages delivered in the absence of a hazard. The dataindicate a main effect of complexity (F(1,17) = 5.16, p = .036), suggesting a more accurateresponse to complex messages relative to simple messages. Increased complexity of the messageprovides greater detail in describing the downstream event, which may account for the improvedcompliance to more complex messages. The main effect of modality and the modality xcomplexity interaction did not show significance (F(2,34) = 1.278, p = .30; F(2,34) = .327, p =.72). Note that the fact that 30% of the messages did not produce observable behavior ofcompliance does not imply that these were not processed. We do know that all messages wereread back. It is possible that, for some of these 30%, drivers did not assess that a change invehicle state was required. On the other hand, the presence of the appropriate downstreammaneuver does not guarantee that the semantic implications of the relevant message wasprocessed, because we did not collect data on vehicle control behavior to the same downstreamevents, in the absence of any messages whatsoever.48

NONHAZARD Compliance MeasureModality x Complexity% accuracy10.80.60.40.2SimpleComplex0Auditory Redundant VisualModalityFigure R11. Compliance accuracy to non-hazard events by modality and percent accuracy, forrelevant messages.For hazard events, the analysis of compliance measures for modality did not revealsignificance (F(2,34) = .571, p = .57). Across modality, participants were 63% complaint to therelevant messages in the presence of a hazard. The relevant messages enacted compliance wellabove the mean for both the hazard and non-hazard events, showing that participants did indeedattend to the messages and adjust their driving behavior appropriately to account for thedownstream events.Irrelevant messagesA factorial ANOVA comparing message modality and message complexity wasexamined for comprehension of the irrelevant messages in the absence of hazards. A one-wayANOVA comparing the effect of modality for the irrelevant comprehension measures wasconducted for hazard events. A comprehension response was analyzed according to the accuracyof the irrelevant message categorization. Participants were to correctly categorize the irrelevantmessages according to the following four categories: 1) radio, 2) e-mail, 3) news, and 4) weather.Three levels of accuracy were used to code the irrelevant data. An accurate assignment ofa category type to the irrelevant message was considered a complete comprehensive responseand was coded with the value of 1. An inaccurate assignment of a category type to the irrelevantmessage was considered a partial comprehensive response and was coded with a value of 0.5.The lack of an assignment of a category type to the irrelevant message was considered a nullcomprehensive response and was coded with a value of 0.Figure R12 plots the comprehension measures for the irrelevant messages as a function ofmodality and complexity, for messages delivered in the absence of a hazard. We note first thatthe overall level of accuracy was high (90.1%), assuring us that drivers did pay attention to the49

meaning of the messages. The factorial ANOVA did not show any significant effects; themanipulations of message modality and complexity (and their interaction) did not producesignificant variance among the data (modality: F(2,34) = .095, p = .91; complexity: F(1,17) =1.449, p = .25; modality x complexity: F(2,34) = .027, p = .97).NONHAZARD ComprehensionModality x Complexity% correct10.80.60.40.20Auditory Redundant VisualModalitySimpleComplexFigure R12. Comprehension accuracy to non-hazard events by modality and percent accuracy,for irrelevant messages.For the messages delivered in the presence of a hazard, the analysis of accuracy measuresfor modality similarly did not reveal significance (F(2,34) = .61, p = .552). The overall accuracyacross modality however was at 94%, showing again that participants were very accurate in theircategorization of the irrelevant messages, even when a hazard occurred.Thus, the compliance/comprehension analysis reveals that the relevant and irrelevantmessages were representative of their categories. Participants attended to the messages and wereapt at deriving the meaning contained in such messages.Individual Differences in Hazard Detection PerformanceIn order to assess the safety-critical drivers, those who present a hazard to others due tothe least safe driving behaviors, a set of “worst-case” performance measures were examined. Inparticular, the reaction time to hazard events for the worst-case performers was analyzed. Thedata for the 6 poorest performers (the slowest reaction times to the hazard events) was used torepresent the worst performers across all subjects. The means collapsed across modality wereused to determine the 6 poorest performers for each level of relevance (a between-subjectsmanipulation). The individual data points for the 6 worst performers within conditions was thenincluded in the data analysis.Figure R13 shows the data for the slowest 6 performers as a function of messagemodality and relevance. A between-subjects ANOVA failed to show a main effect of modality orrelevance (F(2,16) = 1.379, p = .28; F(1,8) = 0.0, p = .998 respectively). Additionally, the50

interaction between message modality and relevance did not show significant (F(2,16) = .214, p= .81). However, the means support the trends seen for the entire subject population (see FigureR6); showing that the worst drivers have tendencies that fall within the normal range of drivers.Though not statistically significant, the graphical trends in the data for the worst performerssupports the idea that relevant messages result in a benefit to auditory presentation and a cost tovisual presentation. Most importantly, the data reveal two things: 1) The slow responders in ourstudy do not show a pattern that is qualitatively different from other responders, and 2) theirmean response time was not exceptionally greater than that of the population as a whole.Hazard RT Worst Performers2.52RT1.510.5RelevantIrrelevant0Auditory Redundant VisualModalityFigure R13. Response time to the hazard events by modality and relevance for the slowest 6performers.DISCUSSIONThe current study was designed to assess the factors that could moderate the differencesbetween auditory and visual delivery of in-vehicle information, primarily as such informationaffected measures of driving performance, and secondarily as such factors would influence theprocessing of the in-vehicle information itself. In particular, we were interested in threemoderating factors: task relevance, task complexity, and the nature of the driving performancevariable (lane keeping versus hazard response). Finally, we were interested in how the use of aredundant combination of auditory and visual delivery could also moderate these effects.Resource Competition and Task InterferenceIn many respects our design was similar to, and our results replicated the effects obtainedby Horrey and Wickens (2002). Like them, we found that modality differences were moreevident in the secondary in-vehicle task, than in the primary task performance measure of lanekeeping, but did note a small overall decrement in primary task driving associated with thepresence of the side task. As in Horrey and Wickens (2002, 2003), we found no modality effecton lane keeping, but a slight visual cost to steering, and a substantial cost to unexpected hazarddetection (both in response time, and response likelihood), associated with the head down visualdisplay of IVT information. These effects support both of our first two hypotheses.51

Both the findings of dissociation between lane keeping (no decrement) and hazardresponse (visual decrement), and the findings of the visual head down cost to hazard response areconsistent with, respectively, the visual channels distinction and the perceptual modalitiesdistinctions of the multiple resource theory (Wickens, 2002). Stated in other terms, hazarddetection and visual IVT information competed for focal visual resource and created interferencein hazard response (Figure R6); but the combination of visual IVT (focal vision) and lanekeeping (ambient vision) exploited separate resources, availing better parallel processing (FigureR3), as also did the combinations of auditory IVT and both lane keeping and hazard detection,exploiting separate modality-defined resources. The finding of auditory modality benefits todriving performance are certainly consistent with the prevalence of modality-comparison studies,using head down displays, reviewed in the Introduction (see Figures I1-I3, and Wickens &Seppelt, 2002).Thus in this study, as in many others, it appears that any auditory costs of pre-emption aregenerally dominated, or at least offset by the visual costs of scanning to the head-down location(and /or requirement to use peripheral vision). Further evidence against the role of an auditorypreemption mechanism operating in the current paradigm, was provided by the finding that theperformance of the side task did not particularly benefit from auditory delivery (Figure R9), abenefit which, had it been observed in conjunction with a driving performance cost to auditorydelivery, would have been consistent with the pattern of effects typical of preemption (Wickens& Liu, 1988; Helleberg & Wickens, 2003; Wickens, Dixon, & Seppelt, 2002). It appears that thisauditory preemption cost only begins to emerge when the visual side task performance isimproved by presenting it at small spatial separations; and even then, the costs are notconsistently observed (Horrey & Wickens, 2002).We do note, in the current data, that the absence of benefit to auditory IVT processingcould be a complete artifact of the driver’s delay strategy in initiating the key press afterreadback was complete. Had our timing finished at the start of voice articulation, we mightindeed have found an auditory benefit to side task delivery, equally consistent with preemptionand multiple resources.Significantly, the current study added to the findings of Horrey and Wickens (2002), andprovided a unique contribution to all of the literature by varying message relevance within asingle experimental paradigm. The latter was done in order to test the third hypotheses,suggested by between-experimental comparisons reviewed in the Introduction, that the benefitsto auditory delivery over visual would be amplified, if that information were relevant to thedriving task (e.g., IVIS information) in contrast to irrelevant information (e.g., e-mail). Thecurrent results appear to be consistent with this hypothesis of a modality and relevanceinteraction. When the message was relevant, and therefore potentially easier to associate with thedriving task, drivers appeared to capitalize upon multiple resource theory, and to be morevigilant for hazards when the messages were delivered aurally than visually. On the other hand,when the message was irrelevant, modality appeared to have little influence on performancedriving performance. Importantly, this latter finding does not advocate visual presentation forirrelevant information. The possible costs of visual display of such information needs to befurther explored because, at wide eccentricities, there would still be expected to be visualresource competition, and only a carefully conceived allocation policy could be guaranteed toprotect the primary task.52

Message relevance, in addition to modulating the effect of modality, appeared to haveone other main effect on driving performance: when messages were irrelevant, and driversstarted processing them, the drivers appeared to be more disrupted by the hazards, as thisdisruption was indexed by the increase in steering wheel velocity (i.e., making more aggressivemaneuvers). Although the link between disruption and steering velocity is an indirect one, it isplausible to postulate such a link, given other associations in the data between steering velocityincrease, and the presence of the concurrent IVT task (see Figure R4).Regarding our fourth hypothesis, we had postulated two alternative effects of increasingIVT complexity. On the one hand, a majority of studies reviewed in the Introduction hadrevealed that increasing complexity of the in-vehicle task, by increasing the amount of visualhead-down time when the IVT was delivered visually, would lead to an increasing visual cost (ordecreasing visual advantage), relative to auditory delivery. On the other hand, some studies in anon-driving environment (e.g., Helleberg & Wickens, 2003) had provided at least some evidencefor an increasing cost of auditory preemption as complexity became greater. This because driversstrategically would need to keep attention fixated on the longer auditory message, in order that itnot be lost from working memory; whereas a correspondingly longer visual message couldsimply be processed with repeated scans at optimal times, while not diverting extra attentionfrom the roadway environment. According to such a perceptual hypothesis, the resources that arediverted are not sensory, but rather, are modality-independent perceptual-cognitive resources,diverted longer to the auditory message than to the visual. This conception is consistent with oneput forth by Latorella (1998).In fact, our current results were more consistent with the first explanation (visualscanning) than the second (strategic auditory preemption). Increasing message complexity, ifanything, produced a greater, rather than a reduced auditory advantage to driving performance.While it is true that long complex auditory messages suffered in their own comprehension(particularly when these messages were irrelevant, Figure R9), this increasing cost was notevident in driving task performance, as drivers again protected the higher-valued task (driving).The trend that we observed here, of side task’s complexity influence on driving performance, isquite consistent with the results observed in the more basic paradigm of Wickens, Dixon, andSeppelt (2002). There we also found that increasing side task complexity (the length of a phonenumber) improved tracking performance in the auditory, relative to the visual condition. Thedifference however between those results and the current findings, is that at the lowest level ofcomplexity, there was an auditory cost (visual benefit) to tracking performance, which indicatedsome form of onset preemption (Spence & Driver, 1997), which was not observed in the currentstudy.Regarding our fifth hypothesis, the current study also added to the very scarce pool ofliterature that has examined AV redundancy assistance in dual task conditions. Of the fewstudies that were located, only one (Liu, 2001), had examined this issue in presenting in-vehicleinformation, observing that redundancy offered the “best of both worlds” of the two single taskmodalities using the 5-level categorization scheme developed by Wickens and Gosney (2003).The current results, in contrast, showed no redundancy gain. While the single modality auditorycondition, as noted above, was generally equal to or better than the single modality visualcondition, in the latter case (auditory better than visual), there were no performance measures inwhich the redundant display exceeded the auditory display, and, indeed, in nearly all cases, an53

“averaging” model best captured the results. That is, offering redundant visual informationappeared to hamper performance relative to pure auditory delivery, presumably by invitingvisual distraction.In this regard however it is important to note that the redundant condition never offeredthe “worst of both worlds”, a syndrome that had been observed in earlier research (Wickens,Goh, Helleberg, Horrey, & Talleur, in press; Helleberg & Wickens, 2003; Seagull, Wickens, &Loeb, 2001; Wickens and Gosney, 2003). Of these studies, the one that revealed the consistent“averaging” model, typical of the data found here, was Wickens and Gosney (2003) experiment2, in which participants were explicitly trained on the strategies for exploiting redundant AVdisplays. Hence it is possible that drivers here spontaneously adopted this strategy. Alternatively,it is possible that explicit training of the strategy to drivers could have brought out the “best ofboth worlds” for the redundant display, or, better still, the “Gestalt” pattern in which redundancyprovides an advantage over either single modality display. This possibility is one that awaitsfurther investigation.Relevance to Attention TheoriesThe findings of the current study can be considered in the context of different models ofattention. The current data clearly support a multiple resource model, at least as multipleresources are defined by visual (focal ambient) channel distinctions, and by peripheral sensorymodalities (auditory-visual). Strayer and Johnston (2001) have offered an alternative view thatthe competition between driving and in-vehicle technology is governed by allocation of a morecentral “attention”. Strayer and Johnston’s results do not contradict a multiple resource view, asthey did not vary the modality of arriving perceptual information, which would be necessary totest predictions of input-defined multiple resources.At the same time, the current results are quite consistent with the Strayer and Johnstonattentional model, in that some component of interference between driving (lane keeping) andIVT processing was modality independent, accounting for the small increase in lane keepingerror associated with processing all IVT message, even those delivered auditorally. There wasalso an associated increase in steering activity, associated with all messages (Figure R4). Itremains unclear from the current data, whether the source of this modality-independentinterference, is competition between driving and IVT processing for general perceptual-cognitiveresources, competition for common response resources , or even competition for some verygeneral “executive control” mechanism. It is plausible, but cannot be determined conclusively bythe above data, that the source may be competition for response-related resources (generating thearticulation of the message readback, concurrently with generating the motor commands forsteering), given the specific association of lane-keeping with steering activity, and given thegenerally limited capacity of the human to generate responses (Pashler, 1998; Wickens, 2002).Such an association of the modality-independent component with response conflict, rather thancognition, is supported by the fact that the response velocity effect increased with messagecomplexity (and therefore with the length of the readback articulation; Figure R4), but was notaffected by the cognitive component of the message (relevance to driving), except during thehazard events.54

Design Implications, and LimitationsThe most important design implications of the current results, are that verbal IVIS (i.e.,driving-relevant) information should not be displayed on a head-down console. Auditorypresentation is preferable in protecting the response to the unexpected roadway hazard. Of coursesuch information can be presented in a head-up display, not tested here (see Horrey & Wickens,2002). The current results also suggest caution in augmenting an auditory display with aredundant visual head-down text display, as this option appears to mitigate the advantages ofmultiple resources offered by the auditory display of driving-relevant information. Training mayeffectively eliminate the redundancy costs, and avail the “best of both worlds”. However analternative solution, if the auditory message is complex or long and may be forgotten, is simplyto provide an easy-to-use repeat function, perhaps on a steering wheel mounted button.A second implication is that the auditory advantage does not hold for driving irrelevantinformation. Such information may always offer greater distraction from driving (than relevantinformation). The processing of such information was more hampered by the driving task (whenit was complex and auditory), and more disrupted driving steering behavior, when a hazard waspresent. Such findings signal a general cost to processing driving-irrelevant information displays.The extent to which this cost might be mitigated by head up display remains to be established.The current study has certain limitations, which dictate some caution in generalizing thecurrent results. First, we only evaluated redundant presentation of verbal information. As notedin the Introduction, there may be advantages of distributing related visual spatial (i.e., map) andverbal information across modalities, presenting the latter as speech rather than text (Wickens &Hollands, 2000). Second, our evaluation took place in the driving simulator, rather than on theroadway. It is possible that our effects might not generalize to the latter environment, inparticular because of the more realistic motion cues associated with actual vehicle control.However such motion cues would have been expected to better support the tracking (i.e., lanekeeping) aspect of driving, than to support the hazard detection aspect. In fact, if anything, therealistic motion feedback of on-the-road vehicle control, could be expected to supplement thoseof ambient vision, allowing effective lane keeping with even more head down time, and therebyamplifying the visual penalty to hazard response, compared to the simulator results obtainedhere. Thirdly, as noted, we did not fully assure relevant message compliance as, to do so, wouldhave required a central condition, containing the same downstream events, with no messages atall.Finally, the potential generality of the current results could be challenged because thesedid not fully evaluate the “worst case” situations, which often contribute to accidents. Our hazardresponses were not fully unexpected, and even the “worst performers” in our study did notrespond much slower to the hazards than did the sampled population as a whole. On the onehand, this latter finding is important, in revealing that the qualitative pattern of interference weobserved, characterizing the “average” driver, can generalize to those slower responders, whomay present the greatest concerns to driver safety. On the other hand, the issue of generalizingexperimental results, with their dependence on generally expected conditions, conventionalstatistics, and the ”psychology of the mean”, is one that will always challenge experimentalresearch (Wickens, 2001), and is a reason why the implications of such research should always55

e joined with more epistemological or naturalistic studies of drivers, based upon accident andincident analysis.ACKNOWLEDGMENTSSpecial thanks to Bill Horrey, Jeff Mayhugh, and Nick Cassavaugh for help withsimulation programming and integration of simulation and script design. Many thanks to RonCarbonari for assistance with results analysis, data collection and reduction. Thanks to HankKaczmarski and Braden Kowitz for providing, maintaining, and installing simulator software andequipment, and an additional thanks to Debbie Carrier for her help with scheduling subjects inthe simulator. Thanks also to Art Kramer for his helpful input and comments.REFERENCESAntin, J.F., Dingus, T.A., Hulse, M.C., & Wierwille, W.W. (1990). An evaluation of theeffectiveness and efficiency of an automobile moving map navigational display.International Journal of Man-Machine Studies, 33, 581-594.Baddeley, A.D. (1986). Working memory. Oxford: Clarendon Press.Baddeley, A.D. (1995). Working memory. In M.S. Gazzaniga et al. (Eds.), The cognitiveneurosciences (pp. 755-784). Cambridge, MA: MIT Press.Braune, R., & Wickens, C.D. (1985). The functional age profile: An objective decision criterionfor the assessment of pilot performance capacities and capabilities. Human Factors, 27,681-694.Burnett, G.E., & Joyner, S.M. (1997). An assessment of moving map and symbol-based routeguidance systems. In Y.I. Noy (Ed), Ergonomics and safety of intelligent driverinterfaces. Human factors in transportation (pp. 115-137). Hillsdale, NJ: LawrenceErlbaum Associates.Dingus, T.A., Hulse, M.C, McGehee, D.V., & Manakkal, R. (1994). Driver performance resultsfrom the Travtek IVHS camera car evaluation study. Proceedings of the 38 th AnnualMeeting of the Human Factors & Ergonomics Society (pp. 1118-1122). Santa Monica,CA: Human Factors Society.Dingus, T.A., McGehee, D.V., Manakkal, N., Jahns, S.K., Carney, C., & Hankey, J.M. (1997).Human factors field evaluation of automotive headway maintenance/collision warningdevices. Human Factors, 39(2), 216-229.Folds, D.J., & Gerth, J.M. (1994). Auditory monitoring of up to eight simultaneous sources.Proceedings of the 38 th Annual Meeting of the Human Factors Society (pp. 505-509).Santa Monica, CA: Human Factors Society.Fracker, M. L., & Wickens, C. D. (1989). Resources, confusions, and compatibility in dual axistracking: Displays, controls, and dynamics. Journal of Experimental Psychology: HumanPerception and Performance, 15, 80-96.Gish, K.W., Staplin, L., Stewart, J., & Perel, M. (1999). Sensory and cognitive factors affectingautomotive head-up display effectiveness. Proceedings of the 78 th Annual TransportationResearch Board. Washington, DC: Traffic Safety Division.56

Helleberg, J. & Wickens, C. D. (2003). Effects of data-link modality and display redundancy onpilot performance: An attentional perspective. The International Journal of AviationPsychology, 13(3), 189-210.Horrey, W., & Wickens, C. (2002). Driving and side task performance: The effects of displayclutter, separation, and modality (AHFD-02-13/GM-02-2). Savoy, IL: University ofIllinois, Aviation Human Factors Division.Horrey, W.J. & Wickens, C.D. (2003). Multiple resource modeling of task interference in vehiclecontrol, hazard awareness and in-vehicle task performance. Proceedings of DrivingAssessment 2003: 2nd International Driving Symposium on Human Factors in DriverAssessment, Training, and Vehicle Design. Park City, UT.Hughes, P.K., & Cole, B.L. (1986). What attracts attention when driving? Ergonomics, 29, 377-391.Hulse, M.C., Dingus, T.A., & Barfield, W. (1998). Description and applications of advancedtraveler information systems. In W. Barfield, & T.A. Dingus (Eds.), Human factors inintelligent transportation systems. Human factors in transportation (pp.359-395).Hurwitz, J., & Wheatley, D.J. (2002). Using driver performance measures to estimate workload.Proceedings of the 46 th Annual Meeting of the Human Factors and Ergonomics Society(pp. 1804-1808). Santa Monica, CA: Human Factors and Ergonomics Society.Isreal, J. (1980). Structural interference in dual-task performance: Behavioral andelectrophysiological data. Unpublished doctoral dissertation, University of Illinois,Champaign.Isreal, J., Wickens, C.D., Chesney, G., & Donchin, E. (1980). The event-related brain potentialas a selective index of display monitoring load. Human Factors, 22, 211-224.Labiale, G. (1990). In-car road information: Comparisons of auditory and visual presentations.Proceedings of the 34 th Annual Meeting of the Human Factors Society (pp. 623-627).Santa Monica, CA: Human Factors Society.Lamble, D., Laakso, M., & Summala, H. (1999). Detection thresholds in car following situationsand peripheral vision: Implications for positioning of visually demanding in-car displays.Ergonomics, 42(6), 807-815.Lansdown, T.C., Brook-Carter, N., & Kersloot, T. (2002). Primary task disruption from multiplein-vehicle systems. ITS Journal, 7(2), 151-168.Latorella, K.A. (1998). Effects of modality on interrupted flight deck performance: implicationsfor data link. Proceedings of the 42 nd Meeting of the Human Factors and ErgonomicsSociety (pp. 87-91). Santa Monica, CA: Human Factors Society.Lee, J.D., Gore, B.F., & Campbell, J.L. (1999). Display alternatives for in-vehicle warning andsign information: Message style, location, and modality. Transportation Human Factors,1(4), 347-375.Liu, Y-C. (2001). Comparative study of the effects of auditory, visual, and multimodalitydisplays on drivers’ performance in advanced traveler information systems. Ergonomics,44(4), 425-442.57

Liu, Y., & Wickens, C.D. (1992). Visual scanning with or without spatial uncertainty anddivided and selective attention. Acta Psychologica, 79, 131-153.Matthews, G., Sparkes, T., & Bygrave, H. (1996). Attentional overload, stress, and simulateddriving performance. Human Performance, 9(1), 77-101.Mollenhauer, M.A, Lee, J., Cho, K., Hulse, M.C., & Dingus, T.A. (1994). The effects of sensorymodality and information priority on in-vehicle signing and information systems.Proceedings of the 38 th Annual Meeting of the Human Factors and Ergonomics Society(pp. 1072-1075). Santa Monica, CA: Human Factors and Ergonomics Society.Noy, Y.I. (1990). Attention and performance while driving with auxiliary in-vehicle displays.Ottawa: Transport Canada, Road Safety.Parkes, A.M., & Burnett, G.E. (1993). An evaluation of medium range “advance information” inroute-guidance displays for use in vehicles. IEEE Vehicle Navigation & InformationSystems Conference. Ottawa, Canada.Pashler, H. E. (1998). The psychology of attention. Cambridge, MA: The MIT Press.Polson, M.C., & Friedman, A. (1988). Task-sharing within and between hemispheres: Amultiple-resources approach. Human Factors, 30, 633-643.Ranney, T.A, Harbluk, J.L., & Noy, Y.I. (2002). The effects of voice technology on test drivingperformance: Implications for driver distraction. Proceedings of the Human Factors andErgonomics 46th Annual Meeting (pp. 1814-1818). Santa Monica, CA: Human Factorsand Ergonomics Society.Rockwell, T. (1972). Skills, judgment, and information acquisition in driving. In T. Forbes (Ed.),Human factors in highway traffic safety research (pp. 133-164). New York: WileyInterscience.Sanders, M.S., & McCormick, E.J. (1993). Human factors in engineering and design (7 th ed.).New York: McGraw-Hill.Sarno, K.J., & Wickens, C.D, (1995). Role of multiple resources in predicting time-sharingefficiency: Evaluation of three workload models in a multiple-task setting. InternationalJournal of Aviation Psychology, 5(1), 107-130.Schons, V., & Wickens, C.D. (1993). Visual separation and information access in aircraftdisplay layout (ARL-93-7/NASA-A3I-93-1. Savoy, IL: University of Illinois, AviationResearch Laboratory.Seagull, F. J, Wickens, C.D., & Loeb, R.G. (2001). When is less more? Attention and workloadin auditory, visual and redundant patient-monitoring conditions. Proceedings of the 45 thAnnual Meeting of the Human Factors & Ergonomics Society. Santa Monica, CA:Human Factors & Ergonomics Society.Serafin, C., Wen, C., Paelke, G., & Green, P. (1993). Car phone usability: A human factorslaboratory test. Proceedings of the 37 th Annual Meeting of the Human Factors andErgonomics Society (pp. 220-224). Santa Monica, CA: Human Factors and ErgonomicsSociety.58

Sorkin, R.D. (1987). Design of auditory and tactile displays. In G. Salvendy (Ed.), Handbook ofhuman factors (pp. 549-576). New York: Wiley.Spence, C., & Driver, J. (1997). Audiovisual links in attention: Implications for interface design.In D. Harris (Ed.), Engineering psychology and cognitive ergonomics. Hampshire:Ashgate Publishing.Srinivasan, R., & Jovanis, P. (1997). Effect of selected in-vehicle route guidance systems ondriver reaction times. Human Factors, 39(2), 200-215.Srinivasan, R., Yang, C., Jovanis, P., Kitamura, R., & Anwar, M. (1994). Simulation study ofdriving performance with selected route guidance systems. Transportation Research,2C(2), 73-90.Stokes, A., Wickens, C., & Kite, K. (1990). Display technology: Human factors concepts.Warrendale, PA: Society of Automotive Engineers.Strayer, D. L. & Johnston, W. A. (2001). Driven to distraction: Dual-task studies of simulateddriving and conversing on cellular telephone. Psychological Science, 12(6), 462-466.Streeter, L.A., Vitello, D., & Wonsiexicz, S.A. (1985). How to tell people where to go:Comparing navigational aids. International Journal of Man-Machine Studies, 22(5), 549-562.Summala, H., Nieminen, T., & Punto, M. (1996). Maintaining lane position with peripheralvision during in-vehicle tasks. Human Factors, 38(3), 442-451.Tindall-Ford, S., Chandler, P., & Sweller, J. (1997). When two sensory modes are better thanone. Journal of Experimental Psychology: Applied, 3(4), 257-287.Tsang, P.S., & Rothschild, R.A. (1985). To speak or not to speak: A multiple resourceperspective. Proceedings of the 29 th Annual Meeting of the Human Factors Society (pp.76-80). Human Factors Society.Tsang, P.S., & Wickens, C.D. (1988). The structural constraints and strategic control of resourceallocation. Human Performance, 1, 45-72.Walker, J., Alicandri, E., Sedney, C., & Roberts, K. (1990). In-vehicle navigation devices:Effects on the safety of driver performance (p. 107). McLean, VA: Performer: FederalHighway Administration, Office of Safety and Traffic Operations Research andDevelopment.Weinstein, L.F., & Wickens, C.D. (1992). Use of nontraditional flight displays for the reductionof central visual overload in the cockpit. International Journal of Aviation Psychology,2(2), 121-142.Wickens, C.D. (1980). The structure of attentional resources. In R. Nickerson (Ed.), Attentionand performance VIII (pp.239-257). Hillsdale, NJ: Lawrence Erlbaum.Wickens, C.D. (1991). Processing resources and attention. In D. Damos (Ed.), Multiple taskperformance. London: Taylor & Francis.Wickens, C.D. (1992). Engineering psychology and human performance (2 nd ed.). New York:HarperCollins.59

Wickens, C.D. (2002). Multiple resources and performance prediction. Theoretical Issues InErgonomics Science, 3(2), 159-177.Wickens, C.D., Braune, R., & Stokes, A. (1987). Age differences in the speed and capacity ofinformation processing: I. A dual-task approach. Psychology and Aging, 2, 70-78.Wickens, C.D., Dixon, S., & Seppelt, B., (2002). In-vehicle displays and control taskinterference: The effects of display location and modality (ARL-02-7/NASA-02-5/GM-02-1). Savoy, IL: University of Illinois, Aviation Research Lab.Wickens, C.D., & Goettl, B. (1984). Multiple resources and display formatting: The implicationsof task integration. In Proceedings of the 28 th Annual Meeting of the Human FactorsSociety (pp. 722-726). Santa Monica, CA. Human Factors Society.Wickens, C., Goh, J., Helleberg, J., Horrey, W., & Talleur, D. (in press). Attentional models ofmulti-task pilot performance using advanced display technology. Human Factors.Wickens, C.D., Gordon, S., & Liu, Y. (1998). An introduction to human factors engineering.New York: Addison Wesley Longman.Wickens, C.D., & Gosney, J.L. (2003). Redundancy, modality, and priority in dual taskinterference. Submission to Human Factors.Wickens, C., Helleberg, J., Goh, J., Xu, X., & Horrey, W. (2001). Pilot Task Management:Testing an Attentional Expected Value Model of Visual Scanning (ARL-01-14/NASA-01-7). Savoy, IL: University of Illinois, Aviation Research Lab.Wickens, C.D., & Hollands, J. (2000). Engineering psychology and human performance (3 rded.). New York: Prentice Hall.Wickens, C.D., & Liu, Y. (1988). Codes and modalities in multiple resources: A success and aqualification. Human Factors, 30, 599-616.Wickens, C. D., Sandry, D., & Vidulich, M. (1983). Compatibility and resource competitionbetween modalities of input, output, and central processing. Human Factors, 25, 227-248.Wickens, C.D., & Seppelt, B.D. (2002). Interference with driving or in-vehicle task information:The effects of auditory versus visual delivery (AHFD-02-18/GM-02-3). Savoy, IL:University of Illinois, Aviation Human Factors Division.Wright, P., Holloway, C.M., & Aldrich, A.R. (1974). Attending to visual or auditory verbalinformation while performing other concurrent tasks. Quarterly Journal of ExperimentalPsychology, 26, 454-463.Yeh, Y.-Y., & Wickens, C.D. (1988). The dissociation of subjective measures of mentalworkload and performance. Human Factors, 30, 111-120.60

APPENDIX A. PARTICIPANT INFORMATIONSubject Gender AgeHealth(1:5)* VisionValid Driver'sLicenseMileage/Yr.1 Female 20 4 20/20 Yes 52 Male 21 5 20/25 Yes 70003 Male 20 5 20/20 Yes 50004 Male 19 5 20/20 Yes 10005 Male 19 5 20/20 Yes 50006 Male 20 5 20/20 Yes 5007 Male 20 5 20/20 Yes 100008 Male 21 5 20/30 Yes 100009 Male 23 5 20/20 Yes 600010 Male 21 4 20/20 Yes 550011 Male 23 5 20/20 Yes 1400012 Male 23 4 20/20 Yes 1600013 Male 22 5 20/20 Yes 600014 Female 28 5 20/20 Yes 1200015 Male 20 4 20/20 Yes 2000016 Male 21 5 20/20 Yes 800017 Male 26 5 20/20 Yes 1200018 Male 22 5 20/20 Yes 500019 Male 21 5 20/20 Yes 1000020 Male 21 5 20/20 Yes 1000021 Male 20 5 20/20 Yes 500022 Male 20 4 20/20 Yes 800023 Male 21 5 20/20 Yes 1600024 Female 19 5 20/20 Yes 10025 Female 20 4 20/20 Yes 100026 Male 24 4 20/20 Yes 1300027 Male 20 4 20/20 Yes 200028 Male 21 5 20/20 Yes 250029 Male 21 4 20/20 Yes 2500030 Male 31 5 20/20 Yes 600031 Male 21 4 20/20 Yes 400032 Male 22 4 20/20 Yes 400033 Male 19 5 20/20 Yes 1500034 Male 25 5 20/20 Yes 1000035 Male 23 5 20/20 Yes 500036 Male 22 5 20/20 Yes 15000* Health rated on a 5-unit Likert scale with 1 = Poor and 5 = Excellent61

APPENDIX B. IN-VEHICLE DISPLAY SPECIFICATIONSIN-VEHICLE DISPLAY*HeightWidthHorizontal Visual SeparationVertical Visual Separation6.0 in.8.25 in.From the center of the steering wheel to thetop left of the IV display: 20.14 degreeslaterally;From the center of the steering wheel to themiddle top of the IV display: 26.57 degreeslaterallyFrom the line of sight (approximately at thelocation of the pedestrian walk sign seen inthe photograph) to the top of the IVdisplay: 7.6 degrees down;From the line of sight to the center of theIV display: 13.1 degrees downDISPLAY TEXTIndividual LettersSentencesHeight 1.0 cm. 1.0 cm.Width 0.6 cm Min: 8.0 cm. - Max: 12.0cm. †*The in-vehicle (IV) display is noted by the bracket on the right side of the photograph.† Sentences were formatted to wrap the text, thus the minimum and maximum valued reflect therange of possible word lengths to fit within the display.62

APPENDIX C. WITHIN-WORLD MESSAGE ORDER AND SECONDARY TASKCODING FOR RELEVANT AND IRRELEVANT CONDITIONSRELEVANT CONDITION:Relevant MessageDriving Environment 1Construction Ahead.Merging vehicle ahead. Beware ofblind driveways.Read-backAccuracy *CommentsSharp curve ahead. Slow to 35MPH.Pedestrian crossing ahead.Slow vehicle ahead.Accident in road ahead. Slow to45 MPH.Emergency vehicle in lane ahead.Blind spots in curved road ahead.Railroad crossing ahead.Object in road ahead.Heavy traffic ahead. Mergingtraffic from the right.Driving Environment 2Slow vehicle ahead. Curve inroad.Merging traffic from the left.School zone ahead. Watch forchildren.Accident in road ahead.Slippery road ahead.63

Object in road ahead.Construction ahead.Traffic congestion ahead.Detour to the right.Speed reduction ahead. SpeedLimit 35 MPH.Heavy traffic ahead. Mergingtraffic from the right.Driving Environment 3Merging vehicle ahead. Beware ofblind driveways.Approaching emergency vehicle.Pedestrian crossing ahead.Railroad crossing ahead.Accident in road ahead.Heavy traffic ahead. Mergingtraffic from the left.Object in road ahead. Usecaution.Construction ahead. Detour to theright.School zone ahead. Watch forchildren.Slow vehicle ahead.* 1 if accurate; .5 if partially accurate; 0 if not accurateMessage Bolded when unexpected events occur64

IRRELEVANT CONDITION:CorrespondingIrrelevantMessageDriving Environment 1LiteRock 98.7.Information requestreceived. Sendreceipt toAccounting.Artic chilldeepens. Drop to –10 degrees.Alternative Rock107.1.Terrorist leaderarrested.Mostly sunny andwarm. Highs inthe 70s.Afternoon flurriesare expected today.Temperatures risingin the earlyevening.Holiday partytonight.San Francisco bansSegway.Smooth Jazz V98.7.Home of the Trip-A-Day Give-Away.Driving Environment 2Tower of Power –What Is Hip?Girl missing afterplane crash.Light snowshowers. Wind toincrease.Smith Job Talktomorrow.ReadbackAccuracy*IrrelevantMessageCategoryRadioE-mailWeatherRadioNationalNewsWeatherWeatherE-mailNationalNewsRadioRadioNationalNewsWeatherE-mailCategoryAccuracy* Comments65

War protestorsarrested.Drew and MikeShow.Partly cloudy.Chemicalwarheads found.Inspectors tocontinue search.Research positionavailable.Applicationdeadline June 5 th .Contact Linda Carr.Office hours from3-5 daily.Driving Environment 3National Censustaken. Latinoslargest U.S.minority.Volleyball practicetonight.Light showerstoday.Library itemavailable.Ex-Serb leaderfaces tribunal.Low temperaturestoday. Cold frontfrom the NorthHighs in the 60s.Sunny tomorrow.94.7 WCSX. Rockto the Hits.Smooth Jazz104.3. Tunes toGroove.Send project fax.NationalNewsRadioWeatherNationalNewsE-mailE-mailNationalNewsE-mailWeatherE-mailNationalNewsWeatherWeatherRadioRadioE-mail* 1 if accurate; .5 if partially accurate; 0 if not accurateMessages Bolded when unexpected events occur66

APPENDIX D. EXAMPLE OF CORRESPONDING RELEVANT AND IRRELEVANTDOWNSTREAM EVENTS67

APPENDIX E. SIMULATOR SICKNESS QUESTIONNAIREThis study will require you to drive in a driving simulator. In the past, some participants havefelt uneasy after participating using the simulator. To help identify people who might be proneto this feeling, we would like to ask the following questions.1. Do you or have you had a history of migraine headaches?If yes, please describe:2. Do you or have you had a history of claustrophobia?If yes, please describe:3. Do you or have you had a history of motion sickness?If yes, please describe:4. Are you or is there a possibility that you might bepregnant?5. Any health problems that affect driving?6. Lingering effects from stroke, tumor, head trauma, infection?7. Suffer from epileptic seizures?8. Any inner ear problems, dizziness, vertigo, or balance problems?9. Are you currently taking any medications?If yes, please list:68

APPENDIX F. PRE-EXPERIMENTAL QUESTIONNAIREPoor ExcellentAge:_____ Sex: M F Handedness: R L Health: 1 2 3 4 5Do you wear Glasses/Contacts on a regular basis? NYHow many years of school have you completed?___________________Phone Number:_______________ E-mail:_______________________Can we call you to participate in additional experiments? YesNoWhere did you hear about us?__________________________________Signature of Participant:________________________Date:___________Name:______________________________________________________(Please Print)Social Security Number_______-_____-_____************************************************************For Office Use OnlyFar Vision:_______ Near Vision:________Color-Blindness:_______Time IN:_________ Time OUT:_______ Total Time:____________Pay for Exp:_______ Parking:__________ Total Pay:_____________69

APPENDIX G. INFORMED CONSENT FORMMultiple Resource Modeling of the Impact of In-Vehicle Technology on Driver WorkloadResearch supported by the General Motors CorporationPrincipal Investigator: Dr. Christopher WickensInstitute of AviationAviation Human Factors DivisionWillard Airport#1 Airport RoadSavoy, IL 61874The purpose of this experiment is to provide data on the sources of in-vehicle distraction. That is, we wishto determine the extent to which in-vehicle technology, such as cell phones, electronic map displays, or e-mail displays diverts the driver's gaze away from the highway. We also wish to establish the extent towhich auditory presentation of some of this information, or presentation on a head-up display, can reducethe distracting effects of such technology, or may actually, increase those distractions.To examine these issues, you will be asked to drive our Saturn driving simulator while performing otherside tasks about which you will be instructed. You should drive as you normally would on the highway. Onsome occasions, we may ask you to wear a small camera attached by a band around your head whichcan record the direction of your gaze. You will report to room B500 Beckman for the experiments.Depending on the particular experiment, it will last from 1-3 - one hour sessions.Eye movements are monitored by a device that reflects infrared light off of the lens and the cornea of theeye. The lens, cornea and other parts of the eye absorb a small amount of energy from the infrared light,but the energy is less than 1% of the Maximum Permissible Exposure level as certified by the AmericanStandards Institute (ANSI Z 136.1-1973). This is about as much energy as you get on a bright sunny day.There are no known risks or physical discomforts associated with this experiment beyond those ofordinary life, and the possibility that the simulation might cause some mild motion sickness. If it does so,please tell the experimenter. You will be paid at the rate of $6/hr. You may terminate your participation atany time, and you will still be paid for the number of hours that you have completed.We thank you for your involvement. If you have any further questions, please let the experimenter knowat any time throughout the experiment, or call Dr. Wickens at 244-8617. If you have any questions aboutthe rights of research subjects, please contact the University Institutional Review Board at 217/333-2670.Statement of ConsentI acknowledge that my participation in this experiment is entirely voluntary and that I am free to withdrawat any time. I have been informed of the general scientific purposes of this experiment and I know that Iwill be compensated at a rate of $6.00/hour for my participation. If I withdraw from the experiment beforeits termination, I will receive my total fee earned to that time. I understand that my data will be maintainedin confidence, and that I may have a copy of this consent form.Signature of participant: _______________________________Signature of experimenter: ______________________________Date :____________Date: __________70

APPENDIX H. EXPERIMENTAL PROTOCOLAdminister Eye ExamProvide participant with Driving Simulator QuestionnaireStart up numbers.bat program on computerStart up VariableSender program on computerEnter subject data in Subject fieldProvide participant with informed consentThank you very much for participating in this study. It should take you approximately 1-1/2 to 2hours to complete. I would like to remind you that you are free to withdraw from this study atany time. Please look through the informed consent.Participant reads / signs form.Administer pre-experimental questionnaire.Today you will be asked to drive through 3 different scenarios. Each drive lasts approximately20-25 minutes. There will be a short break between each of the three drives to ensure that youare not experiencing any symptoms of simulator sickness. The drives consist of curved andstraight sections of rural and residential roads. Some hills will be encountered. During eachdrive, you will be asked to perform a secondary task in addition to the primary driving task (thistask will be described in detail momentarily).Seat participant in the driving simulator. Adjust seat and mirrors to fit to the driver.The controls in the vehicle will respond as they would in normal driving. Please operate thevehicle as you would normally and properly navigate the vehicle to ensure safe driving. It isimportant that you obey all traffic laws and abide by the posted speed limits. The speed limit inrural sections of the drive is 55 MPH. Stay within 5 MPH of this speed. The speed limit inresidential sections is 35 MPH. Stay within 5 MPH of this speed. If your speed exceeds or fallsbelow the specified range, you will be informed and asked to adjust your speed accordingly. Inaddition to maintaining a proper speed limit, we ask that you keep the vehicle in the center of thelane throughout the drives. Please respond to traffic events as you would normally (addressesresponse to IVTs). No stop signs or stoplights are present in the drives. The transitions betweenrural and residential sections should be smooth and are indicated by speed limit signs.In order to familiarize you with the driving task and the different driving environments (rural andresidential), I will now run you through a brief 2-minute practice drive. Please familiarizeyourself with the controls in the vehicle and attend to the posted speed limits.(Note: motion sickness may be a side effect of driving in the simulator. You will be asked on anumber of occasions during each drive if you feel sick; please report ANY feelings of nausea ordiscomfort. If you begin to feel sick, please inform me so that we stop the experiment in order toprevent you from experiencing further discomfort.)71

Start practice trial. (If participant experiences motion sickness, end the experiment and thank theparticipant for their time. Offer payment for one hour if they end the experiment after thepractice trial).RELEVANT CONDITIONS:As I mentioned earlier, you will be performing a secondary task in addition to the driving task.Messages that carry direct relevance to the driving task will be presented to you eitherauditorially, visually, or both auditorially and visually. These messages are examples of thefuture intelligent transportation systems. The messages will inform you of events or conditionsthat will take place in the road ahead, such as traffic congestion, or school zones. You willreceive a message approximately every 2 minutes; a total of 10 messages will be presented toyou in every scenario. The visual messages will appear on the display located next to the steeringwheel in the center console. The auditory messages will sound from the car speakers. I willinform you before each scenario in which modality (visual, auditory, or auditory-visualcombined) the messages will be presented to you.When the messages either appear on the display or sound through the speakers (or both), you arerequired to read them back word for word. As soon as you notice the appearance of the messageon the screen or hear the auditory message, press either one of the two buttons located on thesteering column (point to the buttons to illustrate) a single time. Begin your read-back of themessage precisely when you press the steering wheel button (or immediately following theauditory message in the auditory and auditory-visual combined conditions). In the visualconditions, the message will stay on the screen for 10 seconds to increase the opportunity for youto notice and read-back the message. In the auditory conditions, the message will be vocalizedonly once. As you would in normal driving with such a system, or in reading road signs, it isimportant that you comply with the information contained in such messages, and use thatinformation to prepare yourself for the situation ahead.IRRELEVANT CONDITIONS:As I mentioned earlier, you will be performing a secondary task in addition to the driving task.“Infotainment” messages that contain information on National News, Weather, Radio stations, orE-mail topics will be presented to you either auditorially, visually, or both auditorially andvisually. The messages will be of one type of information with each presentation; however, the 4different types of information will be presented to you randomly through each drive. You willreceive a message approximately every 2 minutes; a total of 10 messages will be presented toyou in every scenario. The visual messages will appear on the display located next to the steeringwheel in the center console. The auditory messages will sound from the car speakers. I willinform you before each scenario in which modality (visual, auditory, or auditory-visualcombined) the messages will be presented to you.When the messages either appear on the display or sound through the speakers (or both), you arerequired to read them back word for word. As soon as you notice the appearance of the messageon the screen or hear the auditory message, press either one of the two buttons located on thesteering column (point to the buttons to illustrate) a single time. Begin your read-back of the72

message precisely when you press the steering wheel button (or immediately following theauditory message in the auditory and auditory-visual combined conditions). In the visualconditions, the message will stay on the screen for 10 seconds to increase the opportunity for youto notice and read-back the message. In the auditory conditions, the message will be vocalizedonly once. After you read-back the message, you must categorize the message by verballyreporting the particular category of information in which it is associated with: News, Weather,Radio, or E-mail. Please be as accurate as possible in your categorization of the message.(The following instructions are relevant to ALL conditions)Again, the messages will be presented to you approximately every 2 minutes.Do you have any questions concerning the secondary task?You should attend to the IVT messages as promptly as possible but also make sure that youproperly navigate for safe driving.Reminder: In response to the secondary task, press a button on the steering wheel as soon as themessage is noticed, then read-back the message in its entirety (this read-back should be at thesame time as when you press the button upon noticing the message OR immediately following theauditory message), (In Irrelevant Conditions: and categorize the message according to theparticular category from which it comes – either News, Weather, Radio, or E-mail). Driveaccording to the speed limit and maintain the vehicle in the center of the lane.Again, please be mindful of any feelings of nausea or discomfort during each of the three drivesand report them immediately.((For the control condition) In this scenario, no secondary task will be presented to you.Respond to traffic events and obey traffic laws as you would normally. Please remember to keepyour speed limit within 5 MPH of the posted speed.)After completion of the 3 experimental blocks, thank participants for their participation andreimburse them for their time.Provide participant with post-experimental questionnaire.73

APPENDIX I. COUNTERBALANCED TRIAL ORDERSubjectRelevance Modality World Order1 Relevant A1 V2 AV3 bds1_A bds2_V bds3_AV2 Irrelevant AV3 V2 A1 bds3_AV_im bds2_V_im bds1_A_im3 Relevant V2 A1 AV3 bds2_V bds1_A bds3_AV4 Irrelevant A2 V1 AV3 bds2_A_im bds1_V_im bds3_AV_im5 Relevant AV3 A1 V2 bds3_AV bds1_A bds2_V6 Irrelevant V1 A2 AV3 bds1_V_im bds2_A_im bds3_AV_im7 Relevant AV2 A1 V3 bds2_AV bds1_A bds3_V8 Irrelevant V3 AV1 A2 bds3_V_im bds1_AV_im bds2_A_im9 Relevant V2 AV3 A1 bds2_V bds3_AV bds1_A10 Irrelevant A2 AV3 V1 bds2_A_im bds3_AV_im bds1_V_im11 Relevant A1 AV2 V3 bds1_A bds2_AV bds3_V12 Irrelevant AV1 A2 V3 bds1_AV_im bds2_A_im bds3_V_im13 Relevant A2 V3 AV1 bds2_A bds3_V bds1_AV14 Irrelevant A1 AV3 V2 bds1_A_im bds3_AV_im bds2_V_im15 Relevant AV1 A3 V2 bds1_AV bds3_A bds2_V16 Irrelevant AV2 A3 V1 bds2_AV_im bds3_A_im bds1_V_im17 Relevant V3 AV2 A1 bds3_V bds2_AV bds1_A18 Irrelevant V3 A1 AV2 bds3_V_im bds1_A_im bds2_AV_im19 Relevant V3 A2 AV1 bds3_V bds2_A bds1_AV20 Irrelevant V2 AV1 A3 bds2_V_im bds1_AV_im bds3_A_im21 Relevant AV1 V2 A3 bds1_AV bds2_V bds3_A22 Irrelevant AV1 V3 A2 bds1_AV_im bds3_V_im bds2_A_im23 Relevant A2 AV1 V3 bds2_A bds1_AV bds3_V24 Irrelevant A3 V1 AV2 bds3_A_im bds1_V_im bds2_AV_im25 Relevant AV3 V1 A2 bds3_AV bds1_V bds2_A26 Irrelevant A3 AV2 V1 bds3_A_im bds2_AV_im bds1_V_im27 Relevant A3 AV1 V2 bds3_A bds1_AV bds2_V28 Irrelevant V2 A3 AV1 bds2_V_im bds3_A_im bds1_AV_im29 Relevant V1 A3 AV2 bds1_V bds3_A bds2_AV30 Irrelevant AV2 V3 A1 bds2_AV_im bds3_V_im bds1_A_im31 Relevant V1 AV3 A2 bds1_V bds3_AV bds2_A32 Irrelevant AV3 A2 V1 bds3_AV_im bds2_A_im bds1_V_im33 Relevant A3 V2 AV1 bds3_A bds2_V bds1_AV34 Irrelevant V1 AV2 A3 bds1_V_im bds2_AV_im bds3_A_im35 Relevant AV2 V1 A3 bds2_AV bds1_V bds3_A36 Irrelevant A1 V3 AV2 bds1_A_im bds3_V_im bds2_AV_im74

APPENDIX J. POST-EXPERIMENTAL QUESTIONNAIRE1. Do you have a valid Driver’s License? Yes No2. How many years have you had a Driver’s License? ___________________3. About how many miles per year do you drive? ___________ miles/year4. How many moving violations have you had in the last two years? _______________5. Have you had any accidents where you were responsible?YesNo_________________________________________________________________________________Use the following scale to respond to questions 4 through 7.1 2 3 4 5Never 1-2 times per month 3-4 times per month 3-4 times per week Everyday6. How often do you drive? 1 2 3 4 57. How often do you drive on city streets? 1 2 3 4 58. How often do you drive on rural / country roads? 1 2 3 4 59. How often do you drive on freeways? 1 2 3 4 5Use the following scale to respond to questions 10 through 17.1 2 3 4 5Never Rarely Occasionally Often AlwaysIf you were in a hurry to get to an important appointment how often would you (remember there are no right or wronganswers):10. Run a red light to get to an appointment sooner 1 2 3 4 511. Drive at 5-15 mph over the speed limit 1 2 3 4 512. Drive around lowered gates at a railway crossing 1 2 3 4 513. Speed in a school zone on a Saturday 1 2 3 4 514. Do a rolling stop through a stop sign(i.e. not a complete stop ) 1 2 3 4 515. Tailgate other people to get them to drive faster 1 2 3 4 516. Get angry at other drivers for being in your way 1 2 3 4 517. Talk on the cellular phone 1 2 3 4 5Use the following scale to respond to questions 18 through 21.1 2 3 4 5Never Rarely Occasionally Often Always18. I felt nauseous in the driving simulator. 1 2 3 4 519. The driving simulator allowed me to brake appropriately. 1 2 3 4 520. The gas pedal and brake in the simulator allowed me toadequately control my speed. 1 2 3 4 521. The steering of the driving simulator allowed me to make maneuverscorrectly. 1 2 3 4 575