Interactive Systems Need Safety Locks - IEEE Xplore

Interactive Systems Need Safety LocksHarold ThimblebyFuture Interaction Laboratory, FIT LabSwansea University, Walesharold@thimbleby.netAbstract. User interfaces often behaveunpredictably on erroneous input — rather thanblocking errors and requiring the user to correctthem. The consequences of this in the context ofmedical devices, which may give patientsundetected overdoses, can be unfortunate. Thesolution should include better design, includingthe concept of safety locks, that block someforms of user error.Keywords. Safety locks, human error, numberentry, user interface design.1. IntroductionMachine guns have triggers, which the usersqueezes to make them fire. Machine guns alsohave safety locks so that they cannot be fired byaccident. Safety locks do not signficantlyinterfere with normal use of a gun, but theysignificantly reduce the probability of accidentalmisuse.Why do guns have safety locks? Well, weknow two things: guns are dangerous, and eventhe best-trained users make slips from time totime (guns may also be left unattended). Safetylocks significantly reduce the chances ofunintended firing.Like a machine gun, any system controlled bya human requires some sort of trigger so that itcan be made to operate when its user wants tooperate it. (Of course many systems are morecomplex than guns: the actions triggered mayrequire further control and refinement.)By analogy, any system that may haveunwanted effects should have its analogues tosafety locks. More specifically:• Any system that may have unwantedeffects that are large compared to thefinal effort to make them happen, or arehard to undo should have safety locks. 11 Safety locks are thus one solution to breaking thelaw of commensurate effort [8].The only argument against safety locks is thatthey may add complexity or cost to a design andmay in themselves cause types of error thatwould not or could not occur without them.For example, safety locks on guns mayslightly slow down shooters taking their firstshot. Somebody may then be shot because theirgun was locked in safe mode and was notimmediately usable to protect them. Almostcertainly it does not make sense for a safety lockto have its own safety lock, so a more consideredposition is that any system with potentiallyunwanted effects should have a safety lock (orequivalent) unless the safety lock itself increasesthe risk of problems more than the problems it issupposed to reduce. In other words, a safety lockis a design trade-off.Even if we want to avoid safety locks so gunsare faster to use, it is also a priority to ensure thata gun user doesn’t get killed by their own gun! Auser being killed by accident would be far moreinefficient than all of the small delays of using asafety lock each time the gun is needed to bedeliberately fired.Computer systems are like machine guns. Asimple action, perhaps intentional, perhaps a slip,can have enormous and quite unintendedconsequences.Imagine: a single keystroke triggers sendingan unfortunate email to thousands of people. Abutton press triggers a patient receiving anoverdose of a drug. Clicking “save” overwrites afile the user wanted to keep, and there is no undoto recover all the work that has been lost.Where are the safety locks when they areneeded?This question is wrapped up within deeper,contextual questions: sometimes the user reallywants to do something unfortunate but whichonly too late they realise was unfortunate. Theuser might really want to send the email, andperhaps nothing the computer can do will make itsafer, and the user only realises their error when29Proceedings of the ITI 2010 32 nd Int. Conf. on Information Technology Interfaces, June 21-24, 2010, Cavtat, Croatia

thousands of complaints start coming back tothem much later.Paul Cairns observes that “safety lock” is aconcept more general than what is needed onobviously dangerous devices like guns. Inelectrical wiring systems, there are fuses. A fuseblowing is an inconvenience (it has to bereplaced) but it stops an electrical fault of somesort escalating into perhaps a fire. A climber’ssafety rope stops them falling uncontrollably. Itdoes not stop them falling in the first place, andit may not save their lives (occasionally safetyropes are cut to save other people’s lives, so thatnot everyone is pulled off together), but a safetyrope is a simple device that increases safety atsmall cost.In this paper, whilst we use machine guns as adramatic background motivation for safety locks,conceptually we see them as a far more generalidea, and ones that could be very widely used inmany sorts of applications.Just because there are different sorts of safetylock and sometimes complicated answers to thequestions does not mean we should ignorefundamental issues…2. Ignoring safety locks in programmingA leading undergraduate textbook onalgorithms [2], “the bible” of the field accordingto its blurb, explicitly says that it does not covererror handling. Almost 1,000 pages of textassumes all data is correct, and, if so, how toprocess it. The book has no sanity checks, noassertions, nothing on exception handling — inshort, no safety locks. This is an example of howwe train undergraduate programmers.If any data originates from a human — ororiginates from a program written by a human —there is no guarantee the data is error-free. In theworst case, the program it is fed into may behavesort-of like a machine gun and have untowardconsequences.3. Ignoring safety in numbersNumber entry is one of the most basic andwidely-encountered tasks.A user presses keys, say, 1, 2, •, then 3 (whereI am using • to mean a decimal point, becausesmall dots are hard to see and confusable withperiods and other English punctuation) 2 and then2 The symbol embossed on the key may be •. On mycomputer keyboard, • is combined with >, but onthe computer will convert the sequence of keypresses, 12•3, to the number 12.3. Often the userwill have to press ENTER or GO or some otherkey to indicate when they have finished enteringthe number.Although we write 12•3 or 12.3 (the keys)and 12.3 (the number), to a computer these arequite different concepts, and one has to becarefully converted to the other. Humansgenerally think of numbers as particularsequences of digits rather than as abstract values,so the subtle differences between keys, numeralsand numbers are often glossed over. Recall thatcomputers use binary; what we write in thispaper as 34, say, may be represented inside thecomputer as 100010 — and even then one wouldsearch in vain to find anything that looks like100010, as the 1s and 0s are represented aselectrical states, not as anything humanlyreadable. Indeed, our habitual ways of talkingabout numbers makes them look very easy, andcorrespondingly makes it quite hard to grasp howcomplex number entry really is.A typical approach to converting keystrokesto numbers is as follows.The computer starts off with the number 0.When a user hits a digit key, the number ismultiplied by 10 and the value of the digit (notthe key itself!) is added to it. So if the userpresses the key with the label ‘3’ on it then thecalculation 0×10+valueof(key3) = 3 isperformed, and then the computer has thenumber 3. This may seem rather obvious andperhaps a little complex for what it achieves, butexactly the same process will work on the nextkey too. If the user keys ‘4’ next, another digitkey, then the same process is repeated: 3×10+valueof(key4) = 34.In this example, as can be seen, the user keyed‘3’ then ‘4’ and the computer determined thatthey entered the numerical value 34. The processis repeated so long as the user is keying digits.Figure 1 shows the algorithm as it would bewritten in a typical programming language.many devices, the • is on a dedicated key. Someelectronics and program working together convertsthe location of the key on the keyboard (and whetherSHIFT has been pressed, etc) to the ASCII orUnicode for a dot. Programmers rarely concernthemselves with such details: in a normal program •always looks like “.” Sometimes the program code forconverting keystrokes to codes is faulty, and then theuser experiences so-called key bounce problems, suchas ignored or extra keys seeming to be pressed.30

var n = 0;key = getKey();while( isDigit(key) ){ n = 10*n+valueOf(key);key = getKey();}return n;Figure 1. Simple code to read a numberThe simple algorithm described above worksvery nicely until the user keys a dot or DELETE,or keys so many digits that the computer losestrack of the value of the large number intended;then things get more complex and the algorithmceases to be so simple…We will not pursue the elementaryprogramming details further in this paper.Nevertheless it is interesting to note that whenthe user has done nothing, the computer isalready thinking zero. However, nothing andzero are not the same, and perhaps this confusion— on the computer’s part — will cause whatmay seem like user errors from time to time.Here’s how:Many devices display the initial number as0.0, which may further confuse the user, as 0.0cannot possibly distinguish between a user whohas keyed nothing, keyed zero, keyed a decimalpoint, keyed a decimal point followed by zero …and so on. To illustrate the problem: if a userwalks up to a device displaying 0.0 and keys 5,the display could legitimately but unpredictablygo to any of these values: 5.0, 0.5 or 0.05. Thisdoes not seem satisfactory for devices that areused in safety critical environments, such as inhealthcare.A user may make a slip when enteringnumbers. Hypothetically, consider that they press12••3. This is clearly not a number; it is not wellformed.What does the program do? Does it treatthis as a “trigger squeeze” for some number orother, or does it recognize it as an inappropriatesequence of actions for number entry, and safelylock it out?If the computer’s number entry algorithm isas trivial as the one sketched above, then theprogram will get as far as 12 and then terminatewith 12.0 as the number, rather than an error.Programmers rarely program number entrycode themselves: it is needed so often it is oftenbuilt-in so that there is a way of doing itautomatically.Consider the programming languageJavaScript, which is one of the most widely usedprogramming languages, as it is used to scriptweb browsers. In JavaScript, keystrokes can beconverted to numbers automatically, and fewprogrammers worry about how this worksbecause it is so easy to do. Yet erroneoussequences of characters are incorrectlyconverted. For example, 1, 2, •, •, 3 is convertedto “12” — it is treated as a valid number, but isgiven the wrong value.JavaScript provides several ways to convertsequences of keystrokes to numbers; forexample, the built-in routine parseFloat convertsthe same string of characters 12..3 into “NaN” —now it is treated as “Not a Number.” It is then upto the programmer to check that NaN is not usedas a number in the rest of the program, as thiswould cause knock-on problems.Trying parseFloat on 1.2.3 gives 1.2. It lookslike parseFloat reads as much of a number as itcan make sense of, then returns that as the valueand ignores the rest. Indeed, parseFloat gives 0as the value for 0m (where ‘m’ is some non-digit,perhaps a letter) but gives NaN for m0.In other words, JavaScript programs can tryto convert a sequence of user actions into anumber using any of the various built-inmechanisms provided by the designers of thelanguage, and depending on how they do it, theywill get different results.Sometimes JavaScript recognizes errors(though it never blocks them), sometimes itignores the error and generates some sort ofnumber. Generally, JavaScript assumes errorscan propagate, meaning that some other part ofthe program has to block NaN and do somethingsensible, rather than treating NaN as somenumber (maybe zero) or worse.If the programmer chooses to handle numbersexplicitly themselves, different things mayhappen again, especially considering the buildingblocks of the language (including parseFloat) areunreliable.The keys of the computer or the keys of adevice are the triggers that make it do things. Wehave shown that pressing some keys can triggerthe computer to work incorrectly. Conceptuallytrivial, but a safety lock would only have toblock invalid number entry, and like a gun’ssafety lock, need not interfere at all with normal— correct — number entry.Many more examples of the problem aregiven in [9], and some details of safety locks andtheir effectiveness are also presented (though the31

present paper introduces and motivates the term‘safety lock’ itself).4. Complex inter-twiningOur observations make clear that parsing anumber, which we’ve discussed thoroughlyabove, displaying a number as the user enters itand editing it are very different activities, andtheir programming has to be very carefullyintegrated.Most likely many errors happen because theoverall properties of number entry are notconsistent: a DELETE key may delete a key butnot have a predictable effect on the parsednumber. We shall see various examples below ofthis and other related problems.5. Does it happen? Does it matter?There have been no reports of mass incorrectnumber entry on the web. People seem to paytheir bills correctly. So it seems like poorprogramming is not a serious problem.Suppose you pay your bills online andaccidentally enter an incorrect number, and youeither over-pay or under-pay your bills. Youprobably say “oops,” and sort out the problem. Itseems you made a mistake, so you fix theproblem.Who would think of complaining to thedesigners of the programs they were using, oreven to the designers of the programminglanguage?In complete contrast, the 1994 Intel processorfloating point problem was a cause célèbrebecause an unusual error, that Intel initially triedto dismiss as very unlikely, was easilyreproduced by everybody who cared to try it [5].Even though virtually nobody needed to do theparticular sum, once people knew what the sumwas, they could reproduce it and see that theresult was incorrect. This undermined thecredibility of the processor, and Intel had toretract and replace affected processors at cost.In contrast, nobody has complained aboutnumber errors, and nobody has said the Intelequivalentof “try doing 4195835÷3145727 withthese particular numbers as it will go wrong.”[5][6] Perhaps if somebody said “try entering1•2•3 and see what happens” would lead toimprovements in the quality of programming,and force developers to introduce safety locks?Whenever you use an interactive system thataccepts numbers, try entering 1•2•3 and see whatit does. Few systems, from Excel toMathematica, from infusion pumps to websearch engines, handle it correctly (see [9]).Evidently, systems do not process numberentry errors dependably. Also, users seem not tocomplain and not to notice. The next question,then, is: does it matter?6. Errors that happen and that matterSyed et al [7] report an unfortunate medicalincident where a nurse entered the number 0.5instead of 5, an error that led to respiratory arrest(i.e., potential death) of a patient. In the paper itis clearly assumed that the design of the deviceused was correct: the device behaved asdesigned, and its logs showed the nurse hadentered 0.5. Problem solved; nurse error.Nurse A apparently entered a morphineconcentration of 0.5 mg per mL instead of 5 mgper mL into the infusion pump; this is what thedevice has logged. With the device initializedwith a concentration that is ten times too low, itwould naturally pump a volume ten times toomuch of morphine into the patient.The patient was finally dosed with 153 mg ofmorphine. The then empty supply of morphinecaused the device to alarm, which led to anothernurse attending and detecting and correcting theerror. The full details of the incident are veryinteresting and are beyond the scope of thepresent paper — for example, nurse A also madea separate error, which had the consequence ofdelaying the impact of the morphine overdose.Nevertheless, the patient arrested and the log ofthe device showed a number entry error.The paper reports that nurse A was uncertainhow to set up the infusion pump, and asked forthe assistance of a second nurse, nurse B. Thissuggests that the human factors engineering ofthe device was substandard or that the training ofthe nurse was inadequate relative to the workthey were supposed to do. While the paper doesnot make clear whether this was supposedly aroutine job for the nurse, it does make clear thatpoor training for nurses was a contributory factorto the error. One might more correctly rephrasethis as poor training relative to the complexity ofthe device was inadequate. Presumably, prior tothe incident the complexity of the device and/orthe inadequacy of the training to enable nurses touse the device effectively was not recognized bymanagement.It is presumably beyond the scope of the Syedet al paper to suggest it: but the manufacturers32

simplifying and fixing their designs so they areeasier and more reliable to use would be morestrategic than retraining all the nurses that use thesystems. Or hospital procurement should avoidpurchasing systems that allow this sort of error.If procurement checked whether systems hadsafety locks, then eventually manufacturerswould provide better systems. In the meantime,we note that it is surprising that manufacturerscontinue to design systems with trivial faults; theexpertise to make them properly is readilyavailable if they want to make use of it.The user apparently made a number entryerror. The infusion pump involved, an AbbottLifecare PCS Plus II Infuser type 4100, does nothave number keys, but has “increase” and“decrease” buttons. Nor does it have a decimalpoint. This reduces the number of keys (numberscan be entered with 2 keys rather than 11) andperhaps makes the device look simpler, but itinevitably creates modes — for example, whichdigit (units, tens, etc) is being increased ordecreased when pressing up or down buttons? Oris a value increased and decreased, and any digitin its decimal representation might change (e.g.,going from 99 to 100 in one button press changes3 digits simultaneously)?Does the rate of increase change the longerthe relevant button is held down (typically, thefirst press increments by 0.1 and if held downlonger increments by 1 then by 10)?Every mode a user interface has increases thechance that the user mistakes the mode thedevice is in, and hence makes errors.The paper indicates that the infusion pumpwas set to a default setting of 0.5, and it suggests(but does not make sufficiently clear) that nurseA selected 0.5 rather than entered it explicitly. Itappears the device was set up so that 0.5 is theinitial value, and the user would then increase ordecrease it to the desired target value.If so, there are various ways to explain theerror, including the following:• A single key press error could havecaused the error; the keying error maynot have been noticed by the nurse. Wenote that the device does not beep whenkeys are pressed, thus making it hard tokeep track of which buttons work.• Misreading the display as 5 would meanthe nurse would accept it with no furtheraction required.Another problem (described in the paper) isthe device has an out-of-range check, whichshould have helped protect the patient. If a doseis to take longer than 4 hours, this is blocked.Unfortunately, entering the wrongconcentration need not make the infusion of thedrug take longer. It appears the device does notcheck concentrations (or the calculated drugdelivery rates that are functions of theconcentrations).7. Other scenariosWe have discussed in some detail ways inwhich the nurse error may have been induced bydevice design. We know that other number errorscause incidents in healthcare and that theinfusion pump involved in the story above is notthe only style of pump. In general, then, there areother potential sources of number error.We now briefly consider other ways thenumber entry error could happen (on a differentdevice) using numeric keys rather thanincrease/decrease keys.(1) We’ve already considered a simplepossibility above. Assume that a • has alreadybeen pressed (perhaps some time ago, or byaccident), but the display will show 0.0, which isalso what it would display if • has not beenpressed. (The standard initial display of 0.0 doesnot tell a user whether • has already beenpressed.) If nurse A now presses 5, the valueentered is 0.5 even though the nurse expects it tobe 5.0 — based on the fact that on manyprevious occasions, pressing 5 when the deviceshows 0.0 has got it to 5.0.(2) Laura Gosby has pointed out that manydevices have a fixed position for the decimalpoint, but digits move right-to-left as they arekeyed. So many devices display 0.0 as theirinitial display. As a user presses 123 in thatorder, the display would change successivelythrough 0.1, 1.2, 12.3. (Cash machines/ATMsoften work like this.) In this scenario, nurse Asimply presses 5, thinks they have pressed 5, butthe device would treat it as 0.5, the wrong valueand out by a factor of ten.(3) Perhaps nurse A keys 0••. This is a simpleslip: 0••5 is not the start of a well-formednumber. Nurse A recognizes this error, andpresses the DELETE or CANCEL key to correctthe erroneous keystroke. On any sensible device(like Microsoft Word) pressing •• DELETE hasthe meaning same as • alone.33

Written around 1790BC, the Code of Hammurabisays things like, “If a builder builds a house forsomeone, and does not build it properly, and thehouse falls down and kills its owner, then thebuilder shall be put to death.” Likewise, today’sprogrammers should be plugged into theirdevices: not only is it a deterrent for badprogramming, but if the programmer is killed bytheir own bad programming, they won’t have anychildren. Eventually evolution will take care ofimproving programming standards…Figure 2. Updating the Code of Hammurabi?However, most medical devices do not keeptrack of how many decimal points a nurse haskeyed: a number either has no decimal points orone. So 0•• is recorded by a device as 0• withonly a single decimal point. But then nurse Apresses DELETE to correct the error. The resultwill be as if the nurse pressed 0, not 0•, becauseeffectively both dots keyed by the user havegone. Next, nurse A presses 5, and the device hasgot 05 as the number entered, presumably equalto the numeric value 5, yet nurse A believes theyhave entered 0.5.This is a tenfold error (but, as it happens, inthe wrong direction for the specific Syed et alscenario). Nevertheless it illustrates how a user’sreasonable expectations (learned from otherfamiliar systems like word processors) may beseriously undermined by poor programming.(4) Another possibility is that the deviceinitially displays 0.0, and as the user keys indigits and dots, the display updates. The nursekeys 5, and the display shows 0.5; if the nursecontinued and keyed 0, the display would updateto 5.0, as the 5 “scrolls” to the left. Who knows?But this is a plausible way that nurse A thinksthey have keyed 5, but the device logs 0.5.(5) The nurse keyed • by mistake as their firstkey press, and noticed this slip. The nurse hitDELETE but the delete did not work, perhapsbecause the nurse did not press the key hardenough. Keying the digit 5 next would have ledto the device treating the number as 0.5, not as 5.(6) Another, sadly common, possibility is thatthe device has timeouts. The nurse presses 0• andthen is maybe distracted for a few seconds(perhaps to say something or attend to thepatient) then continues by pressing 5. The nurseknows they pressed 0•5, but the device timed-outafter the • and ignored it. The nurse has, so far asthe device is concerned, entered 5.0.(7) On devices with up/down keys, theup/down keys may change the value displayedby 0.1. The user pressed UP and the numberbeing entered cycles via 0.9 to 0.0 (or 0.1),because some other operation is required for UPto increment units and tens digits. On seeing thenumber reach 5, the nurse may have thought “Ihave increased the number, and it has changed to5, therefore I have finished.” Unfortunately, thisis also a misreading error: the number is 0.5,even though the nurse saw it increase from 0.5 to0.6, 0.7… etc. This error would be even morelikely if the user interface accelerates the rate ofchange the longer UP is pressed.(8) We know that the infusion pump involveduses 7 segment digit displays. A display of 0.5(which was selected) rather than 5 (which shouldhave been selected) may easily have beenmisread, as decimal points in 7 segment displaysare not very salient. Possibly the decimal pointwas faulty; possibly the machine was positionedso the dot could not be seen because of parallax.Possibly 0.5 is displayed as 05, with a slightlysmaller decimal 5 but no decimal point at all.(9) The hypothetical scenarios above werelimited to the specific 0.5 or 5.0 confusion,though they obviously generalize to other values,such as 0.1 and 1.0 confusions. The literatureexplores many other forms of numericalconfusion — for example, Johnson et al [3] showhow keying 135•0 can be taken as 1,350 becausethe device gratuitously ignores decimal points onvalues larger than 100. Cairns and Thimbleby [8]show how keying 10•5 gets 10.5 but 100•5 gets1,005 on the Baxter Colleague. In these twoexamples, the device’s handling of • depends oncontingent modes that may not be, and probablyare not, obvious to the user.(10) All the hypothetical examples aboveassume the user made a slip in the operation of thedevice. The solution to these sorts of problem arebetter device design and/or better user training.(We prefer better device design, since users willeventually make slips regardless of their training.)A different possibility is that the user intended toenter 0.5, but by mistake. Perhaps the prescriptionwas misread? Perhaps there was a smudge beforethe 5 that was misread as a •? Perhaps if the dosehad been written as 5•0 it would have been lesslikely to have been misread as 0•5? Most probably,if the training of everybody was that . is neverwritten when • is intended, the (hypothetical)misreading would me far less likely. 33 A user may intend to make a mistake, for example knowingperfectly well what the correct dose is; this is then a violation.34

Sadly, these ten hypothetical scenarios (inaddition to the ones possible on the actual deviceinvolved in the incident, which we describedearlier) are all plausible: user interfaces are oftenbadly designed, and it is all too easy to imaginethe errors not being blocked by safety locks.In summary, there are many hypotheticalways that a device can log an apparent “usererror” which has been induced by design, not byunreasonable behavior on the part of the user.Logs should at least record the exact keystrokesthe user performed and their timings, not thefinal value the device somehow works out,perhaps erroneously, from the keystrokes.• Device logs should allow investigators todistinguish between different possiblecauses of error.One can infer in the Syed et al case that thedevice log was not sufficiently clear to suggestalternatives other than “obvious nurse error” tothe authors of the paper.8. DiscussionThe list of possible causes of error is notintended to be exhaustive, nor are all possibilitiessuggested equally likely on the particular deviceat the centre of the reported case. The point is toraise a variety of ways in which the absence ofsafety locks can cause problems. Almostcertainly one or more safety locks on the devicein question failed or, more likely, were neverdesigned into the device.Sadly, without more detailed information (ofthe device condition and set up, including defaultvalues, and of the incident itself) one can onlyspeculate as to the specifics.The Syed et al paper mentions documentedcases of concentration errors made before on thesame device. Evidently, concentration errors arewell known and encountered relatively often.One wonders whether there is a systematicreason for this class of error other thanindependent human error. The unvaryingcommon factor in all the errors is the device (andits design).The paper says,“While the primary error involved incorrectprogramming 4 of the PCA 5 pump.”We would beg to disagree. The primary error,in our opinion, lies in the design of the device.Moreover, fixing the design would solve many ofthe paper’s recommendations, such as improvedtraining. Why have better training for somethingthat is over-complex, as it would be morestrategic to improve the device — andreprogramming the device (e.g., in a firmwareupdate) could be done without any training costs.The Syed et al paper makes sixrecommendations, including better nurse training(they do not mention that perhaps prescriptionsshould be written out more carefully, whichwould be an issue of pharmacy or consultanttraining).None of their recommendations cover betterprocurement of devices (why are bad devicespurchased?) nor better design of devices (whyare bad devices designed in the first place?) Noris there any explicit learning from theinvestigation that might lead to better devicedesign.Although the paper was written from amedical perspective, from our point of view, likemany if not all such papers it is too vague aboutthe design and use of the device. The assumptionis that the nurse made the error, not that thedevice induced or contributed to the error.9. ConclusionsSyed et al is only one paper, but its concernsand approach are not unusual [1]. There was amedical mishap, and the paper makes it clear thatthis error was not a one-off event for this device.The healthcare implications are discussed.Maybe there will be some management changes.Design was not criticized in detail, excepttangentially in mentioning a human factors paperthat shows that human factors can reduce error[4]. The paper leaves unexplored whether humanfactors redesign would have avoided the problemdiscussed.In summary:• Many devices have no safety locks innumber entry;• Number entry errors may causesignificant problems;4 Programming: i.e., the sequence of keystrokesperformed by the nurse, not programming as in thesoftware programming of the device.5 PCA: patient controlled analgesia; i.e., an infusionpump with features for the patient to have somecontrol over their drug dose.35

• Analyses do not discuss details of use insufficient detail to be certain of relevantdesign factors;• Although human factors can improveperformance and safety, no connection ismade between particular incidents andgeneral solutions.Although the present paper has concentratedon number entry (because of its familiarity,clarity and ubiquity), there are problems with allforms of user interface. Number entry is, for thesake of exposition, merely easier to demonstrateas flawed.Machine guns are dangerous, but they aremade considerably safer without compromisingtheir effectiveness by having safety locks. Nowimagine this: there is a machine gun available onthe international market without a safety lock. Ithink we would all know about it: films would bemade of exploits with it!Now imagine that somebody has been shot bya machine gun, but nobody mentions that themachine gun involved has no safety lock. If agun has no safety lock, it would be likely to bementioned in any accident investigation. Yetnobody mentions it. For some reason, people arenot thinking about it.Bizarre with guns, maybe, but the storyimplies nobody is familiar with safety lockswhen it comes to programming and complexdevices like hospital infusion pumps. Nobodynotices safety lock absence. This paper hasargued, as is more obvious with small arms, thatsafety locks in programming could lead to livesbeing saved. Elsewhere [9], we’ve shown theycould halve mortality.The need for safety locks on machine gunscan be easily and dramatically demonstrated.Imagine there is no safety lock: press the trigger,and the gun sprays bullets …and the gun’s recoilcreates chaos…Without a safety lock, a gun is clearly lethal.Guns have the advantage, both for the rhetoric ofthis paper and in reality, that there is a very smallpsychological distance between pressing thetrigger and a dangerous bullet emerging noisily!With any other device, try entering 1•2•3 (ormake some other keying error) and see whathappens. Sadly, patients with an avoidableoverdose of morphine die quietly without anynoisy drama.10. AcknowledgementsThis work was funded by EPSRC grant nos.EP/G003971/1 and EP/G059063/1. The author isgrateful for discussions with Chitra Acharya,Paul Cairns, and Laura Gosby.11. References[1] Blandford, A, Buchanan, G, Furniss, D,Curzon, P, Thimbleby, H. Few are Looking:Invisible Problems with Interactive MedicalDevices, Workshop on Interactive Systemsin Healthcare (WISH) in press, ProceedingsACM CHI 2010, Atlanta.[2] Cormen, TH, Leiserson, CE, Rivest, RL,Stein, C. Introduction to Algorithms. 3rdedition. MIT Press, 2010.[3] Johnson, TR, Tang, X, Graham, MJ, Brixey,J, Turley, JP, Zhang, J, Keselman, A, Patel,VL. Attitudes toward Medical Device UseErrors and the Prevention of AdverseEvents, The Joint Commission Journal onQuality and Patient Safety, 33(11):689-694,2007.[4] Lin, L, Vicente KJ, Doyle DJ. PatientSafety, Potential Adverse Drug Events, andMedical Device Design: A Human FactorsEngineering Approach. Journal BiomedInform 2001, 34: 274-284.[5] Moler, CB. A Tale of Two Numbers,MATLAB News and Notes 1995, 10–12.[6] Soderquist, P, Leeser, M. Area andPerformance Tradeoffs in Floating PointDivide and Square-Root Implementations,ACM Computing Surveys 1996, 28(3):519-564.[7] Syed, S, Paul, JE, Hueftlein, M, Kampf, M,McLean, RF. Morphine Overdose fromError Propagation on an Acute Pain Service.Canadian Journal of Anesthesia 2006,53(6):586-590.[8] Thimbleby, H. User Interface Design.Addison-Wesley, 1990.[9] Thimbleby, H, Cairns P. Reducing NumberEntry Errors: Solving a Widespread, SeriousProblem. Journal Royal Society Interface, inpress, 2010.36