Encyclopedia of Computer Science and Technology

408 risks of computingHafner, Katie. The Well: A Story of Love, Death & Real Life in theSeminal Online Community. New York: Carol & Graf, 2001.Howard Rheingold [home page]. Available online. URL: http://www.rheingold.com. Accessed November 12, 2007.Kimball, Lisa, and Howard Rheingold. “How Online Social NetworksBenefit Organizations.” Available online. URL: http://www.rheingold.com /Associates/onlinenetworks.html.Accessed November 12, 2007.Rheingold, Howard. Smart Mobs: The Next Social Revolution. NewYork: Basic Books, 2003.———. Tools for Thought: The History and Future of Mind-ExpandingTechnology. 2nd rev. ed. Cambridge, Mass.: MIT Press,2000.———. The Virtual Community: Homesteading on the ElectronicFrontier. Revised ed. Cambridge, Mass.: MIT Press, 2000.[The first edition is also available online. URL: http://www.rheingold.com/vc/book/. Accessed November 12, 2007.]Smart Mobs Blog. Available online. URL: http://www.smartmobs.com/. Accessed November 12, 2007.The WELL Available online. URL: http://www.well.com. AccessedNovember 12, 2007.risks of computingProgrammers and managers of software development aregenerally aware of the need for software to properly dealwith erroneous data (see error handling). They knowthat any significant program will have bugs that must berooted out (see bugs and debugging). Good software engineeringpractices and a systematic approach to assuring thereliability and quality of software can minimize problemsin the finished product (see software engineering andquality assurance, software). However, serious bugs arenot always caught, and sometimes the consequences can becatastrophic. For example, in the Therac 25 computerizedX-ray cancer treatment machine, poorly thought-out commandentry routines plus a counter overflow resulted inthree patients being killed by massive X-ray overdoses. Theoverdoses ultimately occurred because the designers hadremoved a physical interlock mechanism they believed wasno longer necessary.Any computer application is part of a much larger environmentof humans and machines, where unforeseen interactionscan cause problems ranging from inconvenience toloss of privacy to potential injury or death. Seeing thesepotential pitfalls requires thinking beyond the specificationsand needs of a particular project. For many years theUsenet newsgroup comp.risks (and its collected form, RisksDigest) have chronicled what amounts to an ongoing symposiumwhere knowledgeable programmers, engineers, andothers have pointed out potential risks in new technologyand suggested ways to minimize them.Unexpected SituationsA common source of risks arises from designers of controlsystems failing to anticipate extreme or unusual environmentalconditions (or interactions between conditions).This is a particular problem for mobile robots, which unliketheir tethered industrial counterparts must share elevators,corridors, and other places with human beings. For example,a hospital robot was not designed to recognize when itwas blocking an elevator door—a situation that could haveblocked a patient being rushed into surgery. A basic principleof coping with unexpected situations is to try to designa fail-safe mode that does not make the situation worse. Forexample, an automatic door should be designed so that if itfails it can be opened manually rather than trapping peoplein a fire or other disaster.Unanticipated InteractionsThe more systems there are that can respond to externalinputs, the greater the risk that a spurious input might triggeran unexpected and dangerous response. For example,the growing number of radio-controlled (wireless) deviceshave great potential for unexpected interactions betweendifferent devices. In one case reported to the Risks Forum, aSwedish policeman’s handheld radio inadvertently activatedhis car’s airbag, which slammed the radio into him. Severalmilitary helicopters have crashed because of radio interference.Banning the use of electronic devices at certain timesand places (for example, aboard an aircraft that is taking offor landing) can help minimize interference with the mostsafety-critical systems.At the same time, regulations themselves introduce therisk that people will engage in other forms of risky behaviorin an attempt to either follow or circumvent the rule. Forexample, the Japanese bullet train system imposed a stiffpenalty for operators who failed to wear a hat. In one casean operator left the train cabin to retrieve his hat while thetrain kept running unsupervised. This minor incident actuallyconceals two additional sorts of risks—that of automatinga system so much that humans no longer pay attention,and the inability of the system to sense the lack of humansupervision.Unanticipated Use of DataThe growing number of different databases that track eventhe intimate details of individual lives has raised many privacyissues (see privacy in the digital age). Designers andmaintainers of such databases had some awareness of thethreat of unauthorized persons breaking into systems andstealing such data (see computer crime and security).However, most people were surprised and alarmed by thenew crime of identity theft, which began to surface in significantnumbers in the mid- to late-1990s (see identity theft).It turned out that while a given database (such as customerrecords, bank information, illicitly obtained DMVrecords, and so on) usually did not have enough informationto allow someone to successfully impersonate another’sidentity, it was not difficult to use several of these sourcestogether to obtain, for example, the information needed toapply for credit in another’s name. In particular, while mostpeople guarded their credit card numbers, they tended notto worry as much about Social Security numbers (SSN).However, since many institutions use the SSN to indextheir records, the number has become a key for unlockingpersonal data.Further, as more organizations put their records onlineand make them Web-accessible, the ability of hackers, privateinvestigators (legitimate or not), and “data brokerage”