Encyclopedia of Computer Science and Technology

324 multitasking“local” memory within the cluster, maintaining coherencethrough bus monitoring is impracticable. Instead, memoryis usually organized into data objects that are distributedoptimally to reduce the necessity for remote access, andthe objects are shared by CPUs requesting them through adirectory system.MultiprogrammingIn order for a program to take advantage of the ability to runon multiple CPUs, the operating system must have facilitiesto support multiprocessing, and the program must be structuredso its various tasks are most efficiently distributedamong the CPUs. These separate tasks are generally calledthreads. A single program can have many threads, eachexecuting separately, perhaps on a different CPU, althoughthat is not required.The operating system can use a number of approachesto scheduling the execution of processes or threads. It cansimply assign the next idle (available) CPU to the thread.It can also give some threads higher priority for access toCPUs, or let a thread continue to “own” its CPU until it hasbeen idle for some specified time.The use of threads is particularly natural for applicationswhere a number of activities must be carried onsimultaneously. For example, a scientific or process controlapplication may have a separate thread reading the databeing returned from each instrument, another thread monitoringfor alarm conditions, and other threads generatinggraphical output.Threads also allow the user to continue interacting witha program while the program is busy carrying out earlierrequests. For example, the user of a Web browser can continueto use menus or navigation buttons while the browseris still loading graphics needed for the currently displayedWeb page. A search program can also launch separatethreads to send requests to multiple search engines or toload multiple pages.Support for multiprogramming and threads can now befound in versions of most popular programming languages,and some languages such as Java are explicitly designed toaccommodate it.Multiprogramming often uses groups or clusters of separatemachines linked by a network. Running software onsuch systems involves the use of communications protocolssuch as the MPI (message-passing interface). This programminginterface has been widely deployed on many platformsfor use with languages such as C/C++ and Fortran.Another popular programming interface is OpenMP, whichfeatures the allocation of execution threads and the distributionof work among them.A Multiprocessed WorldThe demand for software that can efficiently use multipleprocessors is likely for some time to outstrip the supply ofprogrammers who can “think in parallel.” One reason isthat today most new PCs have two processing “cores,” withfour-core systems available and more to come (see microprocessor).This means that many mainstream applicationswill eventually need to be rewritten for the newhardware environment. Another factor is that as the priceper processor continues to decline and high-end multiprocessingmachines are reaching 1 “petaflop” (1 quadrillionoperations per second), many supercomputer applicationswill also need to be rewritten.Meeting this demand not only takes training, it alsotakes appropriate languages and other tools. In recentyears, therefore, the Defense Advanced Research ProjectsAgency (DARPA) has funded research in “High ProductivityComputer Systems” by such companies as Cray, Sun,and IBM. New languages built “from the ground up” formultiprocessing include Sun’s Fortress, Cray’s Chapel, andIBM’s new project, code-named X10. Ultimately, systemsfor developing multiprocessing software should take mostof the architectural details off the hands of the programmer,allowing performance to smoothly “scale up” with theincreasing number of processors.Further ReadingAnthes, Gary. “Languages for Supercomputing Get ‘Suped’ Up.”Computerworld, March 12, 2007. Available online. URL:http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=283477. Accessed April 5, 2007.Culler, David E., and Jaswinder Pal Singh. Parallel Computer Architecture:A Hardware/Software Approach. San Francisco: MorganKaufmann, 1999.Dongara, Jack, et al., eds. Sourcebook of Parallel Computing. SanFrancisco: Morgan Kaufmann, 2003.Feldman, Michael. “Our Manycore Future.” HPC Wire. Availableonline. URL: http://www.hpcwire.com/hpc/1295541.html.Accessed April 5, 2007.Merritt, Rick. “Where Are the Programmers? Enrollment WanesJust as Computer Scientists Grapple with Problem of Parallelism.”EE Times, March 12, 2007. Available online. URL:http://www.eetimes.com/showArticle.jhtml?articleID=197801653. Accessed April 5, 2007.Quinn, Michael J. Parallel Programming in C with MPI and OpenMP.New York: McGraw-Hill, 2003.multitaskingUsers of modern operating systems such as Microsoft Windowsare familiar with multitasking, or running severalprograms at the same time. For example, a user might bewriting a document in a word processor, pause to check thee-mail program for incoming messages, type a page addressinto a Web browser, then return to writing. Meanwhile, theoperating system may be running a number of other programstucked unobtrusively into the background, such as avirus checker, task scheduler, or system resource monitor.Each running program “takes turns” using the PC’s centralprocessor. In early versions of Windows, multitaskingwas cooperative, with each program expected to periodicallyyield the processor to Windows so it could be assigned to thenext program in the queue. One weakness of this approachis that if a program crashes, the CPU might be “locked up”and the system would have to be rebooted. However, WindowsNT, 2000, and XP (as well as operating systems suchas UNIX) use preemptive multitasking. The operating systemassigns a “slice” of processing (CPU) time to a programand then switches it to the next program regardless of what