Encyclopedia of Computer Science and Technology

concurrent programming 113Concurrent programming is the organization of programsso that two or more tasks can be executed at thesame time. Each task is called a thread. Each thread is itselfa traditional sequentially ordered program. One advantageof concurrent programming is that the processor can beused more efficiently. For example, instead of waiting forthe user to enter some data, then performing calculations,then waiting for more data, a concurrent program can havea data-gathering thread and a data-processing thread. Thedata-processing thread can work on previously gathereddata while the data-gathering thread waits for the user toenter more data. The same principle is used in multitaskingoperating systems such as UNIX or Microsoft Windows.If the system has only a single processor, the programsare allocated “slices” of processor time according to somescheme of priorities. The result is that while the processorcan be executing only one task (program) at a time, forpractical purposes it appears that all the programs are runningsimultaneously (see multitasking).Multiprocessing involves the use of more than one processoror processor “core.” In such a system each task (or eveneach thread within a task) might be assigned its own processor.Multiprocessing is particularly useful for programs thatinvolve intensive calculations, such as image processing orpattern recognition systems (see multiprocessing).Programming IssuesRegular programs written for operating systems such asMicrosoft Windows generally require no special codeto deal with the multitasking environment, because theoperating system itself will handle the scheduling. (Thisis true with preemptive multitasking, which has generallysupplanted an earlier scheme where programs were responsiblefor yielding control so the operating system could giveanother program a turn.)Managing threads within a program, however, requiresthe use of programming languages that have special statements.Depending on the language, a thread might bestarted by a fork statement, or it might be coded in a waysimilar to a traditional subroutine or procedure. (The differenceis that the main program continues to run while theprocedure runs, rather than waiting for the procedure toreturn with the results of its processing.)The coordination of threads is a key issue in concurrentprogramming. Most problems arise when two or morethreads must use the same resource, such as a processorregister (at the machine language level) or the contentsof the same variable. Let’s say two threads, A and B, havestatements such as: Counter = Counter + 1. Thread A getsthe value of Counter (let’s say it’s 10) and adds one to it.Meanwhile, thread B has also fetched the value 10 fromCounter. Thread A now stores 11 back in counter. ThreadB, now adds 1 and stores 11 back in Counter. The result isthat Counter, which should be 12 after both threads haveprocessed it, contains only 11. A situation where the resultdepends on which thread gets to execute first is called arace condition.One way to prevent race conditions is to specify thatcode that deals with shared resources have the ability to“lock” the resource until it is finished. If thread A can lockthe value of Counter, thread B cannot begin to work with ituntil thread A is finished and releases it. In hardware terms,this can be done on a single-processor system by disablinginterrupts, which prevents any other thread from gainingaccess to the processor. In multiprocessor systems, an interlockmechanism allows one thread to lock a memory locationso that it can’t be accessed by any other thread. Thiscoordination can be achieved in software through the useof a semaphore, a variable that can be used by two threadsto signal when it is safe for the other to resume processing.In this scheme, of course, it is important that a threadnot “forget” to release the semaphore, or execution of theblocked thread will halt indefinitely.A more sophisticated method involves the use of messagepassing, where processes or threads can send a varietyof messages to one another. A message can be used to passdata (when the two threads don’t have access to a sharedmemory location). It can also be used to relinquish accessto a resource that can only be used by one process at a time.Message-passing can be used to coordinate programs orthreads running on a distributed system where differentthreads may not only be using different processors, but runningon separate machines (a cluster computing facility).Programming language support for concurrent programmingoriginally came through devising new dialectsof existing languages (such as Concurrent Pascal), buildingfacilities into new languages (such as Modula-2), or creatingprogram libraries for languages such as C and C++.However, in recent years concurrent programming languagesand techniques have been unable to keep up withthe growth in multiprocessor computers and distributedcomputing (such as “clusters” of coordinated machines).With most new desktop PCs having two or more processingcores, there is a pressing need to develop new programsthat can carry out tasks (such as image processing) usingmultiple streams of execution. Meanwhile, in very highperformancemachines (see supercomputer), the DefenseAdvanced Research Projects Agency (DARPA) has been tryingto work with manufacturers to develop languages towork better with computers that may have hundreds ofprocessors as well as distributed systems or clusters. Suchlanguages include Sun’s Fortress, intended as a modernreplacement for Fortran for scientific applications.The new generation of concurrent languages tries toautomate much of the allocation of processing, allowingprogrammers to focus on their algorithms rather thanimplementation issues. For example, program structuressuch as loops can be automatically “parallelized,” such asby assigning them to separate cores.Further ReadingAnthes, Gary. “Languages for Supercomputing Get ‘Suped’ Up.”Computerworld.com, March 12, 2007. Available online. URL:http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=283477&intsrc=hm_list.Accessed June 24, 2007.Ben-Ari, M. Principles of Concurrent & Distributed Programming.2nd ed. Reading, Mass.: Addison-Wesley, 2006.