A Practical Introduction to Data Structures and Algorithm Analysis

8File Processing and ExternalSortingEarlier chapters presented basic data structures and algorithms that operate on datastored in main memory. Some applications require that large amounts of informationbe stored and processed — so much information that it cannot all fit into mainmemory. In that case, the information must reside on disk and be brought into mainmemory selectively for processing.You probably already realize that main memory access is much faster than accessto data stored on disk or other storage devices. The relative difference in accesstimes is so great that efficient disk-based programs require a different approach toalgorithm design than most programmers are used to. As a result, many programmersdo a poor job when it comes to file processing applications.This chapter presents the fundamental issues relating to the design of algorithmsand data structures for disk-based applications. 1 We begin with a descriptionof the significant differences between primary memory and secondary storage.Section 8.2 discusses the physical aspects of disk drives. Section 8.3 presents basicmethods for managing buffer pools. Section 8.4 discusses the Java model forrandom access to data stored on disk. Section 8.5 discusses the basic principles forsorting collections of records too large to fit in main memory.1 Computer technology changes rapidly. I provide examples of disk drive specifications and otherhardware performance numbers that are reasonably up to date as of the time when the book waswritten. When you read it, the numbers might seem out of date. However, the basic principles do notchange. The approximate ratios for time, space, and cost between memory and disk have remainedsurprisingly steady for over 20 years.279