Algorithms and Data Structures for External Memory

More documents

Recommendations

Info

13 Dynamic and Kinetic Data Structures In this chapter, we consider two scenarios where data items change: dynamic (in which items are inserted and deleted) and kinetic (in which the data items move continuously along specified trajectories). In both cases, queries can be done at any time. It is often useful for kinetic data structures to allow insertions and deletions; for example, if the trajectory of an item changes, we must delete the old trajectory and insert the new one. 13.1 Dynamic Methods for Decomposable Search Problems In Chapters 10–12, we have already encountered several dynamic data structures for the problems of dictionary lookup and range search. In Chapter 12, we saw how to develop optimal EM range search data structures by externalizing some known internal memory data structures. The key idea was to use the bootstrapping paradigm, together with weight-balanced B-trees as the underlying data structure, in order to consolidate several static data structures for small instances of range searching into one dynamic data structure for the full problem. The bootstrapping technique is specific to the particular data structures 119
120 Dynamic and Kinetic Data Structures involved. In this section, we look at a technique that is based upon the properties of the problem itself rather than upon that of the data structure. We call a problem decomposable if we can answer a query by querying individual subsets of the problem data and then computing the final result from the solutions to each subset. Dictionary search and range searching are obvious examples of decomposable problems. Bentley developed the logarithmic method [83, 278] to convert efficient static data structures for decomposable problems into general dynamic ones. In the internal memory setting, the logarithmic method consists of maintaining a series of static substructures, at most one each of sizes 1, 2, 4, 8, . . . . When a new item is inserted, it is initialized in a substructure of size 1. If a substructure of size 1 already exists, the two substructures are combined into a single substructure of size 2. If there is already a substructure of size 2, they in turn are combined into a single substructure of size 4, and so on. For the current value of N, itis easy to see that the kth substructure (i.e., of size 2 k ) is present exactly when the kth bit in the binary representation of N is 1. Since there are at most log N substructures, the search time bound is logN times the search time per substructure. As the number of items increases from 1 to N, the kth structure is built a total of N/2 k times (assuming N is a power of 2). If it can be built in O(2 k ) time, the total time for all insertions and all substructures is thus O(N logN), making the amortized insertion time O(logN). If we use up to three substructures of size 2 k at a time, we can do the reconstructions in advance and convert the amortized update bounds to worst-case [278]. In the EM setting, in order to eliminate the dependence upon the binary logarithm in the I/O bounds, the number of substructures must be reduced from logN to log B N, and thus the maximum size of the kth substructure must be increased from 2 k to B k . As the number of items increases from 1 to N, the kth substructure has to be built NB/B k times (when N isapowerofB), each time taking O � B k (log B N)/B � I/Os. The key point is that the extra factor of B in the numerator of the first term is cancelled by the factor of B in the denominator of the second term, and thus the resulting total insertion time over all N insertions and all log B N structures is O � N(log B N) 2� I/Os, which is
Page 1 and 2:
TCSv2n4.qxd 4/24/2008 11:56 AM Page
Page 4 and 5:
Algorithms and Data Structures for
Page 6 and 7:
Foundations and Trends R○ in Theo
Page 8 and 9:
Foundations and Trends R○ in Theo
Page 10 and 11:
Preface I first became fascinated a
Page 12:
Preface xi I would like to thank my
Page 15 and 16:
xiv Contents 6 Lower Bounds on I/O
Page 18 and 19:
1 Introduction The world is drownin
Page 20 and 21:
However, not all memory references
Page 22 and 23:
1.1 Overview 5 Our general goal is
Page 24:
Table 1.1 Paradigms for I/O efficie
Page 27 and 28:
10 Parallel Disk Model (PDM) spindl
Page 29 and 30:
12 Parallel Disk Model (PDM) D = nu
Page 31 and 32:
14 Parallel Disk Model (PDM) The pr
Page 33 and 34:
16 Parallel Disk Model (PDM) 2.3 Re
Page 35 and 36:
18 Parallel Disk Model (PDM) archit
Page 38 and 39:
3 Fundamental I/O Operations and Bo
Page 40:
As Table 3.1 indicates, the multipl
Page 43 and 44:
26 Exploiting Locality and Load Bal
Page 45 and 46:
28 Exploiting Locality and Load Bal
Page 47 and 48:
30 External Sorting and Related Pro
Page 49 and 50:
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
Page 59 and 60:
Page 61 and 62:
Page 63 and 64:
Page 65 and 66:
Page 67 and 68:
Page 69 and 70:
Page 71 and 72:
Page 74 and 75:
6 Lower Bounds on I/O In this chapt
Page 76 and 77:
6.1 Permuting 59 in backwards order
Page 78 and 79:
6.2 Lower Bounds for Sorting and Ot
Page 80:
6.2 Lower Bounds for Sorting and Ot
Page 83 and 84:
66 Matrix and Grid Computations the
Page 86 and 87: 8 Batched Problems in Computational
Page 88 and 89: 8.1 Distribution Sweep 71 Goodrich
Page 90 and 91: 8.1 Distribution Sweep 73 the curre
Page 92 and 93: Time (seconds) Time (seconds) 7000
Page 94 and 95: 9 Batched Problems on Graphs The pr
Page 96 and 97: Table 9.2 Best known I/O bounds for
Page 98 and 99: 9.2 Special Cases 81 that contains
Page 100 and 101: 10 External Hashing for Online Dict
Page 102 and 103: 2 000 000 001 010 011 100 101 110 1
Page 104 and 105: 10.2 Directoryless Methods 87 a tot
Page 106 and 107: 11 Multiway Tree Data Structures In
Page 108 and 109: 11.1 B-trees and Variants 91 “sha
Page 110 and 111: 11.3 Parent Pointers and Level-Bala
Page 112 and 113: 11.4 Buffer Trees 95 I/Os by follow
Page 114: 11.4 Buffer Trees 97 between update
Page 117 and 118: 100 Spatial Data Structures and Ran
Page 138 and 139: 13.2 Continuously Moving Items 121
Page 140 and 141: 14 String Processing In this chapte
Page 142 and 143: 5 a b a 3 c a 0 b a 4 b 6 6 a b a b
Page 144 and 145: 14.3 Suffix Trees and Suffix Arrays
Page 146 and 147: 15 Compressed Data Structures The f
Page 148 and 149: 15.1 Data Representations and Compr
Page 150 and 151: 15.2 External Memory Compressed Dat
Page 152 and 153: 15.2 External Memory Compressed Dat
Page 154: 15.2 External Memory Compressed Dat
Page 157 and 158: 140 Dynamic Memory Allocation For s
Page 159 and 160: 142 External Memory Programming Env
Page 161 and 162: 144 External Memory Programming Env
Page 163 and 164: 146 Conclusions intriguing connecti
Page 165 and 166: 148 Notations and Acronyms lowercas
Page 167 and 168: 150 Notations and Acronyms � �
Page 169 and 170: 152 References Proceedings of the A
Page 171 and 172: 154 References [41] L. Arge, K. H.
Page 173 and 174: 156 References Computer Science, pp
Page 175 and 176: 158 References ACM Conference on In
Page 177 and 178: 160 References [132] F. K. H. A. De
Page 179 and 180: 162 References [165] R. W. Floyd,
Page 181 and 182: 164 References [195] M. R. Henzinge
Page 183 and 184: 166 References [228] K. Küspert,
Page 185 and 186:
168 References [261] D. R. Morrison
Page 187 and 188:
170 References [293] J. T. Robinson
Page 189 and 190:
172 References [325] “TerraServer
Page 191:
174 References [356] O. Wolfson, P.
show all

Algorithms and Data Structures for External Memory

Create successful ePaper yourself

Delete template?

Save as template?