Applying OLAP Pre-Aggregation Techniques to ... - Jacobs University

More documents

Recommendations

Info

78 5. Pre-Aggregation Support Beyond Basic Aggregate Operations ter 3, we see that the result returned by a group of affected cells for a given nonstandard aggregate operation such as scaling is not likely to be useful in computing another non-standard aggregate operation such as edge detection. This is because non-standard operations differ significantly with respect to the way their affected cells are determined. Nevertheless, this result may be useful in computing the same type of non-standard operation under certain conditions. For example, the result of scaling by a factor of 8 could be used to compute scaling by a factor of 10 (assuming that both operations utilize the same resampling method). This result, however, is not likely to be useful in edge detection for the same object. We therefore simplify the problem of offering pre-aggregation support to nonstandard aggregations by treating each type of non-standard operation separately. This simplification is similar to those found in data warehousing techniques where preaggregation algorithms cover a specific type of queries. For instance, pre-aggregation algorithms exist for queries that include a group-by clause in their predicates, while other algorithms are used for queries without join conditions. We now focus on pre-aggregation support for one non-standard aggregate operation, scaling, for the following reasons: • One of the most frequent operations in GIS and remote-sensing imaging applications is downscaling of some dataset or part thereof, such as obtaining a 1 GB overview of a 10 TB dataset. • Scaling is a very expensive operation as it normally requires a full scan of the dataset, plus costly main memory operations. Therefore, query optimization is critical to this class of retrieval operations. • Scaling is the only operation that has already been supported by pre-aggregation, at least for 2D datasets. This provides a point of reference to compare the effectiveness of our algorithms against existing techniques. Although the framework discussed in the following sections is centered around scaling operations, it can be adapted to support other non-standard aggregate operations by modifying the matching conditions as discussed later in this chapter. 5.2 Conceptual Framework A common optimization technique that speeds up scaling operations is to materialize selected downscaled versions of an object, e.g., using image pyramids. When evaluating a scaling operation with target scale factor s, the pyramid level with the largest scale factor s ′ is determined, where s ′ < s. This relationship between scaling operations places them within a lattice framework similar to that used for data cubes in data warehouse/OLAP applications [92]. Our conceptual framework and greedy algorithm for the selection of pre-aggregates is based on the work of Harinarayan et al. presented in [92]. The use of this approach was motivated by the similarities between our datasets (multidimensional arrays) and OLAP data cubes. Furthermore,
5.2 Conceptual Framework 79 Figure 5.1. Sample Lattice Diagram for a Workload with Five Scaling Operations the lattice framework and the greedy algorithm have proven successful in a variety of business applications. 5.2.1 Lattice Representation A scaling lattice consists of a set of queries L and dependence relations ≼ denoted by 〈L, ≼〉. The ≼ operator imposes a partial ordering on the queries of the lattice. Consider two queries q 1 and q 2 . We say q 1 ≼ q 2 if q 1 can be answered using only the results of q 2 . The base node of the lattice is the scaling operation with the smallest scale vector upon which every query is dependent. Lattices are commonly represented in a diagram in which the elements are nodes, and there is a path downward from q 1 to q 2 if and only if q 1 ≼ q 2 . The selection of pre-aggregates, that is, queries for materialization, is equivalent to selecting vertices from the underlying nodes of the lattice. Fig. 5.1 shows a lattice diagram for a workload containing five queries. Each node has an associated label that represents a scaling operation for a given dataset, scale-vector and resampling method. In our framework, we use the following function to define scaling operations: where scale(objName[lo 1 : hi 1 , ..., lo n : hi n ], ⃗s, resMeth) (5.1) • objName[lo 1 : hi 1 , ..., lo n : hi n ]: is the name of the multidimensional raster image to be scaled. The operation can be restricted to a specific area of the raster object. In that case, the area is specified by defining lower (lo n ) and upper (hi n ) bounds for each dimension. If the spatial domain is omitted, the operation is performed on the full spatial extent defining the raster image. • ⃗s: is a vector where each element is a numeric value that represents the scale factor used in a specific dimension of the raster image. • resMeth: specifies the resampling method to be applied to the original raster object.
Page 1:
Applying OLAP Pre-Aggregation Techn
Page 5 and 6:
Acknowledgments I would like to exp
Page 7 and 8:
Abstract Large multidimensional arr
Page 9 and 10:
Contents 1 Introduction and Problem
Page 11 and 12:
List of Figures 2.1 3D Array . . .
Page 13 and 14:
List of Tables 3.1 UNO and FAO Suit
Page 15 and 16:
Chapter 1 Introduction and Problem
Page 17 and 18:
Relevant and complementary question
Page 19 and 20:
1.2 Publications Related to this Th
Page 21 and 22:
Chapter 2 Background and Related Wo
Page 23 and 24:
2.1 Array Databases 17 Figure 2.2 s
Page 25 and 26:
2.1 Array Databases 19 toward the s
Page 27 and 28:
2.1 Array Databases 21 • Bilinear
Page 29 and 30:
2.1 Array Databases 23 given image
Page 31 and 32:
2.2 On-Line Analytical Processing (
Page 33 and 34: 2.2 On-Line Analytical Processing (
Page 39 and 40: 2.3 Discussion 33 spatial-vector da
Page 41 and 42: 2.3 Discussion 35 • Both applicat
Page 43 and 44: Chapter 3 Fundamental Geo-Raster Op
Page 45 and 46: 3.2 Geo-Raster Operations 39 3.1.2
Page 47 and 48: 3.2 Geo-Raster Operations 41 multip
Page 49 and 50: 3.2 Geo-Raster Operations 43 Table
Page 51 and 52: 3.2 Geo-Raster Operations 45 turn i
Page 53 and 54: 3.2 Geo-Raster Operations 47 (a) Or
Page 55 and 56: 3.2 Geo-Raster Operations 49 Query
Page 57 and 58: 3.2 Geo-Raster Operations 51 contai
Page 59 and 60: 3.2 Geo-Raster Operations 53 is the
Page 61 and 62: 3.2 Geo-Raster Operations 55 3.2.4
Page 63 and 64: 3.2 Geo-Raster Operations 57 As in
Page 65 and 66: 3.2 Geo-Raster Operations 59 Local
Page 67 and 68: 3.3 Summary 61 Slicing The slicing
Page 69 and 70: Chapter 4 Answering Basic Aggregate
Page 71 and 72: 4.1 Framework 65 pre-aggregated res
Page 73 and 74: 4.2 Cost Model 67 By partitioning t
Page 75 and 76: 4.2 Cost Model 69 Cost of independe
Page 77 and 78: 4.3 Implementation 71 Algorithm 1 Q
Page 79 and 80: 4.4 Experimental Results 73 Query E
Page 81 and 82: 4.5 Summary 75 pre-aggregates: inde
Page 83: Chapter 5 Pre-Aggregation Support B
Page 87 and 88: 5.2 Conceptual Framework 81 Benefit
Page 89 and 90: 5.4 Answering Scaling Operations Us
Page 91 and 92: 5.5 Experimental Results 85 Algorit
Page 93 and 94: 5.5 Experimental Results 87 (a) Que
Page 95 and 96: 5.5 Experimental Results 89 (a) Sel
Page 97 and 98: 5.5 Experimental Results 91 vectors
Page 99 and 100: 5.5 Experimental Results 93 root no
Page 101 and 102: 5.5 Experimental Results 95 Figure
Page 107 and 108: 5.6 Summary 101 we considered non-u
Page 109 and 110: Chapter 6 Conclusion One of the big
Page 111 and 112: 6.1 Future Work 105 more non-spatio
Page 113 and 114: Bibliography [1] Blakeley J. A., La
Page 115 and 116: BIBLIOGRAPHY 109 [22] Moon B., Vega
Page 117 and 118: BIBLIOGRAPHY 111 [47] ESRI Inc. Arc
Page 119 and 120: BIBLIOGRAPHY 113 [73] Stefanovic N.
Page 121: BIBLIOGRAPHY 115 [97] Kotidis Y. an
show all

Applying OLAP Pre-Aggregation Techniques to ... - Jacobs University

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?