Applying OLAP Pre-Aggregation Techniques to ... - Jacobs University

Chapter 1
Introduction and Problem Statement
Scientific computing platforms and infrastructures are making new kinds of experiments
possible, resulting in the generation of vast volumes of arrays of data. This
is happening in many specialized application areas such as meteorology, oceanography,
hydrology, astronomy, medical imaging, and exploration systems for oil, natural
gas, coal, and diamonds. These datasets range from uniformly spaced points
(cells) along a single dimension to multidimensional arrays containing several different
types of data. For example, astronomy and earth sciences operate on two- or
three-dimensional spatial grids, often using a plethora of spherical coordinate systems.
Furthermore, nearly all sciences must deal with data series over time. It is frequently
necessary to understand relationships between consecutive elements in time,
or to analyze entire sequences of observations, and such datasets may represent spatial,
temporal, or spatio-temporal information. For example, if ocean measurements
such as temperature, salinity, and oxygen are recorded every hour at spacings of every
one meter in depth, and every ten meters in two horizontal dimensions, the result is
a four-dimensional array with three spatial dimensions and one temporal dimension,
and three values attached to each cell of the array.
In the past, arrays were typically stored in files and then manipulated by programs
that operated on these files. Nowadays, with science moving toward being computational
and data based, the trend is toward a new class of database system which provides
support for not only traditional, or coded, data types such as text, integers, etc.,
but also richer data types like multidimensional arrays. This new trend of databases is
referred to as Array Databases.
Implementing an efficient array database management system (DBMS) can be very
challenging. Typically, there are two approaches that can be taken to store array
datasets in a DBMS. In the first, the values of each cell are stored in a separate row,
along with fields describing the position of the cell in the array. The most obvious
drawback of this approach is the need for a large multidimensional index to efficiently
find rows in the table. Moreover, the space taken by a multidimensional index is larger
than the size of the table itself if all dimensions forming an array are used as the key.
In the second approach, a multidimensional array is written to a Binary Large Object
(BLOB), which is stored in a field of a table in the database. Applications then fetch
9