considering autocorrelation in predictive models - Department of ...

4 Introduction
analysis of such data needs to take this into account.
Such work either removes the autocorrelation dependencies during pre-processing and then use traditional
algorithms (e.g., (Hardisty and Klippel, 2010; Huang et al, 2004)) or modifies the classical
machine learning, data mining and statistical methods in order to consider the autocorrelation (e.g., (Bel
et al, 2009; Rinzivillo and Turini, 2004, 2007)). There are also approaches which use a relational setting
(e.g., (Ceci and Appice, 2006; Malerba et al, 2005)), where the autocorrelation is usually incorporated
through the data structure or defined implicitly through relationships among the data and other data
properties.
However, one limitation of most of the approaches that take autocorrelation into account is that they
assume that autocorrelation dependencies are constant (i.e., do not change) throughout the space/network
(Angin and Neville, 2008). This means that possible significant variability in autocorrelation dependencies
in different points of the space/network cannot be represented and modeled. Such variability could
result from a different underlying latent structure of the space/network that varies among its parts in
terms of properties of nodes or associations between them. For example, different research communities
may have different levels of cohesiveness and thus cite papers on other topics with varying degrees. As
pointed out by Angin and Neville (2008), when autocorrelation varies significantly throughout a network,
it may be more accurate to model the dependencies locally rather than globally.
In the dissertation, we extend the predictive clustering framework in the context of PCTs that are
able to deal with data (spatial and network) that do not follow the i.i.d. assumption. The distinctive characteristic
of the proposed approach is that it explicitly considers the non-stationary (spatial and network)
autocorrelation when building the predictive models. Such a method not only extends the applicability of
the predictive clustering approach, but also exploits the autocorrelation phenomenon and uses it to make
better predictions and better models.
In traditional PCTs (Blockeel, 1998), the tree construction is performed by maximizing variance
reduction. This heuristic guarantees, in principle, accurate models since it reduces the error on the
training set. However, it neglects the possible presence of autocorrelation in the training data. To address
this issue, we propose to simultaneously maximize autocorrelation for spatial/network domains. In this
way, we exploit the spatial/network structure of the data in the PCT induction phase and obtain predictive
models that naturally deal with the phenomenon of autocorrelation.
The consideration of autocorrelation in clustering has already been investigated in the literature,
both for spatial clustering (Glotsos et al, 2004) and network clustering (Jahani and Bagherpour, 2011).
Motivated by the demonstrated benefits of considering autocorrelation, we exploit some characteristics
of autocorrelated data to improve the quality of PCTs. The consideration of autocorrelation in clustering
offers several advantages, since it allows us to:
• determine the strength of the spatial/network arrangement on the variables in the model;
• evaluate stationarity and heterogeneity of the autocorrelation phenomenon across space;
• identify the possible role of the spatial/network arrangement/distance decay on the predictions
associated with each of the nodes of the tree;
• focus on the spatial/network “neighborhood” to better understand the effects that it can have on
other neighborhoods and vice versa.
These advantages of considering spatial autocorrelation in clustering, identified by (Arthur, 2008),
fit well into the case of PCTs. Moreover, as recognized by (Griffith, 2003), autocorrelation implicitly
defines a zoning of a (spatial) phenomenon: Taking this into account reduces the effect of autocorrelation
on prediction errors. Therefore, we propose to perform clustering by maximizing both variance reduction