Semantic Interpretation of Digital Aerial Images Utilizing ...

28 Chapter 3. From Appearance and 3D to Interpreted Image Pixels
interpretation of images. On the one hand, the aerial images provide a nearly constant
stuff object scale that is mainly defined by the GSD of the aerial project. The local interpretation
can be thus performed for one scale which is mainly defined by the patch size.
On the other hand, the huge variability of the mapped data and the undefined object’s
shapes make a top-down recognition strategy intractable to solve. In addition the training
sample generation does not need entirely annotated objects, but rather an efficient assigning
of object classes by drawing strokes. We therefore concentrate on a local explanation
of the images and introduce the exact object extraction in a later step (see Chapter 5).
We extensively exploit statistical Sigma Points [Julier and Uhlmann, 1996] features,
directly derived from the well-established covariance descriptors [Tuzel et al., 2006] in
combination with RF classifiers [Breiman, 2001] to compactly describe and classify color,
basic texture and elevation measurements within local image regions. The combination
of the derived statistical features and RF classifiers provides several advantages for largescale
computations in aerial imagery. Since the aerial imagery consists of multiple information
sources, there is a need to reasonably combine these low-level information cues.
We therefore apply a Sigma Points feature representation to compactly describe different
channels considering a small local neighborhood. Compared to computed histograms
over multi-spectral data, these descriptors are low-dimensional and enable a simple integration
of appearance and height information, that is then represented on a Euclidean
vector space. Moreover, they can be quickly computed for each pixel using integral image
structures and also support parallel computation techniques.
Randomized forests have proven to give robust and accurate results for challenging multiclass
classification tasks [Lepetit and Fua, 2006, Shotton et al., 2008]. RFs are very efficient
at runtime since the final decision is made on fast binary decisions between a small
number of selected feature attributes. In addition, the classifier can be efficiently trained
on a large amount of data and can handle some errors in the training data.
In the following sections we first review related work in the context of semantic image
interpretation and classification. Then, we outline the core parts of our semantic interpretation,
consisting of a powerful feature representation (Section 3.4), the classifier (Section
3.5) and refinement steps to obtain an improved final class labeling (see Sections 3.6
and 3.7).
3.3 Related Work
While recently proposed approaches aim at extracting coarse scene geometry directly
from interpretation results [Hoiem et al., 2007,Saxena et al., 2008,Gould et al., 2009,Liu
et al., 2010] or try to jointly estimate classification and dense reconstruction [Ladicky
et al., 2010b], we rather focus on directly integrating available 3D data in our interpre-