[Studies in Computational Intelligence 481] Artur Babiarz, Robert Bieda, Karol Jędrasiak, Aleksander Nawrat (auth.), Aleksander Nawrat, Zygmunt Kuś (eds.) - Vision Based Systemsfor UAV Applications (2013, Sprin

236 H. Josiński et al.
Principal component analysis is one of the classic linear methods of
dimensionality reduction. Given a set of high-dimensional feature vectors, PCA
projects them onto low-dimensional subspace spanned by principal components.
The first principal component is the direction which maximizes variance. The
second principal component maximizes variance in a direction orthogonal to (i.e.,
uncorrelated with) the first component. Generally, the k-th principal component
(k > 1) indicates the direction along which variance is maximum among all
directions orthogonal to the preceding k-1 components. PCA is most useful in the
case when data lie on or close to a linear subspace of the data set [22].
Isometric features mapping and locally linear embedding are algorithms
proposed for manifold learning which is an approach to non-linear dimensionality
reduction. Manifold learning is the process of estimating a low-dimensional
structure which underlies a collection of high-dimensional data [23]. In other
words, manifold learning algorithms essentially attempt to duplicate the behavior
of PCA but on manifolds instead of linear subspaces [22].
The Isomap algorithm [24] comprises the following stages:
1. Estimate pairwise the geodesic distances (distances along a manifold) between
all data points. To perform this estimation, first construct a k-nearest neighbor
graph weighted by the Euclidean distances. Then compute the shortest-path
distances using Dijkstra’s or Floyd-Warshall’s algorithm. As the number of
data points increases, this estimate converges to the true geodesic distances.
2. Use the Multidimensional Scaling (MDS) [23] to find points in lowdimensional
Euclidean space whose interpoint distances match the distances
estimated in the previous step.
A D-dimensional manifold can be arbitrarily well-approximated by a d-
dimensional linear subspace by taking a sufficiently small region about any point.
Consequently, the LLE algorithm finds a local representation of each point based
on its k nearest neighbors (measured by Euclidean distance) assuming that within
the neighborhood the manifold is approximately flat and then reconstructs a lowdimensional
representation with a similar configuration (see Fig. 1 – an often cited
illustration from Saul and Roweis [25]).
The LLE algorithm consists of the following three steps:
1. Assign k nearest neighbors to each data point X i , 1 ≤ i ≤ n in D-dimensional
space.
2. Compute the weights W ij in order to reconstruct linearly each data point from
its k nearest neighbors. The weights are chosen to minimize the reconstruction
error:
( )
ε W = X − W X
i ij j
i j∈N ( i)
2
, (1)
where N(i) denotes the set of k nearest neighbors of the data point X i . The
reconstruction weights sum to 1 and W ij = 0 if j∉ N()
i .