PhD Thesis Semi-Supervised Ensemble Methods for Computer Vision

Chapter 4
SemiBoost and Visual Similarity
Learning
As we have motivated in the previous chapter, semi-supervised learning is an increasingly
important learning paradigm with various potential practical applications. Additionally,
we have seen that there already exist a large amount of different SSL approaches.
A large subset of SSL methods, especially those based on graphs and the manifold
assumption, rely on a priori given similarities or distance measures which are needed
in order to compare samples (labeled and unlabeled) in feature space. It is clear that the
accuracy of these similarities determines the success of the semi-supervised learners. One
way to obtain good similarities is to learn similarity or distance functions that can then be
used for further reasoning or processing.
In the following, we present an approach that combines visual similarity learning
and semi-supervised boosting using manifold regularization. In particular, we use a limited
amount of labeled samples in order to, first, train a discriminant distance function
and, second, use this distance function as a metric in order to guide a variant of Semi-
Boost [Mallapragada et al., 2009] through exploiting a huge set of unlabeled data.
4.1 Learning Distance Functions
In machine learning problems, the distance metrics are often given in huge matrices that
encode standard distances such as the Euclidean or the Chi-Squared. Although this works
for many applications it has several fundamental problems: First, the data stored in matrices
grows with O(n 2 ), where n is the number of samples. Second, it is often hard to
decide which is the best similarity metric in order to solve a certain task. Especially in
computer vision, it is difficult to determine what makes some digital images similar and
43