PhD Thesis Semi-Supervised Ensemble Methods for Computer Vision

4.2. SemiBoost 47
Therefore, at each step of boosting, we compute the pseudo-labels and weights of
unlabeled samples and use them to find the best weak learner and its corresponding weight
similar to the AdaBoost algorithm. Note that if no unlabeled data is used, the algorithm
reduces to standard boosting. The detailed steps of the algorithm are summarized in
Algorithm 4.1.
Algorithm 4.1 SemiBoost
Require: labeled training data (x, y) ∈ X L and unlabeled data x ′ ∈ X U
Require: Similarity measure s(x, x ′ )
Require: Weak learners f i
Require: weight parameters λ u , λ l
Require: max iterations T
1: for t = 1, 2, . . . , T do
2: Compute p i and q i for every given sample
3: ŷ x = sign(p x − q x )
4: w x = |p x − q x |
5: Train weak classifier f t (x)
6: Compute α t using Equation (4.13)
7: F (x) ← F (x) + α t f t (x)
8: end for
4.2.2 Learning Visual Similarities
Being a manifold-based approach SemiBoost has the power to exploit both labeled and
unlabeled samples if a similarity measure s(x, x ′ ) is given. The similarity can be obtained
from a distance measure d(x, x ′ ), e.g., by using a radial basis function
s(x, x ′ ) = e
(
− d(x,x′ ) 2
σ 2 ), (4.14)
where σ 2 is the scale parameter [Zhu et al., 2003]. The crucial point is how to measure the
distance d(x, x ′ ) between points. Popular measurements are, for example, the Euclidean
distance ||x − x ′ || or the Chi-Square distance (x−x′ ) 2
. 2(x+x ′ )
As already mentioned above, a more powerful and flexible approach is to learn the
distance function from labeled data, which is also known as metric learning. The advantage
of discriminative learning of distance functions is that the metric can much better
support task-specific classification. We use boosting to learn pair-wise distance functions
similar to the method proposed by Hertz et al. [Hertz et al., 2004]. A distance function is
a function of pairs of data points to be positive real numbers, usually (but not necessary)