PhD Thesis Semi-Supervised Ensemble Methods for Computer Vision

72 Chapter 5. On-line Semi-Supervised Boosting
as outlier and the weight is thus decreased. LogitBoost can be considered as lying between
these two extremes; i.e., if a sample is misclassified with high accuracy the logit-loss
neither increases the weight nor decreases the weight but keeps it constant to one.
Figure 5.4: Weight update functions for different loss functions.
5.2.3 Competitive Study
In this section, we will conduct a competitive study with the proposed on-line GradientBoost
on machine learning data in order to study the influence of the different loss
functions. Therefore, we chose 8 benchmark datasets from UCI and LIBSVM repositories,
which are shown in Table 6.1. We compare the performance of on-line AdaBoost and
GradientBoost by using exponential, Logit, DoomII, and Savage losses. Note, when using
exponential loss, we get the on-line formulation of RealBoost of Friedman et al. [Friedman
et al., 2000]. For these experiments, we randomly introduce label noise into the
training set and train the on-line classifiers for 5 epochs. We repeat all these experiments
for 3 times and report the average test errors.
We also repeat these experiments for two different kinds of weak learners, i.e., (i)
decision stumps which assume the feature responses being Gaussian distributed, where
the means µ + , µ − and the standard deviations σ + , σ − are estimated with the help of
a Kalman filter [Grabner and Bischof, 2006], and (ii) fixed-binned on-line histograms.
Some variants of on-line GradientBoost, e.g., RealBoost, need confidence-rated weak