PhD Thesis Semi-Supervised Ensemble Methods for Computer Vision

118 Chapter 9. Visual Object Tracking
9.2 An one-shot semi-supervised learning formulation for
tracking
Tracking-by-detection, in principle, is an ill-posed problem. The complicated task in
tracking using an appearance-based on-line classifier is to continuously apply self-training
while avoiding wrong updates that my cause drifting. As already discussed above, the
problem with these approaches is that the self-updating process may easily cause drifting
in case of wrong updates. Even worse, the tracking-by-detection approach suffers
also from the fact that usually on-line counterparts of supervised learning algorithms are
used, which are not designed for handling ambiguity of class labels; for example, despite
the fact that boosting is known to by highly susceptible to label noise – as we have
seen in Section 5.2 – it is widely used in self-learning based tracking methods. This
severe problem of adaptive tracking-by-detection methods can also be explained by the
exploration-exploitation problem or the stability-plasticity dilemma [Grossberg, 1998]:
If the classifier is trained only with the first frame, it is less error-prone to occlusions and
can virtually not drift. However, non adaptive classifiers are not able to follow an object
undergoing rapid appearance and viewpoint changes. On the other hand, on-line classifiers
that perform self-learning on their confidence maps are highly adaptive but easily
drift in case of wrong updates.
As can be easily observed, the only time when the classifier can assume having correct
labels is at frame t = 0. During ongoing tracking, the classifier observes exclusively unlabeled
samples and can only rely on its own believes. Hence, from a learning perspective,
we have to solve a learning task where the individual samples arrive sequentially, are only
labeled at the beginning and the rest of the time unlabeled, respectively. We thus state the
following proposition:
Tracking-by-detection is an one-shot semi-supervised learning problem!
Following this observation, we further argue that one should apply semi-supervised
learning methods rather than supervised ones, which is more intuitive. Hence, in Figure
9.4 we present a modified tracking loop where labeled data exist only in the first
frame. In all subsequent frames t with t = 1, · · · , ∞, we exploit subpatches as unlabeled
samples. Since this is a classical semi-supervised learning formulation, we can use one of
the semi-supervised boosting approaches (i.e., on-line SemiBoost and on-line SERBoost)
or semi-supervised random forests, introduced in this thesis as learners. As the semisupervised
boosting approaches need a prior classifier that “guides” the on-line learner