PhD Thesis Semi-Supervised Ensemble Methods for Computer Vision

Chapter 8
Multiple Instance Learning with
Random Forests
Semi-supervised learning algorithms have to learn from ambiguously labeled samples
because the true labels of unlabeled samples are unknown. In machine learning, there
exist a second learning paradigm called multiple-instance learning (MIL) [Keeler et al.,
1990, Dietterich et al., 1997] which is very similar to SSL and also has to resolve ambiguities
during the learning process. In particular, in multiple-instance learning, training
samples are provided in form of bags, where each bag consists of several instances. Labels
are only provided for the bags and not for the instances. The labels of instances inside
positive bags are unknown, but it is guaranteed that at least one instance has a positive
label. Contrary, in negative bags all instances can be considered as being negative. (See
also Figure 8.1 for an illustration of the principle.)
In this chapter, we present a multiple-instance learning algorithm based on random
forests. We thus call the method MILForests. MILForests bring the advantages of random
forests – i.e., speed, multi-class capability, multi-processing, noise resistance, etc.–
to multiple-instance learning, where usually different methods have been applied. In turn,
extending random forests in order to allow for multiple-instance learning allows vision
tasks where RFs are typically applied to benefit from the flexibility of MIL. MILForests
are very similar to conventional random forests. However, since the training data is provided
in form of bags, during learning the real class labels of instances inside bags are
unknown. In the following, we will show that multiple-instance learning is a special case
of SSL because all instances inside negative bags can be considered as labeled samples
and all instances inside positive bags as unlabeled, respectively. Based on this insight we
can use a similar optimization approach as for the semi-supervised RFs, i.e., deterministic
annealing, in order to find the hidden class labels of instances in positive bags.
103