Real-time feature extraction from video stream data for stream ...

3.2. Supervised Learning

A machine learning algorithm takes its input X train and looks **for** some model. A common

type of a model is a function ˆf that maps elements **from** the input space X to elements

**from** the output space Y :

ˆf : X → Y

When this model gets applied to an example **from** the test **data** x (i) ∈ X test , it produces

an output ŷ ∈ Y . Simplifying we can assume, that each ŷ is a label.

According to [Hastie et al., 2001] the field of machine learning splits into two major subtasks:

supervised and unsupervised learning. The distinctions results **from** the presence

or absence of labels in the training **data** X train .

3.2. Supervised Learning

Supervised Learning is the task of inferring a model ˆf **from** labeled training **data**. The

input **data** set X train **for** supervised learning algorithms again consists out of N examples,

each represented by a p-dimensional vector. One **feature** of each example is designated

as the true label **for** that example. There**for**e we can also view X train as a set of pairs

(⃗x, y), where y is the label.

X train = {(⃗x (1) , y (1) ), ...., (⃗x (N) , y (N) )} ⊂ X × Y

Based on this input **data**, the machine learning algorithm infers a function that predicts

labels ŷ ∈ Y **for** unlabeled test **data** X test . In case the number of possible outputs is

discrete (often y ∈ {+1, −1}), the model is called a classifier. If the predicted output is

continuous (Y ⊆ R), the model is a regression function. Respectively the learning tasks

are called classification and regression.

Definition 8 (Classification) Classification is the machine learning task of inferring

a function ˆf : X → Y **from** nominally labeled training **data** X train . The model is chosen

with respect to a quality criterion g that must be optimized.

Definition 9 (Regression) Regression is the machine learning task of inferring a function

ˆf : X → Y **from** numerically labeled training **data** X train . The model is chosen with

respect to a quality criterion g that must be optimized. In many cases the label is only

binominal.

The above mentioned introducing example of recognizing handwritten digits on mailing

envelopes is a typical example **for** a classification problem. The input of the learning

algorithm consists of labeled examples of handwritten digits. The inferred model later

predicts nominal labels ŷ ∈ {0, 1, 2, ..., 9} **for** unlabeled examples.

Examples **for** supervised learning algorithms are linear regression, k-nearest-neighbor

algorithms (kNN), support vector machines (SVMs) or decision tree learners, just to

mention a few. In following kNN and decision trees are explained in further detail, as

they are used in chapter 7.

29