CRANFIELD UNIVERSITY Eleni Anthippi Chatzimichali ...

1.6.2 k-fold Cross-Validation
Cross-validation (Wold, 1978) is the most popular validation technique. In
-fold cross-validation, the entire dataset is randomly split into mutually exclusive
subsets – the folds – of approximately equal size (Kohavi, 1995; Duan et al., 2003;
Izenman, 2008). The algorithm of -fold cross-validation performs iterations in
total. As demonstrated in Figure 1-7, in each iteration, one subset is considered to be
the test set, while the remaining folds are used to train the classifier. Therefore,
each fold will be used exactly once for testing. As thoroughly described in the
previous section, each test set is kept aside and should in no way be used during the
development of the model (Brereton, 2006; Westerhuis et al., 2008). The total
prediction rate of the classifier is calculated by averaging the individual test results
over the iterations.
A great advantage of -fold cross-validation is that all examples in the dataset are
eventually used for both training and testing. Thus, the bias of this estimate is reduced
compared to the holdout method. In general, the outcome of -fold cross-validation
depends highly on the split of the initial dataset into folds. Since this partition is not
canonical, it may often lead to instances of high variance, especially for large values
of (Glasmachers, 2008); thus, or are most commonly applied for
cross-validation (Clarke et al. 2009).
Figure 1-7 k-fold Cross-Validation
In -fold cross-validation, the initial dataset is partitioned into mutually exclusive folds of
approximately equal size. In every run, a single fold is omitted and the remaining sets are used in
the model’s training process. One can conclude that the split and the allocation of samples into the
folds may easily influence the prediction rates. The figure has been extracted from
http://research.cs.tamu.edu/prism/lectures/iss/iss_l13.pdf
25