Brain–Computer Interfaces - Index of

118 B. Blankertz et al.
2.2 Regularized Linear Classification
There is some debate about whether to use linear or non-linear methods to classify
single EEG trials; see the discussion in [33]. In our experience, linear methods perform
well, if an appropriate preprocessing of the data is performed. E.g., band-power
features itself are far from being Gaussian distributed due to the involved squaring
and the best classification of such features is nonlinear. But applying the logarithm
to those features makes their distribution close enough to Gaussian such that linear
classification typically works well. Linear methods are easy to use and robust. But
there is one caveat that applies also to linear methods. If the number of dimensions
of the data is high, simple classification methods like Linear Discriminant Analysis
(LDA) will not work properly. The good news is that there is a remedy called
shrinkage (or regularization) that helps in this case. A more detailed analysis of
the problem and the presentation of its solution is quite mathematical. Accordingly,
the subsequent subsection is only intended for readers interested in those
technical details.
2.2.1 Mathematical Part
For known Gaussian distributions with the same covariance matrix for all classes,
it can be shown that Linear Discriminant Analysis (LDA) is the optimal classifier
in the sense that it minimizes the risk of misclassification for new samples drawn
from the same distributions [34]. Note that LDA is equivalent to Fisher Discriminant
Analysis and Least Squares Regression [34]. For EEG classification, the assumption
of Gaussianity can be achieved rather well by appropriate preprocessing of the data.
But the means and covariance matrices of the distributions have to be estimated
from the data, since the true distributions are not known.
The standard estimator for a covariance matrix is the empirical covariance (see
equation (1) below). This estimator is unbiased and has under usual conditions good
properties. But for extreme cases of high-dimensional data with only a few data
points given, the estimation may become inprecise, because the number of unknown
parameters that have to be estimated is quadratic in the number of dimensions.
This leads to a systematic error: Large eigenvalues of the original covariance
matrix are estimated too large, and small eigenvalues are estimated too small; see
Fig. 4. This error in the estimation degrades classification performance (and invalidates
the optimality statement for LDA). Shrinkage is a common remedy for the
systematic bias [35] of the estimated covariance matrices (e.g. [36]):
Let x1, ..., xn ∈ Rd be n feature vectors and let
ˆ� = 1
n − 1
n�
(xi −ˆμ)(xi −ˆμ) ⊤
i=1
be the unbiased estimator of the covariance matrix. In order to counterbalance the
estimation error, ˆ� is replaced by
(1)