Machine Learning - DISCo

Hypotheses, estimation of accuracy
(continued)
bias and variance in estimate, 129,
151, 152
errors in, 129-131, 151
evaluation of, 128-129
justification of, in inductive vs. analytical
learning, 334-336
representations of, 23
testing of, 144-145
Hypothesis space, 14-15
bias in, 40-42, 46, 129
finite, sample complexity for, 207-214,
225
infinite, sample complexity for, 214-220
VC dimension of, 214-217
Hypothesis space search
by BACKPROPAGATION algorithm, 97,
106, 122-123
comparison with decision tree
learning, 106
comparison with KBANN and
TANGENTPROP algorithms, 350-35 1
by CANDIDATE-ELIMINATION algorithm,
64
in concept learning, 23-25, 46-47
constraints on, 302-303
by FIND-S algorithm, 27-28
by FOIL algorithm, 286-287, 357-361
by genetic algorithms, 250, 259
by gradient descent, 90-91
by ID3 algorithm, 60-62,64, 76
by KBANN algorithm, 346
by learning algorithms, 24
by LEARN-ONE-RULE, 277
in machine learning, 14-15, 18
use of prior knowledge, 339-340, 362
ID3 algorithm, 55-64,77
backtracking and, 62
CANDIDATE-ELIMINATION algorithm,
comparison with, 61-62
choice of attributes in, 280-281
choice of decision tree, 63
cost-sensitive measures, 75-76
extensions to, 77. See also C4.5
algorithm
inductive bias of, 63-64, 76
LEARN-ONE-RULE, search comparison
with, 277
limitations of, 61-62
overfitting in, 67-68
search of hypothesis space, 60-62, 64,
76
sequential covering algorithms,
comparison with, 280-281
specialized for concept learning, 56
use of information gain in, 58-60
ID5R algorithm, comparison with GABIL,
258
ILP. See Inductive logic programming
Image encoding in face recognition, 114
Imperfect domain theory:
in EBNN algorithm, 356
in explanation-based learning, 330
in FOCL algorithm, 360
in KBANN algorithm, 344-345
Incremental explanation methods, 328
Incremental gradient descent. See
Stochastic gradient descent
INCREMENTAL VERSION SPACE MERGING
algorithm, 47
Inductive-analytical learning, 334-363
advantages of, 362
explanation-based learning and, 330
learning problem, 337-338
prior knowledge methods to alter search,
339-340,362
properties of ideal systems, 337
weighting of components in EBNN
algorithm, 351-352,355
weighting prior knowledge in, 338
Inductive bias, 39-45, 137-138. See also
Occam's razor; Preference bias;
Restriction bias
of BACKPROPAGATION algorithm, 106
bias-free learning, 40-42
of CANDIDATE-ELIMINATION algorithm,
43-46, 63-64
in decision tree learning, 63-66
definition of, 43
in explanation-based learning, 322-323
of FIND-S algorithm, 45
of ID3 algorithm, 63-64,76
of inductive learning algorithms, 42-46
of k-NEAREST NEIGHBOR algorithm, 234