Artificial Intelligence and Soft Computing: Behavioral ... - Arteimi.info

D = distribution by which the sample examples are drawn,
m = cardinality of examples in the training set,
H = the set of possible hypothesis,
and f = function that is approximated by the hypothesis h.
We now define an error of hypothesis h by
Error(h) = P(h(x) ≠ f(x) ⏐xεD)
A hypothesis h is said to be approximately correct when
error(h) ≤ ε,
where ε is a small positive quantity.
When an approximate hypothesis h is true, it must lie within the ε-ball around
f. When h lies outside the ε-ball we call it a bad hypothesis. [13]
Now, suppose a hypothesis hb ∈ Hbad is supported by first m examples. The
probability that a bad hypothesis is consistent with an example ≤(1-ε). If we
consider m examples to be independent, the probability that all m samples will
be consistent with hypothesis hb is ≤(1-ε) m . Now, if Hbad has to contain a
consistent hypothesis, at least one of the hypothesis of Hbad should be
consistent. The probability of this happening is bounded by the sum of
individual probabilities.
H bad
Fig. 13.15: The ball around f.
Thus P(a consistent hb ∈ Hbad) ≤ | Hbad | (1 - ε ) m
≤ |H| (1 - ε) m
where | Hbad | and |H| denotes the cardinality of them respectively. If we put a
small positive upper bound δ to the above quantity: we find
f
H