Machine Learning - DISCo

section, if we were to repeat this experiment many times, each time drawing a
different random sample Si of size n, we would expect to observe different values
for the various errors,(h), depending on random differences in the makeup of
the various Si. We say in such cases that errors, (h), the outcome of the ith such
experiment, is a random variable. In general, one can think of a random variable
as the name of an experiment with a random outcome. The value of the random
variable is the observed outcome of the random experiment.
Imagine that we were to run k such random experiments, measuring the random
variables errors, (h), errors, (h) . . . errors, (h). Imagine further that we then
plotted a histogram displaying the frequency with which we observed each possible
error value. As we allowed k to grow, the histogram would approach the form
of the distribution shown in Table 5.3. This table describes a particular probability
distribution called the Binomial distribution.
0.14
0.12
0.1
0.08
'F 0.06
0.04
0.02
Binomial dishibution for n = 40, p =0.3
0
0 5 10 15 20 25 30 35 40
A Binomial distribution gives the probability of observing r heads in a sample of n independent
coin tosses, when the probability of heads on a single coin toss is p. It is defined by the probability
function
n!
P(r) = pr(l - p)"-'
r!(n - r)!
If the random variable X follows a Binomial distribution, then:
0 The probability Pr(X = r) that X will take on the value r is given by P(r)
0 The expected, or mean value of X, E[X], is
0 The variance of X, Var(X), is
0 The standard deviation of X, ax, is
Var (X) = np(1- p)
For sufficiently large values of n the Binomial distribution is closely approximated by a Normal
distribution (see Table 5.4) with the same mean and variance. Most statisticians recommend using
the Normal approximation only when np(1- p) 2 5.
TABLE 53
The Binomial distribution.