Foundations of Data Science

Generation of random numbers according to a given probability distribution
Suppose one wanted to generate a random variable with probability density p(x) where
p(x) is continuous. Let P (x) be the cumulative distribution function for x and let u be
a random variable with uniform probability density over the interval [0,1]. Then the random
variable x = P −1 (u) has probability density p(x).
Example: For a Cauchy density function the cumulative distribution function is
P (x) =
∫ x
t=−∞
1 1
π 1 + t dt = 1 2 2 + 1 π tan−1 (x) .
Setting u = P (x) and solving for x yields x = tan ( π ( u − 1 2))
. Thus, to generate a
random number x ≥ 0 using the Cauchy distribution, generate u, 0 ≤ u ≤ 1, uniformly
and calculate x = tan ( π ( u − 1 2))
. The value of x varies from −∞ to ∞ with x = 0 for
u = 1/2.
12.4.10 Bayes Rule and Estimators
Bayes rule
Bayes rule relates the conditional probability of A given B to the conditional probability
of B given A.
Prob (B|A) Prob (A)
Prob (A|B) =
Prob (B)
Suppose one knows the probability of A and wants to know how this probability changes
if we know that B has occurred. Prob(A) is called the prior probability. The conditional
probability Prob(A|B) is called the posterior probability because it is the probability of
A after we know that B has occurred.
The example below illustrates that if a situation is rare, a highly accurate test will
often give the wrong answer.
Example: Let A be the event that a product is defective and let B be the event that a
test says a product is defective. Let Prob(B|A) be the probability that the test says a
product is defective assuming the product is defective and let Prob ( B|Ā) be the probability
that the test says a product is defective if it is not actually defective.
What is the probability Prob(A|B) that the product is defective if the test say it is
defective? Suppose Prob(A) = 0.001, Prob(B|A) = 0.99, and Prob ( B|Ā) = 0.02. Then
Prob (B) = Prob (B|A) Prob (A) + Prob ( B|Ā) Prob ( Ā )
= 0.99 × 0.001 + 0.02 × 0.999
= 0.02087
395