Slides in PDF - of Marcus Hutter

Marcus Hutter - 26 - Universal Induction & Intelligence
Example: Bayes’ and Laplace’s Rule
Assume data is generated by a biased coin with head probability θ, i.e.
H θ :=Bernoulli(θ) with θ ∈ Θ := [0, 1].
Finite sequence: x = x 1 x 2 ...x n with n 1 ones and n 0 zeros.
Sample infinite sequence: ω ∈ Ω = {0, 1} ∞
Basic event: Γ x = {ω : ω 1 = x 1 , ..., ω n = x n } = set of all sequences
starting with x.
Data likelihood: p θ (x) := p(Γ x |H θ ) = θ n 1
(1 − θ) n 0
.
Bayes (1763): Uniform prior plausibility: p(θ) := p(H θ ) = 1
( ∫ 1
0 p(θ) dθ = 1 instead ∑ i∈I p(H i) = 1)
Evidence: p(x) = ∫ 1
0 p θ(x)p(θ) dθ = ∫ 1
0 θn 1
(1 − θ) n 0
dθ = n 1!n 0 !
(n 0 +n 1 +1)!