Slides in PDF - of Marcus Hutter

More documents

Recommendations

Info

Marcus Hutter - 46 - Universal Induction & Intelligence How to Choose the Prior The probability axioms allow relating probabilities and plausibilities of different events, but they do not uniquely fix a numerical value for each event, except for the sure event Ω and the empty event {}. We need new principles for determining values for at least some basis events from which others can then be computed. There seem to be only 3 general principles: • The principle of indifference — the symmetry principle • The maximum entropy principle • Occam’s razor — the simplicity principle Concrete: How shall we choose the hypothesis space {H i } and their prior p(H i ) –or– M = {ν} and their weight w ν .
Marcus Hutter - 47 - Universal Induction & Intelligence Indifference or Symmetry Principle Assign same probability to all hypotheses: p(H i ) = 1 |I| for finite I p(H θ ) = [Vol(Θ)] −1 for compact and measurable Θ. ⇒ p(H i |D) ∝ p(D|H i ) ∧ = classical Hypothesis testing (Max.Likelihood). Prev. Example: H θ =Bernoulli(θ) with p(θ) = 1 for θ ∈ Θ := [0, 1]. Problems: Does not work for “large” hypothesis spaces: (a) Uniform distr. on infinite I = IN or noncompact Θ not possible! (b) Reparametrization: θ ❀ f(θ). Uniform in θ is not uniform in f(θ). Example: “Uniform” distr. on space of all (binary) sequences {0, 1} ∞ : p(x 1 ...x n ) = ( 1 2 )n ∀n∀x 1 ...x n ⇒ p(x n+1 = 1|x 1 ...x n ) = 1 2 always! Inference so not possible (No-Free-Lunch myth). Predictive setting: All we need is p(x).
Page 1 and 2: Foundations of Universal Induction
Page 3 and 4: Marcus Hutter - 3 - Universal Induc
Page 11 and 12: Marcus Hutter - 11 - Universal Indu
Page 45: Marcus Hutter - 45 - Universal Indu
Page 91 and 92: Language Tree (Re)construction base
Page 97 and 98:
Marcus Hutter - 97 - Universal Indu
Page 99 and 100:
Marcus Hutter - 99 - Universal Indu
Page 101 and 102:
Marcus Hutter - 101 - Universal Ind
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111:
show all

Slides in PDF - of Marcus Hutter

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?