Algorithmic Learning Theory - of Marcus Hutter

Editors’ Introduction 3
John Case, Timo Kötzing and Todd Paddock study a setting of learning
in the limit in which the time to produce the final hypothesis is derived from
some ordinal which is updated step by step downwards until it reaches zero,
via some “feasible” functional. Their work first proposes a definition of feasible
iteration of feasible learning functionals, and then studies learning hierarchies
defined in terms of these notions; both collapse results and strict hierarchies
are established under suitable conditions. The paper also gives upper and lower
runtime bounds for learning hierarchies related to these definitions, expressed in
terms of exponential polynomials.
John Case and Samuel Moelius III study iterative learning. This is a variant of
the Gold-style learning model described above in which each of a learner’s output
conjectures may depend only on the learner’s current conjecture and on the
current input element. Case and Moelius analyze two extensions of this iterative
model which incorporate parallelism in different ways. Roughly speaking, one of
their results shows that running several distinct instantiations of a single learner
in parallel can actually increase the power of iterative learners. This provides
an interesting contrast with many standard settings where allowing parallelism
only provides an efficiency improvement. Another result deals with a “collective”
learner which is composed of a collection of communicating individual learners
that run in parallel.
Sanjay Jain, Frank Stephan and Nan Ye study some basic questions about
how hypothesis spaces connect to the class of languages being learned in Goldstyle
models. Building on work by Angluin, Lange and Zeugmann, their paper
introduces a comprehensive unified approach to studying learning languages in
the limit relative to different hypothesis spaces. Their work distinguishes between
four different types of learning as they relate to hypothesis spaces, and gives
results for vacillatory and behaviorally correct learning. They further show that
every behaviorally correct learnable class has a prudent learner, i.e., a learner
using a hypothesis space such that it learns every set in the hypothesis space.
Sanjay Jain and Frank Stephan study Gold-style learning of languages in
some special numberings such as Friedberg numberings, in which each set has
exactly one number. They show that while explanatorily learnable classes can
all be learned in some Friedberg numberings, this is not the case for either behaviorally
correct learning or finite learning. They also give results on how other
properties of learners, such as consistency, conservativeness, prudence, iterativeness,
and non U-shaped learning, relate to Friedberg numberings and other
numberings.
Complexity aspects of learning. Connections between complexity and learning
have been studied from a range of different angles. Work along these lines has
been done in an effort to understand the computational complexity of various
learning tasks; to measure the complexity of classes of functions using parameters
such as the Vapnik-Chervonenkis dimension; to study functions of interest
in learning theory from a complexity-theoretic perspective; and to understand
connections between Kolmogorov-style complexity and learning. All four of these
aspects were explored in research presented at ALT 2007.