The dissertation of Andreas Stolcke is approved: University of ...

CHAPTER 8. FUTURE DIRECTIONS 183towards a unified induction framework would therefore study how specializing algorithms can be combinedwith posterior probability maximization to advantage.8.5 New applicationsWe already remarked that natural language is a less than ideal application domain for exclusivelyor mainly syntax-oriented probabilistic characterizations, due to the wide range of extra-syntactic constraintsit is subject to. Formal language models are starting to be used more widely, in such areas as computationalbiology, graphical modeling (‘picture grammars’), and document structure analysis. Probabilistic learningapproaches, especially structural learning, still await study in these areas.Still, natural language remains an important topic. The difficulties cited earlier seem to indicatethat no single learning algorithm or paradigm can hope to be a practical way of inducing suitable models fromlarge corpora. A natural strategy therefore is to make more selective, and combined use of partial solutions, asindicated at the end of Chapter 4. A major problem in this regard, apart from the obvious one of identifyingtheright partial solutions worth combining, will be to find the unifying principles that allow different approachesto ‘talk’ to one another. Here it seems that probability theory (and the derived information theoretic concepts,such as description length) can again play a central role.8.6 New types of probabilistic modelsUltimately, probabilistic approaches to language need to identify alternatives to the traditionalformalism used to date. We have seen that each such formalism defines itself by the underlying notion ofderivation structure and the set of conditional independence assumptions made in defining the probabilities ofderivations. In choosing these, there is always a design trade-off between capturing the relevant probabilisticcontingencies found in a domain, and the computational expense of the associated algorithms for parsing,estimation, etc. It seems that properties of learnability (by model merging or otherwise), both complexityand robustness, should be added to these criteria. Model merging is a natural theoretical framework in thiscontext, since it applies widely across model classes, and can thus serve as a basis for comparison in questionsof learnability.