Dr. Vasant Honavar Department of Computer Science honavar@cs ...

Topics in Grammatical Inference and Computational Learning Theory
Grammatical Inference, variously referred to as automata induction, grammar induction, and automatic language
acquisition, refers to the process of learning of grammars and languages from data. Machine learning of grammars finds a
variety of applications in syntactic pattern recognition, adaptive intelligent agents, diagnosis, computational biology,
systems modeling, prediction, natural language acquisition, data mining and knowledge discovery. Regular grammars are
the simplest class of formal grammars in the Chomsky hierarchy. An understanding of the issues and problems
encountered in learning regular languages (or equivalently, identification of the corresponding deterministic finite
automaton (DFA)) are therefore likely to provide insights into the problem of learning more general classes of languages.
Under the standard complexity theoretic assumption P ≠NP, Pitt and Warmuth showed that no polynomial time algorithm
can be guaranteed to produce a DFA that has approximately the same number of states as the target DFA from a set of
labeled examples corresponding to a DFA. When examples are drawn at random (as in the PAC setting), results proved
by Kearns and Valiant suggest that an efficient algorithm for learning DFA would entail efficient algorithms for solving
problems such as breaking the RSA cryptosystem, factoring Blum integers, and detecting quadratic residues, all of which
are known to be hard under standard cryptographic assumptions.
Against the background of strong negative results we investigated the feasibility of learning regular languages from
examples under additional assumptions concerning the distribution from which the examples are drawn, thereby
addressing the problem posed by Pitt, in his seminal paper: Are DFA PAC-identifiable if examples are drawn from the
uniform distribution, or some other known simple distribution? We showed that:
(a) The class of simple DFA (i.e., DFA whose canonical representations have logarithmic Kolmogorov complexity) is
efficiently PAC learnable under the Solomonoff Levin universal distribution (Parekh and Honavar, 1999)
(b) If the examples are sampled at random according to the universal distribution by a teacher that is
knowledgeable about the target concept, the entire class of DFA is efficiently PAC learnable under the universal
distribution, that is, DFA are efficiently learnable under the PACS Model (Parekh and Honavar, 1999; Parekh
and Honavar, 2001)
(c) Any concept that is learnable under Gold’s model for learning from characteristic samples, Goldman and
Mathias’ polynomial teachability model, and the model for learning from example based queries is also learnable
under the PACS model (Parekh and Honavar, 2000; 2001).
Multi-Agent Systems for Integrated Host and Network Based Intrusion Detection (in collaboration with Johnny Wong
and Les Miller, funded in part by a Department of Defense grant)
This research was aimed at the development of approaches for monitoring complex Distributed Systems (e.g., computer
systems, communication networks, power systems) for coordinated attacks using information from multiple are equipped
with sensors and measurement devices. Both host and network-based approaches were investigated as part of this
research. Results of this research include:
(a) New tools for formal specification of intrusions using colored Petri nets and software fault trees (Helmer et al., 2002)
(b) Design and implementation of a multi-agent system for detection of coordinated or concerted attacks on distributed
computing systems in particular by monitoring different processes, resources, users, events, and extract and
integrate relevant information from disparate sources over multiple space and time scales (Wong et al., 2001; Helmer
et al., 2003; Wang et al., 2005)
(c) Development of data mining approaches for learning predictive rules for anomaly and misuse detection (Helmer et
al., 2002).
Agent-Based Approaches to Cooperative Traffic Management in Large Communication Networks (in collaboration
with Johnny Wong and Armin Mikler)
With the unprecedented growth in size and complexity of modern communication networks, the development of intelligent
and adaptive approaches to system management (including such functions as routing, congestion control, traffic and load
management, etc.) present several research challenges. Routing in a communication network refers to the task of
Dec 2005
12