Planning under Uncertainty in Dynamic Domains - Carnegie Mellon ...

2.4. Approaches based on Markov decision processes 13action in its plan to achieveany goal or subgoal. This is necessary since each individualaction may fail to achieve the goal with some probability. Buridan can provably nda plan that achieves the threshold probability of success, if a non-branching planexists that does so. C-Buridan is an extension to Buridan that can create branchingplans, using a technique similar to that introduced with cnlp [Peot & Smith 1992].Essentially, one new way that C-Buridan can x a candidate plan in whichtwo actionsmight interfere with each other is to add a test and restrict the actions to dierentconditional branches based on the test. Information about the branch that an actionbelongs to is then propagated to the action's descendants and ancestors in the goaltree.Cassandra [Pryor & Collins 1993; 1996] is another planner based on snlp anduses an explicit representation of decision steps. Bagchi et al. describe a plannerthat uses a spreading activation technique to choose actions with highest expectedutility [Bagchi, Biswas, & Kawamura 1994]. Onder and Pollack also extend snlpwith explicit reasoning about the contingencies to plan for [Onder & Pollack 1997].Haddawy's thesis work [Haddawy 1991] introduces a logic with extensions to representprobabilities of terms, intended for representing plans. He and his group havedeveloped hierarchical task-network (htn) planners that use a similar representation[Haddawy & Suwandi 1994; Haddawy, Doan, & Goodwin 1995; Haddawy, Doan, &Kahn 1996].While this thesis concentrates mainly on extensions to classical planning thatsupport probabilistic planning in uncertain domains, other work develops extensionsto enable planners to nd high-quality plans according to some specied criteria.Perez [Perez & Carbonell 1994; Perez 1995] describes how to learn control knowledgefrom interactions with a human expert that allows Prodigy to produce higher-qualityplans. Williamson and Hanks [Williamson & Hanks 1994; Williamson 1996] developa decision-theoretic extension to ucpop.The planner described in this thesis is unique in its ability to reason explicitlyabout exogenous events, and to perform a relevance analysis of those events. Vere[Vere 1983] introduced a planner that could reason about exogenous events that wereknown to occur at some xed time, but not about uncertain events.2.4 Approaches based on Markov decision processesIn the past ve years, much of the research activity in AI planning under uncertaintyhas built on standard techniques for solving Markov decision problems (mdps) andpartially-observable Markov decision problems pomdps [Dean et al. 1995; Littman1996; Boutilier, Brafman, & Geib 1997]. This formalism has the advantage of a soundmathematical underpinning, and the standard solution techniques aim at an optimalpolicy while most systems based on classical planning try to meet a lower bound.However these standard techniques are not tractable since they require enumerating