Planning under Uncertainty in Dynamic Domains - Carnegie Mellon ...

3.2. A representation for planning under uncertainty 25the choice to move a particular step from the tail plan to the head plan. As analternative tomoving a step to the head plan, a goal node represents the decision tofocus on an open condition in the tail plan (also known as a goal). The children of agoal node must all be operator nodes, which represent the decision to use a particularoperator schema to achieve the goal represented by the parent node. The children ofan operator node must all be bindings nodes, which represent the decision to use aparticular set of bindings to instantiate the operator represented by the parent node.The combination of a goal node, an operator node and a bindings node togetherrepresent the act of prodigy 4.0 adding a step into the tail plan.While the tail plan and the head plan together represent one candidate plan, thesearch tree represents all the candidate plans that have been examined and also allthe ways that new candidate plans can be expanded. A path from the root node toany other node in the search tree corresponds to a candidate partial plan, which canbe reconstructed by following the decisions encoded in the nodes in the path. Forexample, the candidate plan in Figure 3.4 is actually generated by prodigy 4.0 asthe sequence of search tree nodes shown in Table 3.2. In this sequence, each nodeisthe child of the node in the line above 3 , so the candidate plan is produced withoutbacktracking over any choices. Other sequences of search nodes could produce thecandidate plan just as well: in particular the order in which prodigy 4.0 works onthe goals (oil-in-barge barge1) and (at barge1 Richmond) doesn't matter. Thechoice of this particular sequence was largely due to search heuristics that prodigy4.0 uses, which will not be discussed here (but see [Blythe & Veloso 1992] and[Carbonell et al. 1992]).3.2 A representation for planning under uncertaintyIn Section 3.1 I provided a brief description of planning domains, planning problemsand plans in prodigy 4.0. Here I extend those denitions to the versions usedby Weaver, in which planning domains and problems also contain information aboutuncertainty in the domain, and plans can specify a number of contingencies. Inaddition to operators, which can have more than one possible outcome, planningdomains in Weaver also contain exogenous events, which specify ways that the worldcan be changed independently of the actions in a plan.In order to give a precise characterisation of the problems that Weaver can representand solve, I describe Weaver's representation scheme in terms of Markov decisionprocesses. Specically, I describe how to take a planning problem in Weaver's languageand construct a Markov decision process M which is equivalent in the sense thatif there is a non-looping policy for M with an expected value greater than 0 then there3 except n5 which is the child of a special node that prodigy 4.0 uses to add the top-level goalsto its list of goals.