decision making with multiple imperfect decision makers - Institute of ...

More documents

Recommendations

Info

Automated Preferences ElicitationMiroslav KárnýDepartment of Adaptive SystemsInstitute of Information Theory and AutomationAcademy of Sciences of the Czech RepublicPod vodárenskou věží 4, 182 08 Prague 8, Czech Republicschool@utia.cas.czTatiana V. GuyDepartment of Adaptive SystemsInstitute of Information Theory and AutomationAcademy of Sciences of the Czech RepublicPod vodárenskou věží 4, 182 08 Prague 8, Czech Republicguy@utia.cas.czAbstractSystems supporting decision making became almost inevitable in the moderncomplex world. Their efficiency depends on the sophisticated interfaces enablinga user take advantage of the support while respecting the increasing on-line informationand incomplete, dynamically changing user’s preferences. The best decisionmaking support is useless without the proper preference elicitation. The paperproposes a methodology supporting automatic learning of quantitative descriptionof preferences. The proposed elicitation serves to fully probabilistic design, whichis an extension of Bayesian decision making.1 IntroductionA feasible and effective solution of preference elicitation problem decides on the efficiency of anyintelligent system supporting decision making. Indeed, to recommend a participant 1 an optimal sequenceof optimal decisions requires knowing some information about what the participant (affectedby the recommended decision, if accepted) considers as “optimal”. Extracting the information aboutthe participant’s preferences or utility is known as preference elicitation or utility elicitation 2 . Thisvital problem has been repeatedly addressed within artificial intelligence, game theory, operationresearch and many sophisticated approaches have been proposed [7], [8], [6], [5]. A number ofapproaches has arisen in connection with applied sciences like economy, social science, clinicaldecision making, transportation, see, for instance, [18], [9]. To ensure feasibility and practical applicability,many decision support systems have been designed under various assumptions on thestructure of preferences. In particular, a broadly accepted additive independence [16] of values ofindividual attributes is not generally valid. In many applications the preferences of attributes aremostly dependent and the assumption above significantly worsens the elicitation results 3 .To benefit from any decision support, the preferences should be known in the form allowing theirprocessing by an intended decision support system. Unless the participant’s preferences are completelyprovided by the participant, they should be learned from either past data or domain-specific1 Participant is also known as user, decision maker, agent.2 The term utility generally has a bit different coverage within decision-making context.3 The assumption can be weakened by a introducing a conditional preferential independence, [4].1
information (technological knowledge, physical laws, etc.). Eliciting the needed information itself isinherently hard task, which success depends on experience and skills of an elicitation expert. Preferencescan be elicited from past data directly collected on the underlying decision-making process orfrom indirect data learned from a number of similar situations. Despite acquiring the probabilisticinformation from data is well-elaborated, learning can be hard, especially when the space of possiblebehaviour is larger than that past data cover. Then the initial preferences for the remaining part ofthe behaviour should be properly assigned.The process of eliciting of the domain-specific information is difficult, time-consuming and errorpronetask 4 . Domain experts provide subjective opinions, typically expressed in different and incompatibleforms. The elicitation expert should elaborate these opinions into a distribution describingpreferences in a consistent way. Significant difficulties emerge when competitive/complementingopinions with respect to the same collection of attributes should be merged. A proper merging theirindividual opinions within a high-dimensional space of possible behaviour is unfeasible. Besidesdomain experts having domain-specific information are often unable to provide their opinion on apart of behaviour due to either limited knowledge of the phenomenon behind or the indifferencetowards the possible instances of behaviour. Then, similarly to the learning preferences from pastdata, the optimal strategy heavily depends on the initial preferences assigned to the part of behaviournot “covered” by the domain-specific information.Process of eliciting information itself requires significant cognitive and computational effort of theelicitation expert. Even if we neglect the cost of this effort 5 , the elicitation result is always verylimited by the expert’s imperfection, i.e. his inability to devote an infinite deliberation effort toeliciting. Unlike imperfection of experts providing domain-specific information, imperfection ofelicitation experts can be eliminated. This motivate the search for a feasible automated support ofpreference elicitation, that does not rely on any elicitation expert.The dynamic decision making strengthes the dependence on the preference elicitation. Indeed, theparticipant acting within a dynamically changing environment with evolving parameters may graduallychange preferences. The intended change may depend on the future behaviour. The overalltask is going harder when participant interacts with other dynamic imperfect participants within acommon environment.The paper concerns a construction of probabilistic description of preferences based on the informationavailable. Dynamic decision making under uncertainty from the perspective of an imperfectparticipant is considered. The participant solves DM task with respect to its environment and basedon a given finite set of opinions gained from providers of domain expertise or learned from the pastdata or both. The set indirectly represents the preferences in a non-unique way 6 . Additionally, theparticipant may be still uncertain about the preferences on a non-empty subset of behaviour. To designan optimal strategy, a participant employs Fully Probabilistic Design (FPD) of DM strategies,[10, 12] whose specification relies on the notion of an ideal closed-loop model which is essentiallya probabilistic description of the preferences. In other words, an ideal closed-loop model describesthe closed-loop behaviour, when the participant’s DM strategy is optimal. FPD searches for the optimalstrategy by minimising the divergence of the current closed-loop description on the ideal one.Adopted FPD implies availability of probabilistic description of the environment and probabilisticdescription of the past closed-loop data.Section 2 specifies assumptions under which the automated preference elicitation is proposed withinthe considered FPD. Section 3 describes construction of the ideal closed-loop distribution based onthe information provided. The proposed solution is discussed in Section 4 followed by the concludingremarks in Section 5.4 It should be mentioned that practical solutions mostly use a laborious and unreliable process of manualtuning a number of parameters of the pre-selected utility function. Sometimes the high number of parametersmakes this solution unfeasible. Then there are attempts to decrease the number of parameters to reach anacceptable feasibility level.5 This effort is usually very high and many sophisticated approaches aim at optimising a trade-off betweenelicitation cost and value of information it provides (often decision quality is considered), see for instance [3].6 Even, when we identify instances of behaviour that cannot be distinguished from the preferences’ viewpoint.2
Page 1 and 2: The 2nd International Workshop onDE
Page 3 and 4: The 2nd International Workshop onDE
Page 5 and 6: Time Title Authors7:30—7:50 Openi
Page 7 and 8: Modeling Humans as Reinforcement Le
Page 9 and 10: solution concept used to predict th
Page 11 and 12: an ɛ-greedy policy parameterizatio
Page 13 and 14: Automated Explanations for MDP Poli
Page 15 and 16: V E = ∑ i∈Eλ π∗s 0(sc i )
Page 17 and 18: Figure 1: User Perception of MDP-Ba
Page 19: [9] Warren B. Powell. Approximate D
Page 23 and 24: The complete fulfilment of preferen
Page 25 and 26: References[1] J.O. Berger. Statisti
Page 27 and 28: (happiness, anger, fear, disgust, s
Page 30 and 31: David H Wolpert, editors, Decision
Page 32 and 33: on some of their concepts, we devel
Page 34 and 35: We assume that they are evaluated t
Page 36 and 37: AcknowledgmentsResearch supported b
Page 38 and 39: Bayesian Combination of Multiple, I
Page 40 and 41: is not in closed form [5], requirin
Page 42 and 43: where α (k)j is updated by adding
Page 44 and 45: Figure 3: Prototypical confusion ma
Page 46 and 47: Artificial Intelligence Designfor R
Page 48 and 49: the game. Each has its own attribut
Page 50 and 51: 4.3 Experimental Results and Limita
Page 52 and 53: Distributed Decision Making byCateg
Page 54 and 55: q 1a 1 (1) b 1(1)a 2(1)b 2(1)a K(1)
Page 56 and 57: Bayes risk0.250.20.150.10.05Collabo
Page 58 and 59: Decision making and working memory
Page 60 and 61: in WM as well as inhibitory tasks [
Page 62 and 63: Difficulty level will be automatica
Page 64 and 65: Overall, the current literature lea
Page 66 and 67: [31] W K Bickel, R Yi, R D Landes,
Page 68 and 69: Each time instant, the agents first
Page 70 and 71:
Figure 2: Two basic decentralized a
Page 72 and 73:
[11] M. Kárný and T.V. Guy. Shari
Page 74 and 75:
these results yields a design metho
Page 76 and 77:
Minimum of (16) is well known from
Page 78 and 79:
Algorithm 2 VB-DP variant of the di
Page 80 and 81:
Further simplications can be achiev
Page 82 and 83:
The basic terms we use are as follo
Page 84 and 85:
3 Connection to the Bayesian soluti
Page 86 and 87:
4 ConclusionThis paper brings an im
Page 88 and 89:
2 Dynamic programming and revisions
Page 90 and 91:
The possible way how to recognize t
Page 92 and 93:
The data used for experiment are da
Page 95:
T.V. Guy, Institute of Information
show all

decision making with multiple imperfect decision makers - Institute of ...

Create successful ePaper yourself

Delete template?

Save as template?