{"type":"ScholarlyArticle","title":"Adaptive Elicitation of Preferences under Uncertainty in Sequential Decision Making Problems","authors":[{"name":"Nawal Benabbou","slug":"nawal-benabbou","given":"Nawal","family":"Benabbou","affiliation":"Sorbonne Universités , UPMC Univ Paris 06 , UMR 7606 , LIP6 CNRS , UMR 7606 , LIP6 , F-75005 , Paris , France"},{"name":"Patrice Perny","slug":"patrice-perny","given":"Patrice","family":"Perny","affiliation":"Sorbonne Universités , UPMC Univ Paris 06 , UMR 7606 , LIP6 CNRS , UMR 7606 , LIP6 , F-75005 , Paris , France; Place Jussieu , 75005 Paris , France"}],"abstract":"This paper aims to introduce an adaptive preference elicitation method for interactive decision support in sequential decision problems. The Decision Maker's preferences are assumed to be representable by an additive utility, initially unknown or imperfectly known. We first study the determination of possibly optimal policies when admissible utilities are imprecisely defined by some linear constraints derived from observed preferences. Then, we introduce a new approach interleaving elicitation of utilities and backward induction to incrementally determine an optimal or near-optimal policy. We propose an interactive algorithm with performance guarantees and describe numerical experiments demonstrating the practical efficiency of our approach.","doi":"10.24963/ijcai.2017/637","url":"https://ijcai.org/proceedings/2017/637","pdf":"https://ijcai.org/proceedings/2017/0637.pdf","markdown":"https://ijcai.org/proceedings/2017/637.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/637","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Uncertainty in AI","track":"Main track","number":637,"pages":{"first":4566,"last":4572},"keywords":["Uncertainty in AI: Sequential Decision Making","Uncertainty in AI: Decision/Utility Theory","Uncertainty in AI: Uncertainty in AI"],"videos":[],"references":[{"position":1,"raw":"[Aziz et al., 2015] Haris Aziz, Markus Brill, Felix Fischer, Paul Harrenstein, Jérôme Lang, and Hans Georg Seedig. Possible and necessary winners of partial tournaments. J. Artif. Intell. Res. (JAIR), 54:493-534, 2015.","title":"Possible and necessary winners of partial tournaments","authors":"Haris Aziz, Markus Brill, Felix Fischer, Paul Harrenstein, Jérôme Lang, Hans Georg, Seedig","year":"2015","venue":"J. Artif. Intell. Res. (JAIR)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Benabbou and Perny, 2015a] Nawal Benabbou and Patrice Perny. Combining preference elicitation and search in multiobjective state-space graphs. In IJCAI, pages 297- 303, 2015.","title":"Nawal Benabbou and Patrice Perny. Combining preference elicitation and search in multiobjective state-space graphs","authors":"Perny Benabbou","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Benabbou and Perny, 2015b] Nawal Benabbou and Patrice Perny. Incremental weight elicitation for multiobjective state space search. In AAAI, pages 1093-1099, 2015.","title":"Nawal Benabbou and Patrice Perny. Incremental weight elicitation for multiobjective state space search","authors":"Perny Benabbou","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boutilier et al., 2004] Craig Boutilier, Ronen I Brafman, Carmel Domshlak, Holger H Hoos, and David Poole. Preference-based constrained optimization with CP-nets. Computational Intelligence, 20(2):137-157, 2004.","title":"Preference-based constrained optimization with CP-nets","authors":"Craig Boutilier, Ronen I Brafman, Carmel Domshlak, David Holger H Hoos, Poole","year":"2004","venue":"Computational Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Boutilier et al., 2006] Craig Boutilier, Relu Patrascu, Pas- cal Poupart, and Dale Schuurmans. Constraint-based opti- mization and utility elicitation using the minimax decision criterion. Artifical Intelligence, 170(8-9):686-713, 2006.","title":"Constraint-based optimization and utility elicitation using the minimax decision criterion","authors":"Craig Boutilier, Relu Patrascu, Pascal Poupart, Dale Schuurmans","year":"2006","venue":"Artifical Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Boutilier, 2013] Craig Boutilier. Computational decision support: regret-based models for optimization and prefer- ence elicitation. P. H. Crowley, T. R. Zentall (Eds.), Com- parative Decision Making: Analysis and Support Across Disciplines and Applications, pages 423-453, 2013.","title":"Computational decision support: regret-based models for optimization and preference elicitation","authors":"Craig Boutilier","year":"2013","venue":"Comparative Decision Making: Analysis and Support Across Disciplines and Applications","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Brafman et al., 2010] Ronen I Brafman, Francesca Rossi, Domenico Salvagnin, Kristen Brent Venable, and Toby Walsh. Finding the next solution in constraint-and preference-based knowledge representation formalisms.","title":"Finding the next solution in constraint-and preference-based knowledge representation formalisms","authors":"Francesca Ronen I Brafman, Domenico Rossi, Kristen Brent Salvagnin, Toby Venable, Walsh","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Braziunas and Boutilier, 2007] Darius Braziunas and Craig Boutilier. Minimax regret based elicitation of generalized additive utilities. In UAI, pages 25-32, 2007.","title":"Minimax regret based elicitation of generalized additive utilities","authors":"Boutilier Braziunas, Darius Braziunas, Craig Boutilier","year":"2007","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chajewska et al., 2000] Urszula Chajewska, Daphne Koller, and Ronald Parr. Making rational decisions using adaptive utility elicitation. In AAAI, pages 363-369, 2000.","title":"Making rational decisions using adaptive utility elicitation","authors":"Urszula Chajewska, Daphne Koller, Ronald Parr","year":"2000","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chajewska et al., 2001] Urszula Chajewska, Daphne Koller, and Dirk Ormoneit. Learning an agent's utility function by observing behavior. In ICML, pages 35-42, 2001.","title":"Learning an agent's utility function by observing behavior","authors":"Urszula Chajewska, Daphne Koller, Dirk Ormoneit","year":"2001","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dasgupta et al., 1995] Pallab Dasgupta, PP Chakrabarti, and SC DeSarkar. Utility of pathmax in partial order heuristic search. J. of algorithms, 55:317-322, 1995.","title":"Utility of pathmax in partial order heuristic search","authors":"Pallab Dasgupta, Chakrabarti, Desarkar","year":"1995","venue":"J. of algorithms","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Drummond and Boutilier, 2013] Joanna Drummond and Craig Boutilier. Elicitation and approximately stable matching with partial preferences. In IJCAI, 2013.","title":"Elicitation and approximately stable matching with partial preferences","authors":"Joanna Drummond, Craig Boutilier","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/26"},{"position":13,"raw":"[Fürnkranz and Hüllermeier, 2010] Johannes Fürnkranz and Eyke Hüllermeier. Preference learning: An introduction. Springer, 2010.","title":"Johannes Fürnkranz and Eyke Hüllermeier. Preference learning: An introduction","authors":"Hüllermeier Fürnkranz","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gelain et al., 2010] Mirco Gelain, Maria Silvia Pini, Francesca Rossi, K Brent Venable, and Toby Walsh. Elic- itation strategies for soft constraint problems with missing preferences: Properties, algorithms and experimental studies. Artificial Intelligence, 174(3):270-294, 2010.","title":"Elicitation strategies for soft constraint problems with missing preferences: Properties, algorithms and experimental studies","authors":"Mirco Gelain, Maria Silvia Pini, Francesca Rossi, Brent Venable, Toby Walsh","year":"2010","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ha and Haddawy, 1997] Vu Ha and Peter Haddawy. Problem-focused incremental elicitation of multi-attribute utility models. In UAI, pages 215-222, 1997.","title":"Problem-focused incremental elicitation of multi-attribute utility models","authors":"Haddawy ; ] Ha, Ha Vu, Peter Haddawy","year":"1997","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hammond, 1988] Peter J. Hammond. Consequentialist foundations for expected utility. Theory and decision, 25(1):25-78, 1988.","title":"Consequentialist foundations for expected utility","authors":"J Peter, Hammond","year":"1988","venue":"Theory and decision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hines and Larson, 2010] Greg Hines and Kate Larson. Pref- erence elicitation for risky prospects. In AAMAS, pages 889-896, 2010.","title":"Preference elicitation for risky prospects","authors":"Larson Hines, Greg Hines, Kate Larson","year":"2010","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kikuti et al., 2011] Daniel Kikuti, Fabio Gagliardi Cozman, and Ricardo Shirota Filho. Sequential decision making with partially ordered preferences. Artificial Intelligence, 175(7):1346-1365, 2011.","title":"Sequential decision making with partially ordered preferences","authors":"Daniel Kikuti, Fabio Gagliardi Cozman, Ricardo Shirota Filho","year":"2011","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Konczak and Lang, 2005] Kathrin Konczak and Jérôme Lang. Voting procedures with incomplete preferences. In MPREF, 2005.","title":"Voting procedures with incomplete preferences","authors":"Kathrin Konczak, Jérôme Lang","year":"2005","venue":"MPREF","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Marinescu et al., 2013] Radu Marinescu, Abdul Razak, and Nic Wilson. Multi-objective constraint optimization with tradeoffs. In Int. Conf. on Principles and Practice of Con- straint Programming, pages 497-512. Springer, 2013.","title":"Multi-objective constraint optimization with tradeoffs","authors":"Radu Marinescu, Abdul Razak, Nic Wilson","year":"2013","venue":"Int. Conf. on Principles and Practice of Constraint Programming","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Perny and Spanjaard, 2002] Patrice Perny and Olivier Span- jaard. On preference-based search in state space graphs. In AAAI, pages 751-756, 2002.","title":"Patrice Perny and Olivier Spanjaard. On preference-based search in state space graphs","authors":"Spanjaard Perny","year":"2002","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Perny et al., 2016] Patrice Perny, Paolo Viappiani, and Ab- dellah Boukhatem. Incremental Preference Elicitation for Decision Making Under Risk with the Rank-Dependent Utility Model. In UAI, pages 597-606, 2016.","title":"Incremental Preference Elicitation for Decision Making Under Risk with the Rank-Dependent Utility Model","authors":"Patrice Perny, Paolo Viappiani, Abdellah Boukhatem","year":"2016","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ramsay, 1988] James O. Ramsay. Monotone regression spline in action. Statistical Science, page 425441, 1988.","title":"Monotone regression spline in action","authors":"James O Ramsay","year":"1988","venue":"Statistical Science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Regan and Boutilier, 2009] Kevin Regan and Craig Boutilier. Regret-based reward elicitation for Markov decision processes. In UAI, pages 444-451, 2009.","title":"Regret-based reward elicitation for Markov decision processes","authors":"Kevin Regan, Craig Boutilier","year":"2009","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Roijers et al., 2014] Diederik M Roijers, Shimon Whiteson, and Frans A Oliehoek. Linear support for multi-objective coordination graphs. In AAMAS, pages 1297-1304, 2014.","title":"Linear support for multi-objective coordination graphs","authors":"Shimon Diederik M Roijers, Frans A Whiteson, Oliehoek","year":"2014","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Von Neumann and Morgenstern, 1947] John Von Neumann and Oskar Morgenstern. Theory of games and economic behavior. 2nd Ed. Princeton University Press, 1947.","title":"John Von Neumann and Oskar Morgenstern","authors":"Von Neumann, Morgenstern","year":"1947","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang and Boutilier, 2003] Tianhan Wang and Craig Boutilier. Incremental utility elicitation with the minimax regret decision criterion. In IJCAI, pages 309-316, 2003.","title":"Incremental utility elicitation with the minimax regret decision criterion","authors":"Wang, Boutilier, Tianhan Wang, Craig Boutilier","year":"2003","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2003/46"},{"position":28,"raw":"[Weng and Zanuttini, 2013] Paul Weng and Bruno Zanuttini. Interactive value iteration for markov decision processes with unknown rewards. In IJCAI, pages 2415-2421, 2013.","title":"Interactive value iteration for markov decision processes with unknown rewards","authors":"Paul Weng, Bruno Zanuttini","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/355"},{"position":29,"raw":"[White et al., 1984] Chelsea C White, Andrew P Sage, and Shigeru Dozono. A model of multiattribute decision mak- ing and trade-off weight determination under uncertainty. IEEE Transactions on Systems, Man, and Cybernetics, 14(2):223-229, 1984.","title":"A model of multiattribute decision making and trade-off weight determination under uncertainty","authors":"Chelsea C White, Andrew P Sage, Shigeru Dozono","year":"1984","venue":"IEEE Transactions on Systems, Man, and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xia and Conitzer, 2008] Lirong Xia and Vincent Conitzer. Determining possible and necessary winners under com- mon voting rules given partial orders. In AAAI, pages 196- 201, 2008.","title":"Determining possible and necessary winners under common voting rules given partial orders","authors":"Conitzer Xia, Lirong Xia, Vincent Conitzer","year":"2008","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}