{"type":"ScholarlyArticle","title":"Multi-Agent Planning with Baseline Regret Minimization","authors":[{"name":"Feng Wu","slug":"feng-wu","given":"Feng","family":"Wu","affiliation":"† School of Computer Science and Technology , University of Science and Technology of China , CHN"},{"name":"Shlomo Zilberstein","slug":"shlomo-zilberstein","given":"Shlomo","family":"Zilberstein","affiliation":"‡ College of Information and Computer Sciences , University of Massachusetts Amherst , USA"},{"name":"Xiaoping Chen","slug":"xiaoping-chen","given":"Xiaoping","family":"Chen","affiliation":"† School of Computer Science and Technology , University of Science and Technology of China , CHN"}],"abstract":"We propose a novel baseline regret minimization algorithm for multi-agent planning problems modeled as finite-horizon decentralized POMDPs. It guarantees to produce a policy that is provably better than or at least equivalent to the baseline policy. We also propose an iterative belief generation algorithm to effectively and efficiently minimize the baseline regret, which only requires necessary iterations to converge to the policy with minimum baseline regret. Experimental results on common benchmark problems confirm its advantage comparing to the state-of-the-art approaches.","doi":"10.24963/ijcai.2017/63","url":"https://ijcai.org/proceedings/2017/63","pdf":"https://ijcai.org/proceedings/2017/0063.pdf","markdown":"https://ijcai.org/proceedings/2017/63.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/63","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Agent-based and Multi-agent Systems","track":"Main track","number":63,"pages":{"first":444,"last":450},"keywords":["Planning and Scheduling: Planning under Uncertainty","Agent-based and Multi-agent Systems: Coordination and cooperation","Planning and Scheduling: Distributed/Multi-agent Planning","Agent-based and Multi-agent Systems: Multi-agent Planning"],"videos":[],"references":[{"position":1,"raw":"[Ahmed et al., 2013] Asrar Ahmed, Pradeep Varakantham, Yossiri Adulyasak, and Patrick Jaillet. Regret based ro- bust solutions for uncertain Markov decision processes. In Proc. of NIPS, pages 881-889, 2013.","title":"Regret based robust solutions for uncertain Markov decision processes","authors":"Asrar Ahmed, Pradeep Varakantham, Yossiri Adulyasak, Patrick Jaillet","year":"2013","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amato et al., 2009] Chistopher Amato, Jilles S. Dibangoye, and Shlomo Zilberstein. Incremental policy generation for finite-horizon DEC-POMDPs. In Proc. of ICAPS, pages 2-9, 2009.","title":"Incremental policy generation for finite-horizon DEC-POMDPs","authors":"Chistopher Amato, Jilles S Dibangoye, Shlomo Zilberstein","year":"2009","venue":"Proc. of ICAPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Amato et al., 2014] Christopher Amato, George D. Konida- ris, and Leslie P. Kaelbling. Planning with macro-actions in decentralized POMDPs. In Proc. of AAMAS, pages 1273-1280, 2014.","title":"Planning with macro-actions in decentralized POMDPs","authors":"Christopher Amato, George D Konidaris, Leslie P Kaelbling","year":"2014","venue":"Proc. of AAMAS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Aras and Dutech, 2010] Raghav Aras and Alain Dutech. An investigation into mathematical programming for finite ho- rizon decentralized POMDPs. Journal of Artificial Intelli- gence Research, 37(1):329-396, 2010.","title":"An investigation into mathematical programming for finite horizon decentralized POMDPs","authors":"Raghav Aras, Alain Dutech","year":"2010","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Banerjee and Kraemer, 2013] Bikramjit Banerjee and Landon Kraemer. Counterfactual regret minimization for decentralized planning. In Workshop on Multiagent Sequential Decision Making, pages 32-39, 2013.","title":"Counterfactual regret minimization for decentralized planning","authors":"Bikramjit Banerjee, Landon Kraemer","year":"2013","venue":"Workshop on Multiagent Sequential Decision Making","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bernstein et al., 2002] Daniel S. Bernstein, Robert Givan, Neil Immerman, and Shlomo Zilberstein. The complex- ity of decentralized control of Markov decision processes. Math. of Operations Research, 27(4):819-840, 2002.","title":"The complexity of decentralized control of Markov decision processes","authors":"S Daniel, Robert Bernstein, Neil Givan, Shlomo Immerman, Zilberstein","year":"2002","venue":"Math. of Operations Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dibangoye et al., 2016] Jilles S. Dibangoye, Christopher Amato, Olivier Buffet, and Franc ¸ois Charpillet. Optimally solving Dec-POMDPs as continuous-state MDPs. Journal of Artificial Intelligence Research, 55:443-497, 2016.","title":"Optimally solving Dec-POMDPs as continuous-state MDPs","authors":"S Jilles, Christopher Dibangoye, Olivier Amato, Franc Buffet, Charpillet ¸ois","year":"2016","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hansen et al., 2004] Eric A. Hansen, Daniel S. Bernstein, and Shlomo Zilberstein. Dynamic programming for parti- ally observable stochastic games. In Proc. of AAAI, pages 709-715, 2004.","title":"Dynamic programming for partially observable stochastic games","authors":"Eric A Hansen, Daniel S Bernstein, Shlomo Zilberstein","year":"2004","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kumar and Zilberstein, 2010] Akshat Kumar and Shlomo Zilberstein. Point-based backup for decentralized POMDPs: Complexity and new algorithms. In Proc. of AAMAS, pages 1315-1322, 2010.","title":"Point-based backup for decentralized POMDPs: Complexity and new algorithms","authors":"Akshat Kumar, Shlomo Zilberstein","year":"2010","venue":"Proc. of AAMAS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kumar et al., 2016] Akshat Kumar, Hala Mostafa, and Shlomo Zilberstein. Dual formulations for optimizing Dec-POMDP controllers. In Proc. of ICAPS, pages 202- 210, 2016.","title":"Dual formulations for optimizing Dec-POMDP controllers","authors":"Akshat Kumar, Hala Mostafa, Shlomo Zilberstein","year":"2016","venue":"Proc. of ICAPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Nair et al., 2003] Ranjit Nair, Milind Tambe, Makoto Yo- koo, David Pynadath, and Stacy Marsella. Taming decen- tralized POMDPs: Towards efficient policy computation for multiagent settings. In Proc. of IJCAI, pages 705-711, 2003.","title":"Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings","authors":"Ranjit Nair, Milind Tambe, Makoto Yokoo, David Pynadath, Stacy Marsella","year":"2003","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2003/103"},{"position":12,"raw":"[Oliehoek et al., 2013] Frans A. Oliehoek, Matthijs T. J. Spaan, Christopher Amato, and Shimon Whiteson. Incre- mental clustering and expansion for faster optimal plan- ning in Dec-POMDPs. Journal of Artificial Intelligence Research, 46:449-509, 2013.","title":"Incremental clustering and expansion for faster optimal planning in Dec-POMDPs","authors":"Frans A Oliehoek, T J Matthijs, Christopher Spaan, Shimon Amato, Whiteson","year":"2013","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Oliehoek, 2013] Frans A. Oliehoek. Sufficient plan-time statistics for decentralized POMDPs. In Proc. of IJCAI, pages 302-308, 2013.","title":"Sufficient plan-time statistics for decentralized POMDPs","authors":"Frans A Oliehoek","year":"2013","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/53"},{"position":14,"raw":"[Pajarinen and Peltonen, 2011] Joni K. Pajarinen and Jaakko Peltonen. Periodic finite state controllers for efficient POMDP and DEC-POMDP planning. In Proc. of NIPS, pages 2636-2644, 2011.","title":"Periodic finite state controllers for efficient POMDP and DEC-POMDP planning","authors":"Peltonen ; ] Pajarinen, K Joni, Jaakko Pajarinen, Peltonen","year":"2011","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Petrik et al., 2016] Marek Petrik, Yinlam Chow, and Mo- hammad Ghavamzadeh. Safe policy improvement by mi- nimizing robust baseline regret. In Proc. of NIPS, pages 2298-2306, 2016.","title":"Safe policy improvement by minimizing robust baseline regret","authors":"Marek Petrik, Yinlam Chow, Mohammad Ghavamzadeh","year":"2016","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rabinovich et al., 2003] Zinovi Rabinovich, Claudia V. Goldman, and Jeffrey S. Rosenschein. The complexity of multiagent systems: The price of silence. In Proc. of AAMAS, pages 1102-1103, 2003.","title":"The complexity of multiagent systems: The price of silence","authors":"Rabinovich","year":"2003","venue":"Proc. of AAMAS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Regan and Boutilier, 2010] Kevin Regan and Craig Bouti- lier. Robust policy computation in reward-uncertain MDPs using nondominated policies. In Proc. of AAAI, pages 1127-1133, 2010.","title":"Robust policy computation in reward-uncertain MDPs using nondominated policies","authors":"Regan, Boutilier, Kevin Regan, Craig Boutilier","year":"2010","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Seuken and Zilberstein, 2007] Sven Seuken and Shlomo Zilberstein. Memory-bounded dynamic programming for DEC-POMDPs. In Proc. of IJCAI, pages 2009-2015, 2007.","title":"Memory-bounded dynamic programming for DEC-POMDPs","authors":"Zilberstein Seuken, Sven Seuken, Shlomo Zilberstein","year":"2007","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/324"},{"position":19,"raw":"[Szer et al., 2005] Daniel Szer, Franc ¸ois Charpillet, and Shlomo Zilberstein. MAA*: A heuristic search algorithm for solving decentralized POMDPs. In Proc. of UAI, pages 576-583, 2005.","title":"MAA*: A heuristic search algorithm for solving decentralized POMDPs","authors":"Daniel Szer, Shlomo Franc ¸ois Charpillet, Zilberstein","year":"2005","venue":"Proc. of UAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu and Jennings, 2014] Feng Wu and Nicholas R. Jen- nings. Regret-based multi-agent coordination with uncer- tain task rewards. In Proc. of AAAI, pages 1492-1499, 2014.","title":"Regret-based multi-agent coordination with uncertain task rewards","authors":"Feng Wu, Nicholas R Jennings","year":"2014","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu et al., 2010a] Feng Wu, Shlomo Zilberstein, and Xiao- ping Chen. Point-based policy generation for decentra- lized POMDPs. In Proc. of AAMAS, pages 1307-1314, 2010.","title":"Point-based policy generation for decentralized POMDPs","authors":"Wu","year":"2010","venue":"Proc. of AAMAS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu et al., 2010b] Feng Wu, Shlomo Zilberstein, and Xiao- ping Chen. Rollout sampling policy iteration for decentra- lized POMDPs. In Proc of UAI, pages 666-673, 2010.","title":"Rollout sampling policy iteration for decentralized POMDPs","authors":"Wu","year":"2010","venue":"Proc of UAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2010c] Feng Wu, Shlomo Zilberstein, and Xiao- ping Chen. Trial-based dynamic programming for multi- agent planning. In Proc. of AAAI, pages 908-914, 2010.","title":"Trial-based dynamic programming for multiagent planning","authors":"Wu","year":"2010","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2011] Feng Wu, Shlomo Zilberstein, and Xia- oping Chen. Online planning for multi-agent systems with bounded communication. Artificial Intelligence, 175(2):487-511, 2011.","title":"Online planning for multi-agent systems with bounded communication","authors":"Feng Wu, Shlomo Zilberstein, Xiaoping Chen","year":"2011","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2012] Feng Wu, Nicholas R. Jennings, and Xiao- ping Chen. Sample-based policy iteration for constrained DEC-POMDPs. In Proc. of ECAI, pages 858-863, 2012.","title":"Sample-based policy iteration for constrained DEC-POMDPs","authors":"Feng Wu, Nicholas R Jennings, Xiaoping Chen","year":"2012","venue":"Proc. of ECAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2013] Feng Wu, Shlomo Zilberstein, and Nicho- las R. Jennings. Monte-carlo expectation maximization for decentralized POMDPs. In Proc. of IJCAI, pages 397-403, 2013.","title":"Monte-carlo expectation maximization for decentralized POMDPs","authors":"Feng Wu, Shlomo Zilberstein, Nicholas R Jennings","year":"2013","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/66"},{"position":27,"raw":"[Xu and Mannor, 2009] Huan Xu and Shie Mannor. Para- metric regret in uncertain Markov decision processes. In Proc. of CDC, pages 3606-3613, 2009.","title":"Parametric regret in uncertain Markov decision processes","authors":"Huan Xu, Shie Mannor","year":"2009","venue":"Proc. of CDC","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zinkevich et al., 2007] Martin Zinkevich, Michael Johan- son, Michael H. Bowling, and Carmelo Piccione. Regret minimization in games with incomplete information. In Proc. of NIPS, pages 1729-1736, 2007.","title":"Regret minimization in games with incomplete information","authors":"Martin Zinkevich, Michael Johanson, Michael H Bowling, Carmelo Piccione","year":"2007","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}