{"type":"ScholarlyArticle","title":"Integrating Decision Sharing with Prediction in Decentralized Planning for Multi-Agent Coordination under Uncertainty","authors":[{"name":"Minglong Li","slug":"minglong-li","given":"Minglong","family":"Li","affiliation":"State Key Laboratory of High Performance Computing , College of Computer , National University of Defense Technology , China"},{"name":"Wenjing Yang","slug":"wenjing-yang","given":"Wenjing","family":"Yang","affiliation":"State Key Laboratory of High Performance Computing , College of Computer , National University of Defense Technology , China"},{"name":"Zhongxuan Cai","slug":"zhongxuan-cai","given":"Zhongxuan","family":"Cai","affiliation":"State Key Laboratory of High Performance Computing , College of Computer , National University of Defense Technology , China"},{"name":"Shaowu Yang","slug":"shaowu-yang","given":"Shaowu","family":"Yang","affiliation":"State Key Laboratory of High Performance Computing , College of Computer , National University of Defense Technology , China"},{"name":"Ji Wang","slug":"ji-wang","given":"Ji","family":"Wang","affiliation":"State Key Laboratory of High Performance Computing , College of Computer , National University of Defense Technology , China"}],"abstract":"The performance of decentralized multi-agent systems tends to benefit from information sharing and its effective utilization. However, too much or unnecessary sharing may hinder the performance due to the delay, instability and additional overhead of communications. Aiming to a satisfiable coordination performance, one would prefer the cost of communications as less as possible. In this paper, we propose an approach for improving the sharing utilization by integrating information sharing with prediction in decentralized planning. We present a novel planning algorithm by combining decision sharing and prediction based on decentralized Monte Carlo Tree Search called Dec-MCTS-SP. Each agent grows a search tree guided by the rewards calculated by the joint actions, which can not only be sampled from the shared probability distributions over action sequences, but also be predicted by a sufficiently-accurate and computationally-cheap heuristics-based method. Besides, several policies including sparse and discounted UCT and DIY-bonus are leveraged for performance improvement. We have implemented Dec-MCTS-SP in the case study on multi-agent information gathering under threat and uncertainty, which is formulated as Decentralized Partially Observable Markov Decision Process (Dec-POMDP). The factored belief vectors are integrated into Dec-MCTS-SP to handle the uncertainty. Comparing with the random, auction-based algorithm and Dec-MCTS, the evaluation shows that Dec-MCTS-SP can reduce communication cost significantly while still achieving a surprisingly higher coordination performance.","doi":"10.24963/ijcai.2019/64","url":"https://ijcai.org/proceedings/2019/64","pdf":"https://ijcai.org/proceedings/2019/0064.pdf","markdown":"https://ijcai.org/proceedings/2019/64.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/64","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Agent-based and Multi-agent Systems","track":"Main track","number":64,"pages":{"first":450,"last":456},"keywords":["Agent-based and Multi-agent Systems: Multi-agent Planning","Agent-based and Multi-agent Systems: Coordination and Cooperation","Planning and Scheduling: Distributed;Multi-agent Planning"],"videos":[],"references":[{"position":1,"raw":"[Antoniades et al., 2003] Adonis Antoniades, H Jin Kim, and Shankar Sastry. Pursuit-evasion strategies for teams of multiple agents with incomplete information. In Inter- national Conference on Decision and Control, volume 1, pages 756-761. IEEE, 2003.","title":"Pursuit-evasion strategies for teams of multiple agents with incomplete information","authors":"Adonis Antoniades, Jin Kim, Shankar Sastry","year":"2003","venue":"International Conference on Decision and Control","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bernstein et al., 2002] Daniel S Bernstein, Robert Givan, Neil Immerman, and Shlomo Zilberstein. The complex- ity of decentralized control of Markov Decision Process- es. Mathematics of Operations Research, 27(4):819-840, 2002.","title":"The complexity of decentralized control of Markov Decision Processes","authors":"Robert Daniel S Bernstein, Neil Givan, Shlomo Immerman, Zilberstein","year":"2002","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Best et al., 2017] Graeme Best, Wolfram Martens, and Robert Fitch. Path planning with spatiotemporal optimal stopping for stochastic mission monitoring. IEEE Trans- actions on Robotics, 33(3):629-646, 2017.","title":"Path planning with spatiotemporal optimal stopping for stochastic mission monitoring","authors":"Graeme Best, Wolfram Martens, Robert Fitch","year":"2017","venue":"IEEE Transactions on Robotics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Best et al., 2018] Graeme Best, Michael Forrai, Ram- gopal R Mettu, and Robert Fitch. Planning-aware com- munication for decentralised multi-robot coordination. In International Conference on Robotics and Automation (I- CRA), pages 1050-1057. IEEE, 2018.","title":"Planning-aware communication for decentralised multi-robot coordination","authors":"Graeme Best, Michael Forrai, Robert Ramgopal R Mettu, Fitch","year":"2018","venue":"International Conference on Robotics and Automation (I-CRA)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Best et al., 2019] Graeme Best, Oliver M Cliff, Timothy Patten, Ramgopal R Mettu, and Robert Fitch. Dec-MCTS: Decentralized planning for multi-robot active perception. The International Journal of Robotics Research, 38(2- 3):316-337, 2019.","title":"Dec-MCTS: Decentralized planning for multi-robot active perception","authors":"Graeme Best, M Oliver, Timothy Cliff, Patten, Robert Mettu, Fitch","year":"2019","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Capitan et al., 2013] Jesus Capitan, Matthijs T J Spaan, Luis Merino, and Anibal Ollero. Decentralized multi- robot cooperation with auctioned POMDPs. The Interna- tional Journal of Robotics Research, 32(6):650-671, 2013.","title":"Decentralized multirobot cooperation with auctioned POMDPs","authors":"Capitan","year":"2013","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2016] Shaofei Chen, Feng Wu, Lincheng Shen, Jing Chen, and Sarvapali D Ramchurn. Decentralized patrolling under constraints in dynamic environments. IEEE Transactions on Systems, Man, and Cybernetics, 46(12):3364-3376, 2016.","title":"Decentralized patrolling under constraints in dynamic environments","authors":"Shaofei Chen, Feng Wu, Lincheng Shen, Jing Chen, Sarvapali D Ramchurn","year":"2016","venue":"IEEE Transactions on Systems, Man, and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Claes et al., 2017] Daniel Claes, Frans Oliehoek, Hendrik Baier, and Karl Tuyls. Decentralised online planning for multi-robot warehouse commissioning. In Proceedings of the 16th Conference on Autonomous Agents and MultiA- gent Systems, pages 492-500. International Foundation for Autonomous Agents and Multiagent Systems, 2017.","title":"Decentralised online planning for multi-robot warehouse commissioning","authors":"Daniel Claes, Frans Oliehoek, Hendrik Baier, Karl Tuyls","year":"2017","venue":"Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ghaffarkhah and Mostofi, 2011] Alireza Ghaffarkhah and Yasamin Mostofi. Communication-aware motion plan- ning in mobile networks. IEEE Transactions on Automatic Control, 56(10):2478-2485, 2011.","title":"Communication-aware motion planning in mobile networks","authors":"Alireza Ghaffarkhah, Yasamin Mostofi","year":"2011","venue":"IEEE Transactions on Automatic Control","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kartal et al., 2015] Bilal Kartal, Julio Godoy, Ioannis Karamouzas, and Stephen J Guy. Stochastic tree search with useful cycles for patrolling problems. In Internation- al Conference on Robotics and Automation (ICRA), pages 1289-1294. IEEE, 2015.","title":"Stochastic tree search with useful cycles for patrolling problems","authors":"Bilal Kartal, Julio Godoy, Ioannis Karamouzas, Stephen J Guy","year":"2015","venue":"International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kearns et al., 2002] Michael Kearns, Yishay Mansour, and Andrew Y Ng. A sparse sampling algorithm for near- optimal planning in large Markov decision processes. Ma- chine learning, 49(2-3):193-208, 2002.","title":"A sparse sampling algorithm for nearoptimal planning in large Markov decision processes","authors":"Michael Kearns, Yishay Mansour, Andrew Y Ng","year":"2002","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kocsis and Szepesvári, 2006] Levente Kocsis and Csaba Szepesvári. Bandit based Monte-Carlo planning. In Eu- ropean conference on machine learning, pages 282-293. Springer, 2006.","title":"Bandit based Monte-Carlo planning","authors":"Szepesvári Kocsis, Levente Kocsis, Csaba Szepesvári","year":"2006","venue":"European conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lindhe and Johansson, 2013] Magnus Lindhe and Kar- l Henrik Johansson. Exploiting multipath fading with a mobile robot. The International Journal of Robotics Re- search, 32(12):1363-1380, 2013.","title":"Exploiting multipath fading with a mobile robot","authors":"Johansson Lindhe, Magnus Lindhe, Karl Henrik, Johansson","year":"2013","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nijssen and Winands, 2012] Pim Nijssen and Mark H M Winands. Monte carlo tree search for the hide-and-seek game Scotland Yard. IEEE Transactions on Computation- al Intelligence and AI in Games, 4(4):282-294, 2012.","title":"Monte carlo tree search for the hide-and-seek game Scotland Yard","authors":"Pim Nijssen, H M Mark, Winands","year":"2012","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nunes and Gini, 2015] Ernesto Nunes and Maria Gini. Multi-robot auctions for allocation of tasks with temporal constraints. In Twenty-Ninth AAAI Conference on Artifi- cial Intelligence, pages 2110-2216, 2015.","title":"Multi-robot auctions for allocation of tasks with temporal constraints","authors":"Gini Nunes, Ernesto Nunes, Maria Gini","year":"2015","venue":"Twenty-Ninth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Oliehoek, 2018] Frans A Oliehoek. Interactive learning and decision making: Foundations, insights & challenges. In International Joint Conferences on Artificial Intelligence (IJCAI), pages 5703-5708, 2018.","title":"Interactive learning and decision making: Foundations, insights & challenges","authors":"A Frans, Oliehoek","year":"2018","venue":"International Joint Conferences on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2018/813"},{"position":17,"raw":"[Quigley et al., 2009] Morgan Quigley, Ken Conley, Bri- an Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y Ng. ROS: an open-source Robot Operating System. In ICRA workshop on open source soft- ware, volume 3, page 5. Kobe, Japan, 2009.","title":"ROS: an open-source Robot Operating System","authors":"Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, Andrew Y Ng","year":"2009","venue":"ICRA workshop on open source software","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Silver et al., 2016] D Silver, A. Huang, C. J. Maddison, A Guez, L Sifre, den Driessche G Van, J Schrittwieser, I Antonoglou, V Panneershelvam, and M Lanctot. Mas- tering the game of Go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.","title":"Mastering the game of Go with deep neural networks and tree search","authors":"A Silver, C J Huang, Maddison, L Guez, Sifre, G Den Driessche, J Van, I Schrittwieser, V Antonoglou, M Panneershelvam, Lanctot","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Silver et al., 2017] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, and Adrian Bolton. Mastering the game of Go without human knowledge. Nature, 550(7676):354-359, 2017.","title":"Mastering the game of Go without human knowledge","authors":"David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton","year":"2017","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2017] Yunlong Wu, Bo Zhang, Shaoshi Yang, Xiaodong Yi, and Xuejun Yang. Energy-efficient join- t communication-motion planning for relay-assisted wire- less robot surveillance. In IEEE Conference on Computer Communications (INFOCOM), pages 1-9. IEEE, 2017.","title":"Energy-efficient joint communication-motion planning for relay-assisted wireless robot surveillance","authors":"Yunlong Wu, Bo Zhang, Shaoshi Yang, Xiaodong Yi, Xuejun Yang","year":"2017","venue":"IEEE Conference on Computer Communications (INFOCOM)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}