{"type":"ScholarlyArticle","title":"Decision-Making Under Uncertainty in Multi-Agent and Multi-Robot Systems: Planning and Learning","authors":[{"name":"Christopher Amato","slug":"christopher-amato","given":"Christopher","family":"Amato","affiliation":"CCIS , Northeastern University , Boston , MA , USA"}],"abstract":"Multi-agent planning and learning methods are becoming increasingly important in today's interconnected world. Methods for real-world domains, such as robotics, must consider uncertainty and limited communication in order to generate high-quality, robust solutions. This paper discusses our work on developing principled models to represent these problems and planning and learning methods that can scale to realistic multi-agent and multi-robot tasks.","doi":"10.24963/ijcai.2018/805","url":"https://ijcai.org/proceedings/2018/805","pdf":"https://ijcai.org/proceedings/2018/0805.pdf","markdown":"https://ijcai.org/proceedings/2018/805.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/805","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Early Career","track":"Early Career","number":805,"pages":{"first":5662,"last":5666},"keywords":["Robotics: Multi-Robot Systems","Machine Learning: Reinforcement Learning","Agent-based and Multi-agent Systems: Multi-agent Planning","Agent-based and Multi-agent Systems: Multi-agent Learning"],"videos":[],"references":[{"position":1,"raw":"[Amato et al., 2013] Christopher Amato, Girish Chowdhary, Al- borz Geramifard, Nazim Kemal Ure, and Mykel J. Kochender- fer. Decentralized control of partially observable Markov deci- sion processes. In CDC, 2013.","title":"Decentralized control of partially observable Markov decision processes","authors":"Christopher Amato, Girish Chowdhary, Alborz Geramifard, Nazim Kemal Ure, Mykel J Kochenderfer","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amato et al., 2014] Christopher Amato, George D. Konidaris, and Leslie P. Kaelbling. Planning with macro-actions in decentralized POMDPs. In AAMAS, 2014.","title":"Planning with macro-actions in decentralized POMDPs","authors":"Christopher Amato, George D Konidaris, Leslie P Kaelbling","year":"2014","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Amato et al., 2015a] Christopher Amato, George D. Konidaris, Ariel Anders, Gabriel Cruz, Jonathan P. How, and Leslie P. Kael- bling. Policy search for multi-robot coordination under uncer- tainty. In RSS, 2015.","title":"Policy search for multi-robot coordination under uncertainty","authors":"Amato","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Amato et al., 2015b] Christopher Amato, George D. Konidaris, Gabriel Cruz, Christopher A. Maynor, Jonathan P. How, and Leslie P. Kaelbling. Planning for decentralized control of mul- tiple robots under uncertainty. In ICRA, 2015.","title":"Planning for decentralized control of multiple robots under uncertainty","authors":"Amato","year":"2015","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Amato et al., 2017] Christopher Amato, George D. Konidaris, Ariel Anders, Gabriel Cruz, Jonathan P. How, and Leslie P. Kael- bling. Policy search for multi-robot coordination under uncer- tainty. The International Journal of Robotics Research, 2017.","title":"Policy search for multi-robot coordination under uncertainty","authors":"Christopher Amato, George D Konidaris, Ariel Anders, Gabriel Cruz, Jonathan P How, Leslie P Kaelbling","year":"2017","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bernstein et al., 2002] Daniel S. Bernstein, Robert Givan, Neil Im- merman, and Shlomo Zilberstein. The complexity of decentral- ized control of Markov decision processes. Mathematics of Op- erations Research, 27(4), 2002.","title":"The complexity of decentralized control of Markov decision processes","authors":"S Daniel, Robert Bernstein, Neil Givan, Shlomo Immerman, Zilberstein","year":"2002","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Burgard et al., 2005] Wolfram Burgard, Mark Moors, Cyrill Stachniss, and Frank E Schneider. Coordinated multi-robot ex- ploration. IEEE Transactions on Robotics, 21(3), 2005.","title":"Coordinated multi-robot exploration","authors":"Wolfram Burgard, Mark Moors, Cyrill Stachniss, Frank E Schneider","year":"2005","venue":"IEEE Transactions on Robotics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Claes et al., 2017] Daniel Claes, Frans Oliehoek, Hendrik Baier, and Karl Tuyls. Decentralised online planning for multi-robot warehouse commissioning. In AAMAS, 2017.","title":"Decentralised online planning for multi-robot warehouse commissioning","authors":"Daniel Claes, Frans Oliehoek, Hendrik Baier, Karl Tuyls","year":"2017","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dibangoye et al., 2016] Jilles S. Dibangoye, Christopher Amato, Olivier Buffet, and Franc ¸ois Charpillet. Optimally solving Dec- POMDPs as continuous-state MDPs. Journal of Artificial Intelli- gence Research, 55, 2016.","title":"Optimally solving Dec-POMDPs as continuous-state MDPs","authors":"S Jilles, Christopher Dibangoye, Olivier Amato, Franc Buffet, Charpillet ¸ois","year":"2016","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Foerster et al., 2016] Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, and Shimon Whiteson. Learning to communicate to solve riddles with deep distributed recurrent Q-networks. In NIPS, 2016.","title":"Learning to communicate to solve riddles with deep distributed recurrent Q-networks","authors":"Jakob N Foerster, M Yannis, Nando Assael, Shimon De Freitas, Whiteson","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Foerster et al., 2017] Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Philip Torr, Pushmeet Kohli, and Shimon Whiteson. Stabilising experience replay for deep multi-agent reinforcement learning. In ICML, 2017.","title":"Stabilising experience replay for deep multi-agent reinforcement learning","authors":"Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Philip Torr, Pushmeet Kohli, Shimon Whiteson","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kaelbling et al., 1998] Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. Planning and acting in partially ob- servable stochastic domains. Artificial Intelligence, 101, 1998.","title":"Planning and acting in partially observable stochastic domains","authors":"Leslie Pack Kaelbling, Michael L Littman, Anthony R Cassandra","year":"1998","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Korsah et al., 2013] G Ayorkor Korsah, Anthony Stentz, and M Bernardine Dias. A comprehensive taxonomy for multi-robot task allocation. The International Journal of Robotics Research, 32(12), 2013.","title":"A comprehensive taxonomy for multi-robot task allocation","authors":"Anthony Ayorkor Korsah, M Bernardine Stentz, Dias","year":"2013","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2015] Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin, and Jonathan P. How. Stick-breaking policy learning in Dec-POMDPs. In IJCAI, 2015.","title":"How. Stick-breaking policy learning in Dec-POMDPs","authors":"Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin, Jonathan P","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2016] Miao Liu, Christopher Amato, Emily Anesta, J. Daniel Griffith, and Jonathan P. How. Learning for decentral- ized control of multiagent systems in large partially observable stochastic environments. In AAAI, 2016.","title":"Learning for decentralized control of multiagent systems in large partially observable stochastic environments","authors":"Miao Liu, Christopher Amato, Emily Anesta, J Daniel Griffith, Jonathan P How","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2017] Miao Liu, Kavinayan Sivakumar, Shayegan Omidshafiei, Christopher Amato, and Jonathan P. How. Learn- ing for multi-robot cooperation in partially observable stochastic environments with macro-actions. In IROS, 2017.","title":"How. Learning for multi-robot cooperation in partially observable stochastic environments with macro-actions","authors":"Miao Liu, Kavinayan Sivakumar, Shayegan Omidshafiei, Christopher Amato, Jonathan P","year":"2017","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Matignon et al., 2007] Laëtitia Matignon, Guillaume J. Laurent, and Nadine Le Fort-Piat. Hysteretic Q-learning: an algorithm for decentralized reinforcement learning in cooperative multi-agent teams. In IROS, 2007.","title":"Hysteretic Q-learning: an algorithm for decentralized reinforcement learning in cooperative multi-agent teams","authors":"Laëtitia Matignon, Guillaume J Laurent, Nadine Le Fort-Piat","year":"2007","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostro- vski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540), 2015.","title":"Human-level control through deep reinforcement learning","authors":"Mnih","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mordatch and Abbeel, 2017] Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations. arXiv preprint 1703.04908, 2017.","title":"Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations","authors":"Abbeel Mordatch","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nguyen et al., 2017] Duc Thien Nguyen, Akshat Kumar, and Hoong Chuin Lau. Policy gradient with value function approxi- mation for collective multiagent planning. In NIPS. 2017.","title":"Policy gradient with value function approximation for collective multiagent planning","authors":"Akshat Duc Thien Nguyen, Hoong Chuin Kumar, Lau","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Oliehoek and Amato, 2016] Frans A. Oliehoek and Christopher Amato. A Concise Introduction to Decentralized POMDPs. Springer, 2016.","title":"A Concise Introduction to Decentralized POMDPs","authors":"Frans A Oliehoek, Christopher Amato","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Oliehoek, 2012] Frans A. Oliehoek. Decentralized POMDPs. In Marco Wiering and Martijn van Otterlo, editors, Reinforcement Learning: State of the Art, volume 12 of Adaptation, Learning, and Optimization. Springer Berlin Heidelberg, 2012.","title":"Decentralized POMDPs","authors":"Frans A Oliehoek","year":"2012","venue":"Reinforcement Learning: State of the Art, volume 12 of Adaptation, Learning, and Optimization","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Omidshafiei et al., 2015] Shayegan Omidshafiei, Ali-akbar Agha- mohammadi, Christopher Amato, and Jonathan P. How. De- centralized control of partially observable Markov decision pro- cesses using belief space macro-actions. In ICRA, 2015.","title":"Decentralized control of partially observable Markov decision processes using belief space macro-actions","authors":"Shayegan Omidshafiei, Ali-Akbar Aghamohammadi, Christopher Amato, Jonathan P How","year":"2015","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Omidshafiei et al., 2016] Shayegan Omidshafiei, Ali-akbar Agha- mohammadi, Christopher Amato, Shih-Yuan Liu, Jonathan P. How, and John Vian. Graph-based cross entropy method for solv- ing multi-robot decentralized POMDPs. In ICRA, 2016.","title":"Graph-based cross entropy method for solving multi-robot decentralized POMDPs","authors":"Shayegan Omidshafiei, Ali-Akbar Aghamohammadi, Christopher Amato, Shih-Yuan Liu, Jonathan P How, John Vian","year":"2016","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Omidshafiei et al., 2017a] Shayegan Omidshafiei, Ali-akbar Agha-mohammadi, Christopher Amato, and Jonathan P. How. Decentralized control of multi-robot partially observable markov decision processes using belief space macro-actions. The International Journal of Robotics Research, 2017.","title":"Decentralized control of multi-robot partially observable markov decision processes using belief space macro-actions","authors":"Omidshafiei","year":"2017","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Omidshafiei et al., 2017b] Shayegan Omidshafiei, Shih-Yuan Liu, Michael Everett, Brett Lopez, Christopher Amato, Miao Liu, Jonathan P. How, and John Vian. Semantic-level decentral- ized multi-robot decision-making using probabilistic macro- observations. In ICRA, 2017.","title":"Semantic-level decentralized multi-robot decision-making using probabilistic macroobservations","authors":"Omidshafiei","year":"2017","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Omidshafiei et al., 2017c] Shayegan Omidshafiei, Jason Pazis, Christopher Amato, Jonathan P. How, and John Vian. Deep de- centralized multi-task multi-agent reinforcement learning under partial observability. In ICML, 2017.","title":"Deep decentralized multi-task multi-agent reinforcement learning under partial observability","authors":"Omidshafiei","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Rashid et al., 2018] Tabish Rashid, Mikayel Samvelyan, Chris- tian Schröder de Witt, Gregory Farquhar, Jakob N. Foerster, and Shimon Whiteson. QMIX: monotonic value function factorisa- tion for deep multi-agent reinforcement learning. arXiv preprint 1803.11485, 2018.","title":"QMIX: monotonic value function factorisation for deep multi-agent reinforcement learning","authors":"Tabish Rashid, Mikayel Samvelyan, Christian Schröder De Witt, Gregory Farquhar, Jakob N Foerster, Shimon Whiteson","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Ren and Sorensen, 2008] Wei Ren and Nathan Sorensen. Dis- tributed coordination architecture for multi-robot formation con- trol. Robotics and Autonomous Systems, 56(4), 2008.","title":"Distributed coordination architecture for multi-robot formation control","authors":"Ren, Sorensen, Wei Ren, Nathan Sorensen","year":"2008","venue":"Robotics and Autonomous Systems","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Rubenstein et al., 2014] Michael Rubenstein, Alejandro Cornejo, and Radhika Nagpal. Programmable self-assembly in a thousand- robot swarm. Science, 345(6198), 2014.","title":"Programmable self-assembly in a thousandrobot swarm","authors":"Alejandro Michael Rubenstein, Radhika Cornejo, Nagpal","year":"6198","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of Go with deep neural net- works and tree search. Nature, 529(7587), 2016.","title":"Mastering the game of Go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den, Julian Driessche, Ioannis Schrittwieser, Veda Antonoglou, Marc Panneershelvam, Lanctot","year":null,"venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sutton et al., 1999] Richard S Sutton, Doina Precup, and Satinder Singh. Between MDPs and semi-MDPs: A framework for tempo- ral abstraction in reinforcement learning. Artificial Intelligence, 112(1), 1999.","title":"Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning","authors":"Doina Richard S Sutton, Satinder Precup, Singh","year":"1999","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}