{"type":"ScholarlyArticle","title":"COG-DICE: An Algorithm for Solving Continuous-Observation Dec-POMDPs","authors":[{"name":"Madison Clark-Turner","slug":"madison-clark-turner","given":"Madison","family":"Clark-Turner","affiliation":"Department of Computer Science University of New Hampshire Durham , NH 03824"},{"name":"Christopher Amato","slug":"christopher-amato","given":"Christopher","family":"Amato","affiliation":"College of Computer and Information Science Northeastern University Boston , MA 02115"}],"abstract":"The decentralized partially observable Markov decision process (Dec-POMDP) is a powerful model for representing multi-agent problems with decentralized behavior. Unfortunately, current Dec-POMDP solution methods cannot solve problems with continuous observations, which are common in many real-world domains. To that end, we present a framework for representing and generating Dec-POMDP policies that explicitly include continuous observations. We apply our algorithm to a novel tagging problem and an extended version of a common benchmark, where it generates policies that meet or exceed the values of equivalent discretized domains without the need for finding an adequate discretization.","doi":"10.24963/ijcai.2017/638","url":"https://ijcai.org/proceedings/2017/638","pdf":"https://ijcai.org/proceedings/2017/0638.pdf","markdown":"https://ijcai.org/proceedings/2017/638.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/638","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Uncertainty in AI","track":"Main track","number":638,"pages":{"first":4573,"last":4579},"keywords":["Agent-based and Multi-agent Systems: Coordination and cooperation","Uncertainty in AI: Markov Decision Processes","Planning and Scheduling: Distributed/Multi-agent Planning"],"videos":[],"references":[{"position":1,"raw":"[Amato et al., 2012] Christopher Amato, Daniel S Bern- stein, and Shlomo Zilberstein. Optimizing memory- bounded controllers for decentralized POMDPs. Proceed- ings of Conference on Uncertainty in Artificial Intelli- gence, 2012.","title":"Optimizing memorybounded controllers for decentralized POMDPs","authors":"Christopher Amato, Daniel S Bernstein, Shlomo Zilberstein","year":"2012","venue":"Proceedings of Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amato et al., 2017] Christopher Amato, George D. Konidaris, Ariel Anders, Gabriel Cruz, Jonathan P. How, and Leslie P. Kaelbling. Policy search for multi-robot coordination under uncertainty. The International Journal of Robotics Research, 2017.","title":"Policy search for multi-robot coordination under uncertainty","authors":"Christopher Amato, George D Konidaris, Ariel Anders, Gabriel Cruz, Jonathan P How, Leslie P Kaelbling","year":"2017","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bai et al., 2010] Haoyu Bai, David Hsu, Wee Sun Lee, and Vien A Ngo. Monte Carlo value iteration for continuous- state POMDPs. In Proceedings of the International Work- shop on the Algorithmic Foundations of Robotics, pages 13-15, 2010.","title":"Monte Carlo value iteration for continuousstate POMDPs","authors":"Haoyu Bai, David Hsu, Wee Sun Lee, Vien A Ngo","year":"2010","venue":"Proceedings of the International Workshop on the Algorithmic Foundations of Robotics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bernstein et al., 2002] Daniel S Bernstein, Robert Givan, Neil Immerman, and Shlomo Zilberstein. The com- plexity of decentralized control of Markov decision pro- cesses. Mathematics of operations research, 27(4):819- 840, 2002.","title":"The complexity of decentralized control of Markov decision processes","authors":"Robert Daniel S Bernstein, Neil Givan, Shlomo Immerman, Zilberstein","year":"2002","venue":"Mathematics of operations research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bernstein et al., 2009] Daniel S Bernstein, Christopher Am- ato, Eric A Hansen, and Shlomo Zilberstein. Policy iteration for decentralized control of Markov decision processes. Journal of Artificial Intelligence Research, 34(1):89, 2009.","title":"Policy iteration for decentralized control of Markov decision processes","authors":"Christopher Daniel S Bernstein, Eric A Amato, Shlomo Hansen, Zilberstein","year":"2009","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brunskill et al., 2008] Emma Brunskill, Leslie Pack Kaelbling, Tomas Lozano-Perez, and Nicholas Roy. Continuous-state POMDPs with hybrid dynamics. In Proceedings of the International Symposium on Artificial Intelligence and Mathematics, 2008.","title":"Continuous-state POMDPs with hybrid dynamics","authors":"Emma Brunskill, Leslie Pack Kaelbling, Tomas Lozano-Perez, Nicholas Roy","year":"2008","venue":"Proceedings of the International Symposium on Artificial Intelligence and Mathematics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hoey and Poupart, 2005] Jesse Hoey and Pascal Poupart. Solving POMDPs with continuous or large discrete obser- vation spaces. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 1332-1338, 2005.","title":"Solving POMDPs with continuous or large discrete observation spaces","authors":"Poupart Hoey, Jesse Hoey, Pascal Poupart","year":"2005","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2005/172"},{"position":8,"raw":"[Kaelbling et al., 1998] Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. Planning and act- ing in partially observable stochastic domains. Artificial Intelligence, 101:1-45, 1998.","title":"Planning and acting in partially observable stochastic domains","authors":"Leslie Pack Kaelbling, Michael L Littman, Anthony R Cassandra","year":"1998","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2015] Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin, and Jonathan P How. Stick- breaking policy learning in Dec-POMDPs. In Proceedings of the International Joint Conference on Artificial Intelli- gence, 2015.","title":"Stickbreaking policy learning in Dec-POMDPs","authors":"Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin, Jonathan P How","year":"2015","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[MacDermed and Isbell, 2013] Liam C MacDermed and Charles Isbell. Point based value iteration with optimal belief compression for Dec-POMDPs. In Proceedings of the Advances in Neural Information Processing Systems, pages 100-108, 2013.","title":"Point based value iteration with optimal belief compression for Dec-POMDPs","authors":"C Liam, Charles Macdermed, Isbell","year":"2013","venue":"Proceedings of the Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Natrella, 2010] Mary Natrella. NIST/SEMATECH e- handbook of statistical methods. 2010.","title":"NIST/SEMATECH ehandbook of statistical methods","authors":"Mary Natrella","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Oliehoek and Amato, 2016] Frans A Oliehoek and Christo- pher Amato. Concise Introduction to Decentralized POMDPs. Springer, 2016.","title":"Concise Introduction to Decentralized POMDPs","authors":"Amato Oliehoek","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Oliehoek et al., 2008] Frans A Oliehoek, Julian FP Kooij, Nikos Vlassis, et al. The cross-entropy method for policy search in decentralized POMDPs. Informatica, 32(4):341- 357, 2008.","title":"The cross-entropy method for policy search in decentralized POMDPs","authors":"Oliehoek","year":"2008","venue":"Informatica","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Omidshafiei et al., 2016] Shayegan Omidshafiei, Ali-akbar Agha-mohammadi, Christopher Amato, Shih-Yuan Liu, Jonathan P How, and John Vian. Graph-based cross entropy method for solving multi-robot decentralized POMDPs. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation, pages 5395- 5402. IEEE, 2016.","title":"Graph-based cross entropy method for solving multi-robot decentralized POMDPs","authors":"Shayegan Omidshafiei, Ali-Akbar Agha-Mohammadi, Christopher Amato, Shih-Yuan Liu, Jonathan P How, John Vian","year":"2016","venue":"Proceedings of the 2016 IEEE International Conference on Robotics and Automation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Omidshafiei et al., 2017] Shayegan Omidshafiei, Ali-Akbar Agha-Mohammadi, Christopher Amato, Shih-Yuan Liu, Jonathan P How, and John Vian. Decentralized control of multi-robot partially observable markov decision pro- cesses using belief space macro-actions. The International Journal of Robotics Research, pages 231-259, 2017.","title":"Decentralized control of multi-robot partially observable markov decision processes using belief space macro-actions","authors":"Shayegan Omidshafiei, Ali-Akbar Agha-Mohammadi, Christopher Amato, Shih-Yuan Liu, Jonathan P How, John Vian","year":"2017","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Spaan and Oliehoek, 2008] Matthijs T. J. Spaan and Frans A. Oliehoek. The MultiAgent Decision Process toolbox: software for decision-theoretic planning in multiagent systems. In Multi-agent Sequential Decision Making in Uncertain Domains, 2008. Workshop at AAMAS.","title":"The MultiAgent Decision Process toolbox: software for decision-theoretic planning in multiagent systems","authors":"T J Matthijs, Frans A Spaan, Oliehoek","year":"2008","venue":"Multi-agent Sequential Decision Making in Uncertain Domains","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}