{"type":"ScholarlyArticle","title":"Automated Generation of Interaction Graphs for Value-Factored Dec-POMDPs","authors":[{"name":"William Yeoh","slug":"william-yeoh","given":"William","family":"Yeoh","affiliation":"New Mexico State University Las Cruces , NM 88011 , USA"},{"name":"Akshat Kumar","slug":"akshat-kumar","given":"Akshat","family":"Kumar","affiliation":"IBM Research New Delhi , India , 110070"},{"name":"Shlomo Zilberstein","slug":"shlomo-zilberstein","given":"Shlomo","family":"Zilberstein","affiliation":"University of Massachusetts Amherst , MA 01003 , USA"}],"abstract":"The Decentralized Partially Observable Markov Decision Process (Dec-POMDP) is a powerful model for multi-agent planning under uncertainty, but its applicability is hindered by its high complexity -- solving Dec-POMDPs optimally is NEXP-hard. Recently, Kumar et al. introduced the Value Factorization (VF) framework, which exploits decomposable value functions that can be factored into subfunctions. This framework has been shown to be a generalization of several models that leverage sparse agent interactions such as TI-Dec-MDPs, ND-POMDPs and TD-POMDPs. Existing algorithms for these models assume that the interaction graph of the problem is given. In this paper, we introduce three algorithms to automatically generate interaction graphs for models within the VF framework and establish lower and upper bounds on the expected reward of an optimal joint policy. We illustrate experimentally the benefits of these techniques for sensor placement in a decentralized tracking application.","doi":null,"url":"https://ijcai.org/proceedings/2013/68","pdf":"https://ijcai.org/proceedings/2013/0068.pdf","markdown":"https://ijcai.org/proceedings/2013/68.md","bibtex":"https://ijcai.org/proceedings/2013/bibtex/68","volume":{"slug":"2013","title":"Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2013","url":"https://ijcai.org/proceedings/2013/","year":2013,"location":"Beijing, China","dates":"3–9 August 2013","editors":"Francesca Rossi","isbn":"978-1-57735-633-2","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":68,"pages":{"first":411,"last":416},"keywords":[],"videos":[],"references":[{"position":1,"raw":"[Becker et al., 2004] Raphen Becker, Shlomo Zilberstein, Victor Lesser, and Claudia Goldman. Solving transition in- dependent decentralized Markov decision processes. Jour- nal of Artificial Intelligence Research, 22:423-455, 2004.","title":"Solving transition independent decentralized Markov decision processes","authors":"Becker","year":"2004","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bernstein et al., 2002] Daniel Bernstein, Robert Givan, Neil Immerman, and Shlomo Zilberstein. The complexity of decentralized control of Markov decision processes. Math- ematics of Operations Research, 27(4):819-840, 2002.","title":"The complexity of decentralized control of Markov decision processes","authors":"Daniel Bernstein, Robert Givan, Neil Immerman, Shlomo Zilberstein","year":"2002","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Guestrin et al., 2001] Carlos Guestrin, Daphne Koller, and Ronald Parr. Multiagent planning with factored MDPs. In Advances in Neural Information Processing Systems (NIPS), pages 1523-1530, 2001.","title":"Multiagent planning with factored MDPs","authors":"Carlos Guestrin, Daphne Koller, Ronald Parr","year":"2001","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Koller and Parr, 1999] Daphne Koller and Ronald Parr. Computing factored value functions for policies in struc- tured MDPs. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pages 1332- 1339, 1999.","title":"Computing factored value functions for policies in structured MDPs","authors":"Parr Koller, Daphne Koller, Ronald Parr","year":"1999","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/1999/189"},{"position":5,"raw":"[Krainin et al., 2007] Michael Krainin, Bo An, and Victor Lesser. An application of automated negotiation to dis- tributed task allocation. In Proceedings of the Interna- tional Conference on Intelligent Agent Technology (IAT), pages 138-145, 2007.","title":"An application of automated negotiation to distributed task allocation","authors":"Bo Michael Krainin, Victor An, Lesser","year":"2007","venue":"Proceedings of the International Conference on Intelligent Agent Technology (IAT)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Krause et al., 2008] Andreas Krause, Ajit Singh, and Car- los Guestrin. Near-optimal sensor placements in Gaussian processes: Theory, efficient algorithms and empirical stud- ies. Journal of Machine Learning Research, 9:235-284, 2008.","title":"Near-optimal sensor placements in Gaussian processes: Theory, efficient algorithms and empirical studies","authors":"Andreas Krause, Ajit Singh, Carlos Guestrin","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kumar and Zilberstein, 2009] Akshat Kumar and Shlomo Zilberstein. Constraint-based dynamic programming for decentralized POMDPs with structured interactions. In Proceedings of the International Joint Conference on Au- tonomous Agents and Multiagent Systems (AAMAS), pages 561-568, 2009.","title":"Constraint-based dynamic programming for decentralized POMDPs with structured interactions","authors":"Zilberstein Kumar, Akshat Kumar, Shlomo Zilberstein","year":"2009","venue":"Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kumar et al., 2011] Akshat Kumar, Shlomo Zilberstein, and Marc Toussaint. Scalable multiagent planning using probabilistic inference. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pages 2140-2146, 2011.","title":"Scalable multiagent planning using probabilistic inference","authors":"Akshat Kumar, Shlomo Zilberstein, Marc Toussaint","year":"2011","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2011/357"},{"position":9,"raw":"[Lesser and Corkill, 1983] Victor Lesser and Daniel Corkill. The Distributed Vehicle Monitoring Testbed: A tool for in- vestigating distributed problem solving networks. AI Mag- azine, 4(3):15-33, 1983.","title":"Victor Lesser and Daniel Corkill. The Distributed Vehicle Monitoring Testbed: A tool for investigating distributed problem solving networks","authors":"Corkill Lesser","year":"1983","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Nair et al., 2005] Ranjit Nair, Pradeep Varakantham, Milind Tambe, and Makoto Yokoo. Networked distributed POMDPs: A synthesis of distributed constraint optimiza- tion and POMDPs. In Proceedings of the National Con- ference on Artificial Intelligence (AAAI), pages 133-139, 2005.","title":"Networked distributed POMDPs: A synthesis of distributed constraint optimization and POMDPs","authors":"Ranjit Nair, Pradeep Varakantham, Milind Tambe, Makoto Yokoo","year":"2005","venue":"Proceedings of the National Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Oliehoek et al., 2008] Frans Oliehoek, Matthijs Spaan, Shi- mon Whiteson, and Nikos Vlassis. Exploiting locality of interaction in factored Dec-POMDPs. In Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 517-524, 2008.","title":"Exploiting locality of interaction in factored Dec-POMDPs","authors":"Frans Oliehoek, Matthijs Spaan, Shimon Whiteson, Nikos Vlassis","year":"2008","venue":"Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Oliehoek, 2010] Frans Oliehoek. Value-Based Planning for Teams of Agents in Stochastic Partially Observable Envi- ronments. PhD thesis, University of Amsterdam, Amster- dam (Netherlands), 2010.","title":"Frans Oliehoek. Value-Based Planning for Teams of Agents in Stochastic Partially Observable Environments","authors":"Oliehoek","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Rabinovich et al., 2003] Zinovi Rabinovich, Claudia Gold- man, and Jeffrey Rosenschein. The complexity of multia- gent systems: The price of silence. In Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 1102-1103, 2003.","title":"The complexity of multiagent systems: The price of silence","authors":"Rabinovich","year":"2003","venue":"Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Szer and Charpillet, 2006] Daniel Szer and Francois Charpillet. Point-based dynamic programming for DEC- POMDPs. In Proceedings of the National Conference on Artificial Intelligence (AAAI), pages 1233-1238, 2006.","title":"Daniel Szer and Francois Charpillet. Point-based dynamic programming for DEC-POMDPs","authors":"Charpillet Szer","year":"2006","venue":"Proceedings of the National Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Velagapudi et al., 2011] Prasanna Velagapudi, Pradeep Varakantham, Paul Scerri, and Katia Sycara. Distributed model shaping for scaling to decentralized POMDPs with hundreds of agents. In Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 955-962, 2011.","title":"Distributed model shaping for scaling to decentralized POMDPs with hundreds of agents","authors":"Prasanna Velagapudi, Pradeep Varakantham, Paul Scerri, Katia Sycara","year":"2011","venue":"Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Witwicki and Durfee, 2010] Stefan Witwicki and Edmund Durfee. From policies to influences: A framework for nonlocal abstraction in transition-dependent Dec-POMDP agents. In Proceedings of the International Joint Confer- ence on Autonomous Agents and Multiagent Systems (AA- MAS), pages 1397-1398, 2010.","title":"From policies to influences: A framework for nonlocal abstraction in transition-dependent Dec-POMDP agents","authors":"Durfee Witwicki, Stefan Witwicki, Edmund Durfee","year":"2010","venue":"Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AA-MAS)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Witwicki and Durfee, 2011] Stefan Witwicki and Edmund Durfee. Towards a unifying characterization for quanti- fying weak coupling in Dec-POMDPs. In Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 29-36, 2011.","title":"Towards a unifying characterization for quantifying weak coupling in Dec-POMDPs","authors":"Durfee Witwicki, Stefan Witwicki, Edmund Durfee","year":"2011","venue":"Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}