{"type":"ScholarlyArticle","title":"Compact Representation of Value Function in Partially Observable Stochastic Games","authors":[{"name":"Karel Horák","slug":"karel-horak","given":"Karel","family":"Horák","affiliation":"Czech Technical University in Prague , FEE , Department of Computer Science"},{"name":"Branislav Bošanský","slug":"branislav-bosansky","given":"Branislav","family":"Bošanský","affiliation":"Czech Technical University in Prague , FEE , Department of Computer Science"},{"name":"Christopher Kiekintveld","slug":"christopher-kiekintveld","given":"Christopher","family":"Kiekintveld","affiliation":"The University of Texas at El Paso , Computer Science Department"},{"name":"Charles Kamhoua","slug":"charles-kamhoua","given":"Charles","family":"Kamhoua","affiliation":"Army Research Laboratory , Network Security Branch"}],"abstract":"Value methods for solving stochastic games with partial observability model the uncertainty of the players as a probability distribution over possible states, where the dimension of the belief space is the number of states. For many practical problems, there are exponentially many states which causes scalability problems. We propose an abstraction technique that addresses this curse of dimensionality by projecting the high-dimensional beliefs onto characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our main contributions are (1) a novel compact representation of the uncertainty in partially observable stochastic games and (2) a novel algorithm using this representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm using the compact representation dramatically increases scalability compared to the state of the art.","doi":"10.24963/ijcai.2019/50","url":"https://ijcai.org/proceedings/2019/50","pdf":"https://ijcai.org/proceedings/2019/0050.pdf","markdown":"https://ijcai.org/proceedings/2019/50.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/50","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Agent-based and Multi-agent Systems","track":"Main track","number":50,"pages":{"first":350,"last":356},"keywords":["Agent-based and Multi-agent Systems: Noncooperative Games","Agent-based and Multi-agent Systems: Algorithmic Game Theory"],"videos":[],"references":[{"position":1,"raw":"[Basilico et al., 2009] Nicola Basilico, Nicola Gatti, and Francesco Amigoni. Leader-follower strategies for robotic patrolling in environments with arbitrary topologies. In 8th International Conference on Autonomous Agents and Multiagent Systems, pages 57-64, 2009.","title":"Leader-follower strategies for robotic patrolling in environments with arbitrary topologies","authors":"Nicola Basilico, Nicola Gatti, Francesco Amigoni","year":"2009","venue":"8th International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brazdil et al., 2018] Tomas Brazdil, Antonin Kucera, and Vojtech Rehak. Solving Patrolling Problems in the Inter- net Environment. In 27th International Joint Conference on Artificial Intelligence, 2018.","title":"Solving Patrolling Problems in the Internet Environment","authors":"Brazdil","year":"2018","venue":"27th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/17"},{"position":3,"raw":"[Chung et al., 2011] Timothy H Chung, Geoffrey A Hollinger, and Volkan Isler. Search and pursuit-evasion in mobile robotics. Autonomous robots, 31(4):299-316, 2011.","title":"Search and pursuit-evasion in mobile robotics","authors":"Timothy H Chung, Geoffrey A Hollinger, Volkan Isler","year":"2011","venue":"Autonomous robots","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fang et al., 2015] Fei Fang, Peter Stone, and Milind Tambe. When Security Games Go Green: Designing Defender Strategies to Prevent Poaching and Illegal Fishing. In 24th International Joint Conference on Artificial Intelligence, 2015.","title":"When Security Games Go Green: Designing Defender Strategies to Prevent Poaching and Illegal Fishing","authors":"Fei Fang, Peter Stone, Milind Tambe","year":"2015","venue":"24th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/367"},{"position":5,"raw":"[Fang et al., 2016] Fei Fang, Thanh Hong Nguyen, Rob Pickles, Wai Y. Lam, Gopalasamy R. Clements, Bo An, Amandeep Singh, Milind Tambe, and Andrew Lemieux. Deploying PAWS: Field optimization of the protection as- sistant for wildlife security. In 30th AAAI Conference on Artificial Intelligence, pages 3966-3973, 2016.","title":"Deploying PAWS: Field optimization of the protection assistant for wildlife security","authors":"Fei Fang, Thanh Hong Nguyen, Rob Pickles, Wai Y Lam, Gopalasamy R Clements, Bo An, Amandeep Singh, Milind Tambe, Andrew Lemieux","year":"2016","venue":"30th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Horák and Bošanský, 2016] Karel Horák and Branislav Bošanský. A Point-Based Approximate Algorithm for One-Sided Partially Observable Pursuit-Evasion Games. In International Conference on Decision and Game The- ory for Security, 2016.","title":"A Point-Based Approximate Algorithm for One-Sided Partially Observable Pursuit-Evasion Games","authors":"Bošanský Horák, Karel Horák, Branislav Bošanský","year":"2016","venue":"International Conference on Decision and Game Theory for Security","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Horák and Bošanský, 2019] Karel Horák and Branislav Bošanský. Solving Partially Observable Stochastic Games with Public Observations. In 33rd AAAI Conference on Artificial Intelligence, 2019.","title":"Solving Partially Observable Stochastic Games with Public Observations","authors":"Bošanský Horák, Karel Horák, Branislav Bošanský","year":"2019","venue":"33rd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Horák et al., 2017] Karel Horák, Branislav Bošanský, and Michal Pěchouček. Heuristic Search Value Iteration for One-Sided Partially Observable Stochastic Games. In 31st AAAI Conference on Artificial Intelligence, 2017.","title":"Heuristic Search Value Iteration for One-Sided Partially Observable Stochastic Games","authors":"Horák","year":"2017","venue":"31st AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kamdem et al., 2017] Gael Kamdem, Charles Kamhoua, Yue Lu, Sachin Shetty, and Laurent Njilla. A markov game theoritic approach for power grid security. In 37th International Conference on Distributed Computing Sys- tems Workshops, pages 139-144. IEEE, 2017.","title":"A markov game theoritic approach for power grid security","authors":"Gael Kamdem, Charles Kamhoua, Yue Lu, Sachin Shetty, Laurent Njilla","year":"2017","venue":"37th International Conference on Distributed Computing Systems Workshops","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kuipers and Fabro, 2006] David Kuipers and Mark Fabro. Control systems cyber security: Defense in depth strate- gies. United States. Department of Energy, 2006.","title":"Control systems cyber security: Defense in depth strategies","authors":"Fabro Kuipers, David Kuipers, Mark Fabro","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2010] Xin Li, William K Cheung, and Jiming Liu. Improving pomdp tractability via belief compression and clustering. IEEE Transactions on Systems, Man, and Cy- bernetics, Part B (Cybernetics), 40(1):125-136, 2010.","title":"Improving pomdp tractability via belief compression and clustering","authors":"Xin Li, William K Cheung, Jiming Liu","year":"2010","venue":"IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Nguyen et al., 2017] Thanh H. Nguyen, Michael P. Well- man, and Satinder Singh. A Stackelberg Game Model for Botnet Data Exfiltration. In International Conference on Decision and Game Theory for Security, 2017.","title":"A Stackelberg Game Model for Botnet Data Exfiltration","authors":"Thanh H Nguyen, Michael P Wellman, Satinder Singh","year":"2017","venue":"International Conference on Decision and Game Theory for Security","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Roy et al., 2005] Nicholas Roy, Geoffrey Gordon, and Se- bastian Thrun. Finding approximate pomdp solutions through belief compression. Journal of artificial intelli- gence research, 23:1-40, 2005.","title":"Finding approximate pomdp solutions through belief compression","authors":"Roy","year":"2005","venue":"Journal of artificial intelligence research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Smith and Simmons, 2004] Trey Smith and Reid Simmons. Heuristic Search Value Iteration for POMDPs. In 20th conference on Uncertainty in artificial intelligence, 2004.","title":"Heuristic Search Value Iteration for POMDPs","authors":"Simmons Smith, Trey Smith, Reid Simmons","year":"2004","venue":"20th conference on Uncertainty in artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Vorobeychik et al., 2014] Yevgeniy Vorobeychik, Bo An, Milind Tambe, and Satinder P. Singh. Computing So- lutions in Infinite-Horizon Discounted Adversarial Pa- trolling Games. In 24th International Conference on Au- tomated Planning and Scheduling, 2014.","title":"Computing Solutions in Infinite-Horizon Discounted Adversarial Patrolling Games","authors":"Yevgeniy Vorobeychik, Bo An, Milind Tambe, Satinder P Singh","year":"2014","venue":"24th International Conference on Automated Planning and Scheduling","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Zhou et al., 2010] Enlu Zhou, Michael C Fu, and Steven I Marcus. Solving continuous-state pomdps via density projection. IEEE Transactions on Automatic Control, 55(5):1101-1116, 2010.","title":"Solving continuous-state pomdps via density projection","authors":"Zhou","year":"2010","venue":"IEEE Transactions on Automatic Control","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}