{"type":"ScholarlyArticle","title":"Planning Algorithms for Zero-Sum Games with Exponential Action Spaces: A Unifying Perspective","authors":[{"name":"Levi H. S. Lelis","slug":"levi-h-s-lelis","given":"Levi H. S.","family":"Lelis","affiliation":"Department of Computing Science , University of Alberta , Canada"}],"abstract":"In this paper we review several planning algorithms developed for zero-sum games with exponential action spaces, i.e., spaces that grow exponentially with the number of game components that can act simultaneously at a given game state. As an example, real-time strategy games have exponential action spaces because the number of actions available grows exponentially with the number of units controlled by the player. We also present a unifying perspective in which several existing algorithms can be described as an instantiation of a variant of NaiveMCTS. In addition to describing several existing planning algorithms for exponential action spaces, we show that other instantiations of this variant of NaiveMCTS represent novel and promising algorithms to be studied in future works.","doi":"10.24963/ijcai.2020/681","url":"https://ijcai.org/proceedings/2020/681","pdf":"https://ijcai.org/proceedings/2020/0681.pdf","markdown":"https://ijcai.org/proceedings/2020/681.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/681","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Survey track","track":"Survey track","number":681,"pages":{"first":4892,"last":4898},"keywords":["Planning and Scheduling: general","Agent-based and Multi-agent Systems: general","Games and Virtual Environments: general","Heuristic Search: general"],"videos":[],"references":[{"position":1,"raw":"[Balla and Fern, 2009] R-K. Balla and A. Fern. UCT for tac- tical assault planning in real-time strategy games. In Pro- ceedings of the International Joint Conference on Artificial Intelligence, pages 40-45, 2009.","title":"UCT for tactical assault planning in real-time strategy games","authors":"R-K Balla, A Fern","year":"2009","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barriga et al., 2017] N. A Barriga, M. Stanescu, and M. Buro. Combining strategic learning and tactical search in real-time strategy games. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digi- tal Entertainment, pages 9-15. AAAI, 2017.","title":"Combining strategic learning and tactical search in real-time strategy games","authors":"N Barriga, M Stanescu, M Buro","year":"2017","venue":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Browne et al., 2012] C. Browne, E. J. Powley, D. White- house, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. P. Liebana, S. Samothrakis, and S. Colton. A survey of monte carlo tree search methods. IEEE Trans- actions on Comp. Int. and AI in Games, 4(1):1-43, 2012.","title":"A survey of monte carlo tree search methods","authors":"C Browne, E J Powley, D Whitehouse, S M Lucas, P I Cowling, P Rohlfshagen, S Tavener, D P Liebana, S Samothrakis, S Colton","year":"2012","venue":"IEEE Transactions on Comp. Int. and AI in Games","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chung et al., 2005] M. Chung, M. Buro, and J. Schaeffer. Monte Carlo planning in RTS games. In Proceedings of the IEEE Symposium on Comp. Int. and Games, 2005.","title":"Monte Carlo planning in RTS games","authors":"M Chung, M Buro, J Schaeffer","year":"2005","venue":"Proceedings of the IEEE Symposium on Comp. Int. and Games","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Churchill and Buro, 2013] D. Churchill and M. Buro. Port- folio greedy search and simulation for large-scale combat in StarCraft. In Proceedings of the Conference on Comp. Int. in Games, pages 1-8. IEEE, 2013.","title":"Portfolio greedy search and simulation for large-scale combat in StarCraft","authors":"D Churchill, M Buro","year":"2013","venue":"Proceedings of the Conference on Comp. Int. in Games","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Churchill and Buro, 2015] D. Churchill and M. Buro. Hier- archical portfolio search: Prismata's robust AI architecture for games with large search spaces. In AAAI Conference on Artificial Intelligence and Interactive Digital Entertain- ment, pages 16-22. AAAI, 2015.","title":"Hierarchical portfolio search: Prismata's robust AI architecture for games with large search spaces","authors":"D Churchill, M Buro","year":"2015","venue":"AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Churchill et al., 2012] D. Churchill, A. Saffidine, and M. Buro. Fast heuristic search for RTS game combat sce- narios. In Proceedings of the AAAI Conference on Arti- ficial Intelligence and Interactive Digital Entertainment. AAAI, 2012.","title":"Fast heuristic search for RTS game combat scenarios","authors":"D Churchill, A Saffidine, M Buro","year":"2012","venue":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Justesen et al., 2014] N. Justesen, B. Tillman, J. Togelius, and S. Risi. Script-and cluster-based UCT for StarCraft. In IEEE Conference on Comp. Int. and Games, pages 1-8, 2014.","title":"Script-and cluster-based UCT for StarCraft","authors":"N Justesen, B Tillman, J Togelius, S Risi","year":"2014","venue":"IEEE Conference on Comp. Int. and Games","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kovarsky and Buro, 2005] A. Kovarsky and M. Buro. Heuristic search applied to abstract combat games. In Ad- vances in Artificial Intelligence: Conference of the Cana- dian Society for Computational Studies of Intelligence, pages 66-78. Springer, 2005.","title":"Heuristic search applied to abstract combat games","authors":"A Kovarsky, M Buro","year":"2005","venue":"Advances in Artificial Intelligence: Conference of the Canadian Society for Computational Studies of Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lelis, 2017] L. H. S. Lelis. Stratified strategy selection for unit control in real-time strategy games. In Proceedings of the International Joint Conference on Artificial Intelli- gence, pages 3735-3741, 2017.","title":"Stratified strategy selection for unit control in real-time strategy games","authors":"L H S Lelis","year":"2017","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/522"},{"position":11,"raw":"[Mariño et al., 2019] J. R. H. Mariño, R. O. Moraes, C. Toledo, and L. H. S. Lelis. Evolving action abstrac- tions for real-time planning in extensive-form games. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 2330-2337, 2019.","title":"Evolving action abstractions for real-time planning in extensive-form games","authors":"J R H Mariño, R O Moraes, C Toledo, L H S Lelis","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Moraes and Lelis, 2018] R. O. Moraes and L. H. S. Lelis. Asymmetric action abstractions for multi-unit control in adversarial real-time games. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 876-883. AAAI, 2018.","title":"Asymmetric action abstractions for multi-unit control in adversarial real-time games","authors":"R O Moraes, L H S Lelis","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Moraes et al., 2018a] R. O. Moraes, J. R. H. Mariño, L. H. S. Lelis, and M. A. Nascimento. Action abstrac- tions for combinatorial multi-armed bandit tree search. In Proceedings of the AAAI Conference on Artificial Intelli- gence and Interactive Digital Entertainment, pages 74-80. AAAI, 2018.","title":"Action abstractions for combinatorial multi-armed bandit tree search","authors":"Moraes","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Moraes et al., 2018b] R. O. Moraes, J. R. H. Mariño, and L. H. S. Lelis. Nested-greedy search for adversarial real- time games. In Proceedings of the AAAI Conference on Ar- tificial Intelligence and Interactive Digital Entertainment, pages 67-73, 2018.","title":"Nested-greedy search for adversarial realtime games","authors":"Moraes","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ontañón and Buro, 2015] S. Ontañón and M. Buro. Adver- sarial hierarchical-task network planning for complex real- time games. In Proceedings of the International Joint Con- ference on Artificial Intelligence, pages 1652-1658, 2015.","title":"Adversarial hierarchical-task network planning for complex realtime games","authors":"S Ontañón, M Buro","year":"2015","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ontañón et al., 2013] S. Ontañón, G. Synnaeve, A. Uriarte, F. Richoux, D. Churchill, and M. Preuss. A survey of real- time strategy game AI research and competition in Star- Craft. IEEE Transactions on Comp. Int. and AI in Games, 5(4):293-311, 2013.","title":"A survey of realtime strategy game AI research and competition in Star-Craft","authors":"S Ontañón, G Synnaeve, A Uriarte, F Richoux, D Churchill, M Preuss","year":"2013","venue":"IEEE Transactions on Comp. Int. and AI in Games","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ontañón, 2013] S. Ontañón. The combinatorial multi- armed bandit problem and its application to real-time strat- egy games. In Proceedings of the AAAI Conference on Ar- tificial Intelligence and Interactive Digital Entertainment, pages 58-64. AAAI, 2013.","title":"The combinatorial multiarmed bandit problem and its application to real-time strategy games","authors":"S Ontañón","year":"2013","venue":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ontañón, 2017] S. Ontañón. Combinatorial multi-armed bandits for real-time strategy games. Journal of Artificial Intelligence Research, 58:665-702, 2017.","title":"Combinatorial multi-armed bandits for real-time strategy games","authors":"S Ontañón","year":"2017","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vinyals et al., 2019] O. Vinyals, I. Babuschkin, W. M. Czar- necki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, J. Oh, D. Horgan, M. Kroiss, I. Danihelka, A. Huang, L. Sifre, T. Cai, J. P. Agapiou, M. Jaderberg, A. S. Vezhnevets, R. Leblond, T. Pohlen, V. Dalibard, D. Budden, Y. Sulsky, J. Mol- loy, T. L. Paine, C. Gulcehre, Z. Wang, T. Pfaff, Y. Wu, R. Ring, D. Yogatama, D. Wünsch, K. McKin- ney, O. Smith, T. Schaul, T. Lillicrap, K. Kavukcuoglu, D. Hassabis, C. Apps, and D. Silver. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782):350-354, 2019.","title":"Grandmaster level in StarCraft II using multi-agent reinforcement learning","authors":"O Vinyals, I Babuschkin, W M Czarnecki, M Mathieu, A Dudzik, J Chung, D H Choi, R Powell, T Ewalds, P Georgiev, J Oh, D Horgan, M Kroiss, I Danihelka, A Huang, L Sifre, T Cai, J P Agapiou, M Jaderberg, A S Vezhnevets, R Leblond, T Pohlen, V Dalibard, D Budden, Y Sulsky, J Molloy, T L Paine, C Gulcehre, Z Wang, T Pfaff, Y Wu, R Ring, D Yogatama, D Wünsch, K Mckinney, O Smith, T Schaul, T Lillicrap, K Kavukcuoglu, D Hassabis, C Apps, D Silver","year":"2019","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2016] C. Wang, P. Chen, Y. Li, C. Holmgård, and J. Togelius. Portfolio online evolution in StarCraft. In Proceedings of the Conference on Artificial Intelligence and Interactive Digital Entertainment, pages 114-120. AAAI, 2016.","title":"Portfolio online evolution in StarCraft","authors":"C Wang, P Chen, Y Li, C Holmgård, J Togelius","year":"2016","venue":"Proceedings of the Conference on Artificial Intelligence and Interactive Digital Entertainment","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ward and Cowling, 2009] C. D. Ward and P. I. Cowling. Monte carlo search applied to card selection in magic: The gathering. In Proceedings of the International Conference on Comp. Int. and Games, pages 9-16, 2009.","title":"Monte carlo search applied to card selection in magic: The gathering","authors":"C D Ward, P I Cowling","year":"2009","venue":"Proceedings of the International Conference on Comp. Int. and Games","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang and Ontañón, 2019] Z. Yang and S. Ontañón. Guiding Monte Carlo Tree Search by Scripts in Real-Time Strat- egy Games. In Proceedings of the Conference on Artificial Intelligence and Interactive Digital Entertainment, pages 100-107, 2019.","title":"Guiding Monte Carlo Tree Search by Scripts in Real-Time Strategy Games","authors":"Z Yang, S Ontañón","year":"2019","venue":"Proceedings of the Conference on Artificial Intelligence and Interactive Digital Entertainment","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}