{"type":"ScholarlyArticle","title":"Enhancing Policy Gradient Algorithms with Search in Imperfect Information Games","authors":[{"name":"Ondřej Kubíček","slug":"ondrej-kubicek","given":"Ondřej","family":"Kubíček","affiliation":"Czech Technical University in Prague , FEE , Dept. of CS ,; Artificial Intelligence Center"}],"abstract":"Sequential decision-making under uncertainty in multi-agent environments is a fundamental problem in artificial intelligence. Games serve as a base model for these problems. Finding optimal plans in games that model real-world scenarios necessitates scalable algorithms. In games with perfect information, algorithms that use a combination of search and deep reinforcement learning can scale to arbitrary-sized games and achieve superhuman performance. In games with imperfect information, the situation is more challenging due to the nature of the search. This work aims to develop algorithms that use search but can scale into larger games than currently possible.","doi":"10.24963/ijcai.2024/964","url":"https://ijcai.org/proceedings/2024/964","pdf":"https://ijcai.org/proceedings/2024/0964.pdf","markdown":"https://ijcai.org/proceedings/2024/964.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/964","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Doctoral Consortium","track":"Doctoral Consortium","number":964,"pages":{"first":8498,"last":8499},"keywords":["DC: Game Theory and Economic Paradigms","DC: Agent-based and Multi-agent Systems","DC: Search","DC: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Brown and Sandholm, 2018] Noam Brown and Tuomas Sandholm. Superhuman ai for heads-up no-limit poker: Libratus beats top professionals. Science, 359(6374):418- 424, 2018.","title":"Noam Brown and Tuomas Sandholm. Superhuman ai for heads-up no-limit poker: Libratus beats top professionals","authors":"Sandholm Brown","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brown et al., 2018] Noam Brown, Tuomas Sandholm, and Brandon Amos. Depth-limited solving for imperfect- information games. NeurIPS, 31, 2018.","title":"Depth-limited solving for imperfectinformation games","authors":"Brown","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Kubíček et al., 2024] Ondřej Kubíček, Neil Burch, and Vil- iam Lisý. Look-ahead search on top of policy networks in imperfect information games, 2024.","title":"Look-ahead search on top of policy networks in imperfect information games","authors":"Ondřej Kubíček, Neil Burch, Viliam Lisý","year":"2024","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2024/480"},{"position":4,"raw":"[Moravčík et al., 2017] Matej Moravčík, Martin Schmid, Neil Burch, et al. Deepstack: Expert-level artifi- cial intelligence in heads-up no-limit poker. Science, 356(6337):508-513, 2017.","title":"Deepstack: Expert-level artificial intelligence in heads-up no-limit poker","authors":"Matej Moravčík, Martin Schmid, Neil Burch","year":"2017","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Perolat et al., 2022] Julien Perolat, Bart De Vylder, Daniel Hennes, et al. Mastering the game of stratego with model-free multiagent reinforcement learning. Science, 378(6623):990-996, 2022.","title":"Mastering the game of stratego with model-free multiagent reinforcement learning","authors":"Julien Perolat, Bart De Vylder, Daniel Hennes","year":"2022","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Schmid et al., 2023] Martin Schmid, Matej Moravčík, Neil Burch, et al. Student of games: A unified learning algo- rithm for both perfect and imperfect information games. Science Advances, 9(46):eadg3256, 2023.","title":"Student of games: A unified learning algorithm for both perfect and imperfect information games","authors":"Martin Schmid, Matej Moravčík, Neil Burch","year":"2023","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Schrittwieser et al., 2020] Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, et al. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, 588(7839):604-609, 2020.","title":"Mastering Atari, Go, chess and shogi by planning with a learned model","authors":"Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert","year":"2020","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Wang et al., 2022] Tony Tong Wang, Adam Gleave, Nora Belrose, et al. Adversarial policies beat professional- level go ais. In Deep Reinforcement Learning Workshop NeurIPS 2022, 2022.","title":"Adversarial policies beat professionallevel go ais","authors":"Tony Tong, Wang, Adam Gleave, Nora Belrose","year":"2022","venue":"Deep Reinforcement Learning Workshop NeurIPS 2022","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Čermák et al., 2020] Jiří Čermák, Viliam Lisý, and Branislav Bošanský. Automated construction of bounded- loss imperfect-recall abstractions in extensive-form games. Artificial Intelligence, 282:103248, 2020.","title":"Automated construction of boundedloss imperfect-recall abstractions in extensive-form games","authors":"Čermák","year":"2020","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}