{"type":"ScholarlyArticle","title":"Temporal Induced Self-Play for Stochastic Bayesian Games","authors":[{"name":"Weizhe Chen","slug":"weizhe-chen","given":"Weizhe","family":"Chen","affiliation":""},{"name":"Zihan Zhou","slug":"zihan-zhou","given":"Zihan","family":"Zhou","affiliation":""},{"name":"Yi Wu","slug":"yi-wu","given":"Yi","family":"Wu","affiliation":""},{"name":"Fei Fang","slug":"fei-fang","given":"Fei","family":"Fang","affiliation":""}],"abstract":"One practical requirement in solving dynamic games is to ensure that the players play well from any decision point onward. To satisfy this requirement, existing efforts focus on equilibrium refinement, but the scalability and applicability of existing techniques are limited. In this paper, we propose Temporal-Induced Self-Play (TISP), a novel reinforcement learning-based framework to find strategies with decent performances from any decision point onward. TISP uses belief-space representation, backward induction, policy learning, and non-parametric approximation. Building upon TISP, we design a policy-gradient-based algorithm TISP-PG. We prove that TISP-based algorithms can find approximate Perfect Bayesian Equilibrium in zero-sum one-sided stochastic Bayesian games with finite horizon. We test TISP-based algorithms in various games, including finitely repeated security games and a grid-world game. The results show that TISP-PG is more scalable than existing mathematical programming-based methods and significantly outperforms other learning-based methods.","doi":"10.24963/ijcai.2021/14","url":"https://ijcai.org/proceedings/2021/14","pdf":"https://ijcai.org/proceedings/2021/0014.pdf","markdown":"https://ijcai.org/proceedings/2021/14.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/14","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":14,"pages":{"first":96,"last":103},"keywords":["Agent-based and Multi-agent Systems: Multi-agent Learning","Machine Learning Applications: Applications of Reinforcement Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}