{"type":"ScholarlyArticle","title":"Constant-Memory Strategies in Stochastic Games: A Theoretical and Empirical Study","authors":[{"name":"Fengming Zhu","slug":"fengming-zhu","given":"Fengming","family":"Zhu","affiliation":""},{"name":"Fangzhen Lin","slug":"fangzhen-lin","given":"Fangzhen","family":"Lin","affiliation":""}],"abstract":"Stochastic games have become a prevalent framework for studying long-term multi-agent interactions, especially in the context of multi-agent reinforcement learning.\r\nIn this work, we comprehensively investigate the concept of constant-memory strategies in stochastic games.\r\nWe first establish some results on best responses and Nash equilibria for behavioral constant-memory strategies, followed by a discussion on the computational hardness of best responding to mixed constant-memory strategies.\r\nThose theoretic insights are later verified on several sequential decision-making testbeds, including the Iterated Prisoner's Dilemma, the Iterated Traveler's Dilemma, and the Pursuit domain.\r\nThis work aims to enhance the understanding of theoretical issues in single-agent planning under multi-agent systems, and uncover the connection between decision models in single-agent and multi-agent contexts.\r\nThe codebase and the full version of this paper is available at github.com/Fernadoo/Const-Mem.","doi":"10.24963/ijcai.2026/56","url":"https://ijcai.org/proceedings/2026/56","pdf":"https://ijcai.org/proceedings/2026/0056.pdf","markdown":"https://ijcai.org/proceedings/2026/56.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/56","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":56,"pages":{"first":490,"last":498},"keywords":["Agent-based and Multi-agent Systems: Agent theories and models","Planning and Scheduling: Planning with Incomplete Information","Game Theory and Economic Paradigms: Noncooperative games","Machine Learning: Partially observable reinforcement learning and POMDPs"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}