{"type":"ScholarlyArticle","title":"Asynchronous Credit Assignment for Multi-Agent Reinforcement Learning","authors":[{"name":"Yongheng Liang","slug":"yongheng-liang","given":"Yongheng","family":"Liang","affiliation":""},{"name":"Hejun Wu","slug":"hejun-wu","given":"Hejun","family":"Wu","affiliation":""},{"name":"Haitao Wang","slug":"haitao-wang","given":"Haitao","family":"Wang","affiliation":""},{"name":"Hao Cai","slug":"hao-cai","given":"Hao","family":"Cai","affiliation":""}],"abstract":"Credit assignment is a critical problem in multi-agent reinforcement learning (MARL), aiming to identify agents' marginal contributions for optimizing cooperative policies. Current credit assignment methods typically assume synchronous decision-making among agents. However, many real-world scenarios require agents to act asynchronously without waiting for others. This asynchrony introduces conditional dependencies between actions, which pose great challenges to current methods. To address this issue, we propose an asynchronous credit assignment framework, incorporating a Virtual Synchrony Proxy (VSP) mechanism and a Multiplicative Value Decomposition (MVD) algorithm. VSP enables physically asynchronous actions to be virtually synchronized during credit assignment. We theoretically prove that VSP preserves both task equilibrium and algorithm convergence. Furthermore, MVD leverages multiplicative interactions to effectively model dependencies among asynchronous actions, offering theoretical advantages in handling asynchronous tasks. Extensive experiments show that our framework consistently outperforms state-of-the-art MARL methods on challenging tasks while providing improved interpretability for asynchronous cooperation.","doi":"10.24963/ijcai.2025/20","url":"https://ijcai.org/proceedings/2025/20","pdf":"https://ijcai.org/proceedings/2025/0020.pdf","markdown":"https://ijcai.org/proceedings/2025/20.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/20","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":20,"pages":{"first":170,"last":178},"keywords":["Machine Learning: ML: Reinforcement learning","Agent-based and Multi-agent Systems: MAS: Multi-agent learning","Game Theory and Economic Paradigms: GTEP: Cooperative games","Agent-based and Multi-agent Systems: MAS: Coordination and cooperation"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}