Emergent Prosociality in Multi-Agent Games Through Gifting

Wang, Woodrow Z.; Beliaev, Mark; Bıyık, Erdem; Lazar, Daniel A.; Pedarsani, Ramtin; Sadigh, Dorsa

doi:10.24963/ijcai.2021/61

Emergent Prosociality in Multi-Agent Games Through Gifting

Woodrow Z. Wang, Mark Beliaev, Erdem Bıyık, Daniel A. Lazar, Ramtin Pedarsani, Dorsa Sadigh

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 434-442.

https://doi.org/10.24963/ijcai.2021/61

Keywords

Agent-based and Multi-agent Systems: Coordination and Cooperation
Agent-based and Multi-agent Systems: Multi-agent Learning
Agent-based and Multi-agent Systems: Noncooperative Games

Abstract

Coordination is often critical to forming prosocial behaviors -- behaviors that increase the overall sum of rewards received by all agents in a multi-agent game. However, state of the art reinforcement learning algorithms often suffer from converging to socially less desirable equilibria when multiple equilibria exist. Previous works address this challenge with explicit reward shaping, which requires the strong assumption that agents can be forced to be prosocial. We propose using a less restrictive peer-rewarding mechanism, gifting, that guides the agents toward more socially desirable equilibria while allowing agents to remain selfish and decentralized. Gifting allows each agent to give some of their reward to other agents. We employ a theoretical framework that captures the benefit of gifting in converging to the prosocial equilibrium by characterizing the equilibria's basins of attraction in a dynamical system. With gifting, we demonstrate increased convergence of high risk, general-sum coordination games to the prosocial equilibrium both via numerical analysis and experiments.

Woodrow Z. Wang, Mark Beliaev, Erdem Bıyık, Daniel A. Lazar, Ramtin Pedarsani, and Dorsa Sadigh. 2021. Emergent Prosociality in Multi-Agent Games Through Gifting. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 434–442. https://doi.org/10.24963/ijcai.2021/61

@inproceedings{ijcai2021p61,
  title     = {Emergent Prosociality in Multi-Agent Games Through Gifting},
  author    = {Wang, Woodrow Z. and Beliaev, Mark and Bıyık, Erdem and Lazar, Daniel A. and Pedarsani, Ramtin and Sadigh, Dorsa},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Prof. Zhi-Hua Zhou},
  pages     = {434--442},
  year      = {2021},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2021/61},
  url       = {https://doi.org/10.24963/ijcai.2021/61},
}

TY  - CPAPER
TI  - Emergent Prosociality in Multi-Agent Games Through Gifting
AU  - Wang, Woodrow Z.
AU  - Beliaev, Mark
AU  - Bıyık, Erdem
AU  - Lazar, Daniel A.
AU  - Pedarsani, Ramtin
AU  - Sadigh, Dorsa
T2  - Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2021
SP  - 434
EP  - 442
DO  - 10.24963/ijcai.2021/61
UR  - https://doi.org/10.24963/ijcai.2021/61
ER  -