Mean Field Games Flock! The Reinforcement Learning Way

Perrin, Sarah; Laurière, Mathieu; Pérolat, Julien; Geist, Matthieu; Élie, Romuald; Pietquin, Olivier

doi:10.24963/ijcai.2021/50

Mean Field Games Flock! The Reinforcement Learning Way

Sarah Perrin, Mathieu Laurière, Julien Pérolat, Matthieu Geist, Romuald Élie, Olivier Pietquin

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 356-362.

https://doi.org/10.24963/ijcai.2021/50

Keywords

Agent-based and Multi-agent Systems: Noncooperative Games
Machine Learning Applications: Applications of Reinforcement Learning
Agent-based and Multi-agent Systems: Multi-agent Learning

Abstract

We present a method enabling a large number of agents to learn how to flock. This problem has drawn a lot of interest but requires many structural assumptions and is tractable only in small dimensions. We phrase this problem as a Mean Field Game (MFG), where each individual chooses its own acceleration depending on the population behavior. Combining Deep Reinforcement Learning (RL) and Normalizing Flows (NF), we obtain a tractable solution requiring only very weak assumptions. Our algorithm finds a Nash Equilibrium and the agents adapt their velocity to match the neighboring flock’s average one. We use Fictitious Play and alternate: (1) computing an approximate best response with Deep RL, and (2) estimating the next population distribution with NF. We show numerically that our algorithm can learn multi-group or high-dimensional flocking with obstacles.

Sarah Perrin, Mathieu Laurière, Julien Pérolat, Matthieu Geist, Romuald Élie, and Olivier Pietquin. 2021. Mean Field Games Flock! The Reinforcement Learning Way. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 356–362. https://doi.org/10.24963/ijcai.2021/50

@inproceedings{ijcai2021p50,
  title     = {Mean Field Games Flock! The Reinforcement Learning Way},
  author    = {Perrin, Sarah and Laurière, Mathieu and Pérolat, Julien and Geist, Matthieu and Élie, Romuald and Pietquin, Olivier},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Prof. Zhi-Hua Zhou},
  pages     = {356--362},
  year      = {2021},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2021/50},
  url       = {https://doi.org/10.24963/ijcai.2021/50},
}

TY  - CPAPER
TI  - Mean Field Games Flock! The Reinforcement Learning Way
AU  - Perrin, Sarah
AU  - Laurière, Mathieu
AU  - Pérolat, Julien
AU  - Geist, Matthieu
AU  - Élie, Romuald
AU  - Pietquin, Olivier
T2  - Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2021
SP  - 356
EP  - 362
DO  - 10.24963/ijcai.2021/50
UR  - https://doi.org/10.24963/ijcai.2021/50
ER  -