{"type":"ScholarlyArticle","title":"Mean Field Games Flock! The Reinforcement Learning Way","authors":[{"name":"Sarah Perrin","slug":"sarah-perrin","given":"Sarah","family":"Perrin","affiliation":""},{"name":"Mathieu Laurière","slug":"mathieu-lauriere","given":"Mathieu","family":"Laurière","affiliation":""},{"name":"Julien Pérolat","slug":"julien-perolat","given":"Julien","family":"Pérolat","affiliation":""},{"name":"Matthieu Geist","slug":"matthieu-geist","given":"Matthieu","family":"Geist","affiliation":""},{"name":"Romuald Élie","slug":"romuald-elie","given":"Romuald","family":"Élie","affiliation":""},{"name":"Olivier Pietquin","slug":"olivier-pietquin","given":"Olivier","family":"Pietquin","affiliation":""}],"abstract":"We present a method enabling a large number of agents to learn how to flock. This problem has drawn a lot of interest but requires many structural assumptions and is tractable only in small dimensions. We phrase this problem as a Mean Field Game (MFG),  where each individual chooses its own acceleration depending on the population behavior. Combining Deep Reinforcement Learning (RL) and Normalizing Flows (NF), we obtain a tractable solution requiring only very weak assumptions. Our algorithm finds a Nash Equilibrium and the agents adapt their velocity to match the neighboring flock’s average one. We use Fictitious Play and alternate: (1) computing an approximate best response with Deep RL, and (2) estimating the next population distribution with NF. We show numerically that our algorithm can learn multi-group or high-dimensional flocking with obstacles.","doi":"10.24963/ijcai.2021/50","url":"https://ijcai.org/proceedings/2021/50","pdf":"https://ijcai.org/proceedings/2021/0050.pdf","markdown":"https://ijcai.org/proceedings/2021/50.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/50","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":50,"pages":{"first":356,"last":362},"keywords":["Agent-based and Multi-agent Systems: Noncooperative Games","Machine Learning Applications: Applications of Reinforcement Learning","Agent-based and Multi-agent Systems: Multi-agent Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}