{"type":"ScholarlyArticle","title":"Learning Fair Cooperation in Mixed-Motive Games with Indirect Reciprocity","authors":[{"name":"Martin Smit","slug":"martin-smit","given":"Martin","family":"Smit","affiliation":"Informatics Institute , University of Amsterdam"},{"name":"Fernando P. Santos","slug":"fernando-p-santos","given":"Fernando P.","family":"Santos","affiliation":"Informatics Institute , University of Amsterdam"}],"abstract":"Altruistic cooperation is costly yet socially desirable. As a result, agents struggle to learn cooperative policies through independent reinforcement learning (RL). Indirect reciprocity, where agents consider their interaction partner’s reputation, has been shown to stabilise cooperation in homogeneous, idealised populations. However, more realistic settings are comprised of heterogeneous agents with different characteristics and group-based social identities. We study cooperation when agents are stratified into two such groups, and allow reputation updates and actions to depend on group information. We consider two modelling approaches: evolutionary game theory, where we comprehensively search for social norms (i.e., rules to assign reputations) leading to cooperation and fairness; and RL, where we consider how the stochastic dynamics of policy learning affects the analytically identified equilibria. We observe that a defecting majority leads the minority group to defect, but not the inverse. Moreover, changing the norms that judge inand out-group interactions can steer a system towards either fair or unfair cooperation. This is made clearer when moving beyond equilibrium analysis to independent RL agents, where convergence to fair cooperation occurs with a narrower set of norms. Our results highlight that, in heterogeneous populations with reputations, carefully defining interaction norms is fundamental to tackle both dilemmas of cooperation and of fairness.","doi":"10.24963/ijcai.2024/25","url":"https://ijcai.org/proceedings/2024/25","pdf":"https://ijcai.org/proceedings/2024/0025.pdf","markdown":"https://ijcai.org/proceedings/2024/25.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/25","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":25,"pages":{"first":220,"last":228},"keywords":["Agent-based and Multi-agent Systems: MAS: Coordination and cooperation","Agent-based and Multi-agent Systems: MAS: Agent-based simulation and emergence","Agent-based and Multi-agent Systems: MAS: Multi-agent learning"],"videos":[],"references":[{"position":1,"raw":"[Anastassacos et al., 2021] Nicolas Anastassacos, Julian García, Stephen Hailes, and Mirco Musolesi. Cooperation and reputation dynamics with reinforcement learning. In Proc. of the 20th International Conference on Autonomous Agents and MultiAgent Systems. IFAAMAS, February 2021.","title":"Cooperation and reputation dynamics with reinforcement learning","authors":"Nicolas Anastassacos, Julian García, Stephen Hailes, Mirco Musolesi","year":"2021","venue":"Proc. of the 20th International Conference on Autonomous Agents and MultiAgent Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anavankot et al., 2023] Amritha Menon Anavankot, Stephen Cranefield, and Bastin Tony Roy Savarimuthu. Towards norm entrepreneurship in agent societies. In Advances in Practical Applications of Agents, Multi-Agent Systems, and Cognitive Mimetics. The PAAMS Collection, Lecture Notes in Computer Science. Springer Nature Switzerland, 2023.","title":"Towards norm entrepreneurship in agent societies","authors":"Amritha Menon Anavankot, Stephen Cranefield, Bastin Tony, Roy Savarimuthu","year":"2023","venue":"Advances in Practical Applications of Agents, Multi-Agent Systems, and Cognitive Mimetics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bara et al., 2023] Jacques Bara, Fernando P. Santos, and Paolo Turrini. The role of space, density and migration in social dilemmas. In Proc. of the 2023 International Con- ference on Autonomous Agents and Multiagent Systems, AAMAS '23. IFAAMAS, May 2023.","title":"The role of space, density and migration in social dilemmas","authors":"Jacques Bara, Fernando P Santos, Paolo Turrini","year":"2023","venue":"Proc. of the 2023 International Conference on Autonomous Agents and Multiagent Systems, AAMAS '23","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chierichetti et al., 2017] Flavio Chierichetti, Ravi Kumar, Silvio Lattanzi, and Sergei Vassilvitskii. Fair clustering through fairlets. Advances in Neural Information Process- ing Systems, NeurIPS'17, 30, 2017.","title":"Fair clustering through fairlets","authors":"Flavio Chierichetti, Ravi Kumar, Silvio Lattanzi, Sergei Vassilvitskii","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Claus and Boutilier, 1998] Caroline Claus and Craig Boutilier. The dynamics of reinforcement learning in cooperative multiagent systems. In Proc. of the Fifteenth National Conference on Artificial Intelligence, AAAI '98. AAAI Press, 1998.","title":"The dynamics of reinforcement learning in cooperative multiagent systems","authors":"Claus, Boutilier, Caroline Claus, Craig Boutilier","year":"1998","venue":"Proc. of the Fifteenth National Conference on Artificial Intelligence, AAAI '98","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Conitzer and Oesterheld, 2023] Vincent Conitzer and Cas- par Oesterheld. Foundations of cooperative ai. Proc. of the AAAI Conference on Artificial Intelligence, 37, June 2023.","title":"Vincent Conitzer and Caspar Oesterheld","authors":"Oesterheld Conitzer","year":"2023","venue":"Proc. of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dafoe et al., 2021] Allan Dafoe, Yoram Bachrach, Gillian Hadfield, Eric Horvitz, Kate Larson, and Thore Grae- pel. Cooperative ai: Machines must learn to find common ground. Nature, 593, May 2021.","title":"Cooperative ai: Machines must learn to find common ground","authors":"Allan Dafoe, Yoram Bachrach, Gillian Hadfield, Eric Horvitz, Kate Larson, Thore Graepel","year":"2021","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[De et al., 2017] Soham De, Dana S Nau, and Michele J Gelfand. Understanding norm change: An evolutionary game-theoretic approach. In Proc. of the 16th Conference on Autonomous Agents and Multiagent Systems, AAMAS 2017, 2017.","title":"Understanding norm change: An evolutionary game-theoretic approach","authors":"Soham De, Dana S Nau, Michele J Gelfand","year":"2017","venue":"Proc. of the 16th Conference on Autonomous Agents and Multiagent Systems, AAMAS 2017","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Edelman et al., 2017] Benjamin Edelman, Michael Luca, and Dan Svirsky. Racial discrimination in the sharing economy: Evidence from a field experiment. American Economic Journal: Applied Economics, 9, April 2017.","title":"Racial discrimination in the sharing economy: Evidence from a field experiment","authors":"Benjamin Edelman, Michael Luca, Dan Svirsky","year":"2017","venue":"American Economic Journal: Applied Economics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fatima et al., 2024] Shaheen Fatima, Nicholas R. Jennings, and Michael Wooldridge. Learning to resolve social dilemmas: A survey. Journal of Artificial Intelligence Re- search, 79, March 2024.","title":"Learning to resolve social dilemmas: A survey","authors":"Shaheen Fatima, Nicholas R Jennings, Michael Wooldridge","year":"2024","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fehr and Fischbacher, 2003] Ernst Fehr and Urs Fis- chbacher. The nature of human altruism. Nature, 425, October 2003.","title":"Ernst Fehr and Urs Fischbacher","authors":"Fischbacher Fehr","year":"2003","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fu et al., 2012] Feng Fu, Martin A. Nowak, Nicholas A. Christakis, and James H. Fowler. The evolution of ho- mophily. Scientific Reports, 2, November 2012.","title":"The evolution of homophily","authors":"Feng Fu, Martin A Nowak, Nicholas A Christakis, James H Fowler","year":"2012","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Genesereth et al., 1986] Michael R. Genesereth, Matthew L. Ginsberg, and Jeffrey S. Rosenschein. Cooperation without communication. In Proc. of the Fifth AAAI National Conference on Artificial Intelligence, AAAI'86. AAAI Press, 1986.","title":"Cooperation without communication","authors":"R Michael, Matthew L Genesereth, Jeffrey S Ginsberg, Rosenschein","year":"1986","venue":"Proc. of the Fifth AAAI National Conference on Artificial Intelligence, AAAI'86","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Griffith, 2010] Meghan Griffith. Why agent-caused actions are not lucky. American Philosophical Quarterly, 47, 2010.","title":"Why agent-caused actions are not lucky","authors":"Meghan Griffith","year":"2010","venue":"American Philosophical Quarterly","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gross and De Dreu, 2019] Jörg Gross and Carsten K. W. De Dreu. The rise and fall of cooperation through rep- utation and group polarization. Nature Communications, 10, February 2019.","title":"The rise and fall of cooperation through reputation and group polarization","authors":"De Gross, ; Dreu, Jörg Gross, Carsten K W De Dreu","year":"2019","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Guo et al., 2023] Hao Guo, Chen Shen, Shuyue Hu, Jun- liang Xing, Pin Tao, Yuanchun Shi, and Zhen Wang. Fa- cilitating cooperation in human-agent hybrid populations through autonomous agents. iScience, 26, 2023.","title":"Facilitating cooperation in human-agent hybrid populations through autonomous agents","authors":"Chen Hao Guo, Shuyue Shen, Junliang Hu, Pin Xing, Yuanchun Tao, Zhen Shi, Wang","year":"2023","venue":"iScience","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Haynes et al., 2017] Chris Haynes, Michael Luck, Peter McBurney, Samhar Mahmoud, Tomáš Vítek, and Simon Miles. Engineering the emergence of norms: A review. The Knowledge Engineering Review, 32, January 2017.","title":"Engineering the emergence of norms: A review","authors":"Chris Haynes, Michael Luck, Peter Mcburney, Samhar Mahmoud, Tomáš Vítek, Simon Miles","year":"2017","venue":"The Knowledge Engineering Review","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hilbe et al., 2018] Christian Hilbe, Laura Schmid, Josef Tkadlec, Krishnendu Chatterjee, and Martin A. Nowak. Indirect reciprocity with private, noisy, and incomplete in- formation. Proc. of the National Academy of Sciences, 115, November 2018.","title":"Indirect reciprocity with private, noisy, and incomplete information","authors":"Hilbe","year":"2018","venue":"Proc. of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Hu and Leung, 2017] Shuyue Hu and Ho-Fung Leung. Lo- cal norm phenomena in multi-agent systems under com- munity networks. In Proc. of the 16th Conference on Autonomous Agents and Multiagent Systems, AAMAS'17, 2017.","title":"Local norm phenomena in multi-agent systems under community networks","authors":"Leung Hu, Shuyue Hu, Ho-Fung Leung","year":"2017","venue":"Proc. of the 16th Conference on Autonomous Agents and Multiagent Systems, AAMAS'17","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Hughes et al., 2018] Edward Hughes, Joel Z Leibo, Matthew Phillips, Karl Tuyls, Edgar Dueñez-Guzman, Antonio García Castañeda, Iain Dunning, Tina Zhu, Kevin McKee, Raphael Koster, Heather Roff, and Thore Graepel. Inequity aversion improves cooperation in intertemporal social dilemmas. In Advances in Neural Information Processing Systems, volume 31.","title":"Inequity aversion improves cooperation in intertemporal social dilemmas","authors":"Edward Hughes, Joel Z Leibo, Matthew Phillips, Karl Tuyls, Edgar Dueñez-Guzman, Antonio García Castañeda, Iain Dunning, Tina Zhu, Kevin Mckee, Raphael Koster, Heather Roff, Thore Graepel","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"Curran Associates, Inc., 2018.","title":null,"authors":"Curran Associates, Inc","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Jaques et al., 2019] Natasha Jaques, Angeliki Lazari- dou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, Dj Strouse, Joel Z. Leibo, and Nando De Freitas. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In Proc. of the 36th International Conference on Machine Learning. PMLR, May 2019.","title":"Social influence as intrinsic motivation for multi-agent deep reinforcement learning","authors":"Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, Dj Strouse, Joel Z Leibo, Nando De Freitas","year":"2019","venue":"Proc. of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kawakatsu et al., 2024] Mari Kawakatsu, Taylor A Kessinger, and Joshua B Plotkin. A mechanistic model of gossip, reputations, and cooperation. Proceedings of the National Academy of Sciences, 121, 2024.","title":"A mechanistic model of gossip, reputations, and cooperation","authors":"Mari Kawakatsu, Taylor A Kessinger, Joshua B Plotkin","year":"2024","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kessinger et al., 2023] Taylor A. Kessinger, Corina E. Tar- nita, and Joshua B. Plotkin. Evolution of norms for judg- ing social behavior. Proc. of the National Academy of Sci- ences, 120, June 2023.","title":"Evolution of norms for judging social behavior","authors":"Taylor A Kessinger, Corina E Tarnita, Joshua B Plotkin","year":"2023","venue":"Proc. of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Krellner and Han, 2022] Marcus Krellner and The Anh Han. Pleasing enhances indirect reciprocity-based cooper- ation under private assessment. Artificial Life, 27, March 2022.","title":"Marcus Krellner and The Anh Han. Pleasing enhances indirect reciprocity-based cooperation under private assessment","authors":"Han Krellner","year":"2022","venue":"Artificial Life","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lupu and Precup, 2020] Andrei Lupu and Doina Precup. Gifting in multi-agent reinforcement learning. In Proc. of the 19th International Conference on autonomous agents and multiagent systems, 2020.","title":"Gifting in multi-agent reinforcement learning","authors":"Precup Lupu, Andrei Lupu, Doina Precup","year":"2020","venue":"Proc. of the 19th International Conference on autonomous agents and multiagent systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Mehrabi et al., 2021] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys, 54, July 2021.","title":"A survey on bias and fairness in machine learning","authors":"Mehrabi","year":"2021","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Morales et al., 2013] Javier Morales, Maite Lopez- Sanchez, Juan A. Rodriguez-Aguilar, Michael Wooldridge, and Wamberto Vasconcelos. Automated synthesis of normative systems. In Proc. of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems, May 2013.","title":"Automated synthesis of normative systems","authors":"Javier Morales, Maite Lopez-Sanchez, Juan A Rodriguez-Aguilar, Michael Wooldridge, Wamberto Vasconcelos","year":"2013","venue":"Proc. of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Nowak and Sigmund, 2005] Martin A. Nowak and Karl Sigmund. Evolution of indirect reciprocity. Nature, 437, October 2005.","title":"Evolution of indirect reciprocity","authors":"Martin A Nowak, Karl Sigmund","year":"2005","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ohtsuki and Iwasa, 2004] Hisashi Ohtsuki and Yoh Iwasa. How should we define goodness?-reputation dynamics in indirect reciprocity. Journal of Theoretical Biology, 231, November 2004.","title":"How should we define goodness?-reputation dynamics in indirect reciprocity","authors":"Iwasa ; ] Hisashi Ohtsuki, Yoh Ohtsuki, Iwasa","year":"2004","venue":"Journal of Theoretical Biology","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Ohtsuki and Iwasa, 2006] Hisashi Ohtsuki and Yoh Iwasa. The leading eight: Social norms that can maintain cooper- ation by indirect reciprocity. Journal of Theoretical Biol- ogy, 239, April 2006.","title":"The leading eight: Social norms that can maintain cooperation by indirect reciprocity","authors":"Iwasa ; ] Hisashi Ohtsuki, Yoh Ohtsuki, Iwasa","year":"2006","venue":"Journal of Theoretical Biology","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Ohtsuki and Iwasa, 2007] Hisashi Ohtsuki and Yoh Iwasa. Global analyses of evolutionary dynamics and exhaustive search for social norms that maintain cooperation by rep- utation. Journal of Theoretical Biology, 244, February 2007.","title":"Global analyses of evolutionary dynamics and exhaustive search for social norms that maintain cooperation by reputation","authors":"Iwasa ; ] Hisashi Ohtsuki, Yoh Ohtsuki, Iwasa","year":"2007","venue":"Journal of Theoretical Biology","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Okada, 2020] Isamu Okada. A review of theoretical studies on indirect reciprocity. Games, 11, September 2020.","title":"A review of theoretical studies on indirect reciprocity","authors":"Isamu Okada","year":"2020","venue":"Games","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Pacheco et al., 2006] Jorge M. Pacheco, Francisco C. San- tos, and Fabio A. C. C. Chalub. Stern-judging: A simple, successful norm which promotes cooperation under indi- rect reciprocity. PLoS Computational Biology, 2, Decem- ber 2006.","title":"Stern-judging: A simple, successful norm which promotes cooperation under indirect reciprocity","authors":"Jorge M Pacheco, Francisco C Santos, Fabio A C C Chalub","year":"2006","venue":"PLoS Computational Biology","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Paiva et al., 2018] Ana Paiva, Fernando P. Santos, and Fran- cisco C. Santos. Engineering pro-sociality with au- tonomous agents. Proc. of the AAAI Conference on Ar- tificial Intelligence, 32, April 2018.","title":"Engineering pro-sociality with autonomous agents","authors":"Ana Paiva, Fernando P Santos, Francisco C Santos","year":"2018","venue":"Proc. of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Panchanathan and Boyd, 2003] Karthik Panchanathan and Robert Boyd. A tale of two defectors: The importance of standing for evolution of indirect reciprocity. Journal of Theoretical Biology, 224, October 2003.","title":"A tale of two defectors: The importance of standing for evolution of indirect reciprocity","authors":"Karthik Panchanathan, Robert Boyd","year":"2003","venue":"Journal of Theoretical Biology","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Peleteiro et al., 2014] Ana Peleteiro, Juan C. Burguillo, and Siang Yew Chong. Exploring indirect reciprocity in com- plex networks using coalitions and rewiring. In Proc. of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS '14. IFAAMAS, 2014.","title":"Exploring indirect reciprocity in complex networks using coalitions and rewiring","authors":"Ana Peleteiro, Juan C Burguillo, Siang Yew, Chong","year":"2014","venue":"Proc. of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS '14","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Pennisi, 2005] Elizabeth Pennisi. How did cooperative be- havior evolve? Science (New York, N.Y.), 309, July 2005.","title":"How did cooperative behavior evolve? Science","authors":"Elizabeth Pennisi","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Rand and Nowak, 2013] David G. Rand and Martin A. Nowak. Human cooperation. Trends in Cognitive Sci- ences, 17, 2013.","title":"Human cooperation","authors":"David G Rand, Martin A Nowak","year":"2013","venue":"Trends in Cognitive Sciences","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Rosenblat et al., 2017] Alex Rosenblat, Karen E.C. Levy, Solon Barocas, and Tim Hwang. Discriminating tastes: Uber's customer ratings as vehicles for workplace discrim- ination. Policy & Internet, 9, 2017.","title":"Discriminating tastes: Uber's customer ratings as vehicles for workplace discrimination","authors":"Alex Rosenblat, Karen E C Levy, Solon Barocas, Tim Hwang","year":"2017","venue":"Policy & Internet","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Santos et al., 2018] Fernando P. Santos, Jorge M. Pacheco, and Francisco C. Santos. Social norms of cooperation with costly reputation building. Proc. of the AAAI Conference on Artificial Intelligence, 32, April 2018.","title":"Social norms of cooperation with costly reputation building","authors":"Fernando P Santos, Jorge M Pacheco, Francisco C Santos","year":"2018","venue":"Proc. of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Santos et al., 2019] Fernando P. Santos, Jorge M. Pacheco, Ana Paiva, and Francisco C. Santos. Evolution of collec- tive fairness in hybrid populations of humans and agents. In Proc. of the Thirty-Third AAAI Conference on Artificial Intelligence, AAAI'19. AAAI Press, 2019.","title":"Evolution of collective fairness in hybrid populations of humans and agents","authors":"Fernando P Santos, Jorge M Pacheco, Ana Paiva, Francisco C Santos","year":"2019","venue":"Proc. of the Thirty-Third AAAI Conference on Artificial Intelligence, AAAI'19","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Santos et al., 2021] Fernando P. Santos, Jorge M. Pacheco, and Francisco C. Santos. The complexity of human co- operation under indirect reciprocity. Philosophical Trans- actions of the Royal Society B: Biological Sciences, 376, 2021.","title":"The complexity of human cooperation under indirect reciprocity","authors":"Fernando P Santos, Jorge M Pacheco, Francisco C Santos","year":"2021","venue":"Philosophical Transactions of the Royal Society B: Biological Sciences","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Savarimuthu and Cranefield, 2011] Bastin Tony Roy Savarimuthu and Stephen Cranefield. Norm creation, spreading and emergence: A survey of simulation models of norms in multi-agent systems. Multiagent and Grid Systems, 7, January 2011.","title":"Norm creation, spreading and emergence: A survey of simulation models of norms in multi-agent systems","authors":"Bastin Tony, Roy Savarimuthu, Stephen Cranefield","year":"2011","venue":"Multiagent and Grid Systems","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Sen and Airiau, 2007] Sandip Sen and Stéphane Airiau. Emergence of norms through social learning. In Proc. of the 20th International Joint Conference on Artifical In- telligence, IJCAI'07. Morgan Kaufmann Publishers Inc., 2007.","title":"Emergence of norms through social learning","authors":"Sandip Sen, Stéphane Airiau","year":"2007","venue":"Proc. of the 20th International Joint Conference on Artifical Intelligence, IJCAI'07","doi":null,"url":null,"internal_url":"/proceedings/2007/243"},{"position":46,"raw":"[Shoham and Tennenholtz, 1997] Yoav Shoham and Moshe Tennenholtz. On the emergence of social conventions: Modeling, analysis, and simulations. Artificial Intelli- gence, 94, July 1997.","title":"On the emergence of social conventions: Modeling, analysis, and simulations","authors":"Tennenholtz Shoham, Yoav Shoham, Moshe Tennenholtz","year":"1997","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Smit and Santos, 2024] Jacobus Smit and Fernando P. San- tos. Fairness and cooperation between independent rein- forcement learners through indirect reciprocity. In Proc. of the 23rd International Conference on Autonomous Agents and Multiagent Systems, 2024.","title":"Fairness and cooperation between independent reinforcement learners through indirect reciprocity","authors":"Jacobus Smit, Fernando P Santos","year":"2024","venue":"Proc. of the 23rd International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Stewart and Raihani, 2023] Alexander J. Stewart and Nichola Raihani. Group reciprocity and the evolution of stereotyping. Proc. of the Royal Society B: Biological Sciences, 290, 2023.","title":"Group reciprocity and the evolution of stereotyping","authors":"Alexander J Stewart, Nichola Raihani","year":"2023","venue":"Proc. of the Royal Society B: Biological Sciences","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Villatoro et al., 2010] Daniel Villatoro, Sandip Sen, and Jordi Sabater-Mir. Of social norms and sanctioning: A game theoretical overview. International Journal of Agent Technologies and Systems (IJATS), 2, January 2010.","title":"Of social norms and sanctioning: A game theoretical overview","authors":"Daniel Villatoro, Sandip Sen, Jordi Sabater-Mir","year":"2010","venue":"International Journal of Agent Technologies and Systems (IJATS)","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Vinitsky et al., 2023] Eugene Vinitsky, Raphael Köster, John P Agapiou, Edgar A Duéñez-Guzmán, Alexander S Vezhnevets, and Joel Z Leibo. A learning agent that ac- quires social norms from public sanctions in decentral- ized multi-agent settings. Collective Intelligence, 2, April 2023.","title":"A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings","authors":"Eugene Vinitsky, Raphael Köster, John P Agapiou, Edgar A Duéñez-Guzmán, Alexander S Vezhnevets, Joel Z Leibo","year":"2023","venue":"Collective Intelligence","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Xu et al., 2019] Jason Xu, Julian García, and Toby Hand- field. Cooperation with bottom-up reputation dynamics. In Proc. of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '19. IFAAMAS, May 2019.","title":"Cooperation with bottom-up reputation dynamics","authors":"Jason Xu, Julian García, Toby Handfield","year":"2019","venue":"Proc. of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '19","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}