{"type":"ScholarlyArticle","title":"Payoff Control in the Iterated Prisoner's Dilemma","authors":[{"name":"Dong Hao","slug":"dong-hao","given":"Dong","family":"Hao","affiliation":"University of Electronic Science and Technology of China , Chengdu , China"},{"name":"Kai Li","slug":"kai-li","given":"Kai","family":"Li","affiliation":"Shanghai Jiao Tong University , Shanghai , China"},{"name":"Tao Zhou","slug":"tao-zhou","given":"Tao","family":"Zhou","affiliation":"University of Electronic Science and Technology of China , Chengdu , China"}],"abstract":"Repeated game has long been the touchstone model for agents’ long-run relationships. Previous results suggest that it is particularly difficult for a repeated game player to exert an autocratic control on the payoffs since they are jointly determined by all participants. This work discovers that the scale of a player’s capability to unilaterally influence the payoffs may have been much underestimated. Under the conventional iterated prisoner’s dilemma, we develop a general framework for controlling the feasible region where the players’ payoff pairs lie. A control strategy player is able to confine the payoff pairs in her objective region, as long as this region has feasible linear boundaries. With this framework, many well-known existing strategies can be categorized and various new strategies with nice properties can be further identified. We show that the control strategies perform well either in a tournament or against a human-like opponent.","doi":"10.24963/ijcai.2018/41","url":"https://ijcai.org/proceedings/2018/41","pdf":"https://ijcai.org/proceedings/2018/0041.pdf","markdown":"https://ijcai.org/proceedings/2018/41.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/41","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Agent-based and Multi-agent Systems","track":"Main track","number":41,"pages":{"first":296,"last":302},"keywords":["Agent-based and Multi-agent Systems: Coordination and Cooperation","Agent-based and Multi-agent Systems: Noncooperative Games"],"videos":[],"references":[{"position":1,"raw":"[Adami and Hintze, 2013] Christoph Adami and Arend Hintze. Evolutionary instability of zero-determinant strategies demonstrates that winning is not everything. Nature Communications, 4:2193, 2013.","title":"Evolutionary instability of zero-determinant strategies demonstrates that winning is not everything","authors":"Hintze Adami, Christoph Adami, Arend Hintze","year":"2013","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Akin, 2012] Ethan Akin. Stable cooperative solutions for the iterated prisoner's dilemma. arXiv preprint, 1211.0969, 2012.","title":"Stable cooperative solutions for the iterated prisoner's dilemma","authors":"Ethan Akin, Akin","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Axelrod and Hamilton, 1981] Robert Axelrod and William Donald Hamilton. The evolution of cooper- ation. Science, 211(4489):1390-1396, 1981.","title":"The evolution of cooperation","authors":"Robert Axelrod, William Donald, Hamilton","year":"1981","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boerlijst et al., 1997] Maarten Boerlijst, Martin A Nowak, and Karl Sigmund. Equal pay for all prisoners. The Amer- ican Mathematical Monthly, 104(4):303-305, 1997.","title":"Maarten Boerlijst, Martin A","authors":"Boerlijst","year":"1997","venue":"The American Mathematical Monthly","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen and Zinger, 2014] Jing Chen and Aleksey Zinger. The robustness of zero-determinant strategies in iterated pris- oner's dilemma games. Journal of Theoretical Biology, 357:46-54, 2014.","title":"The robustness of zero-determinant strategies in iterated prisoner's dilemma games","authors":"Chen, Zinger ; ] Jing Chen, Aleksey Zinger","year":"2014","venue":"Journal of Theoretical Biology","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Claus and Boutilier, 1998] Caroline Claus and Craig Boutilier. The dynamics of reinforcement learning in cooperative multiagent systems. In Proceedings of the 15th International Conference on Artificial Intelligence, 1998:746-752, 1998.","title":"The dynamics of reinforcement learning in cooperative multiagent systems","authors":"Claus, Boutilier, Caroline Claus, Craig Boutilier","year":"1998","venue":"Proceedings of the 15th International Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Crandall et al., 2018] Jacob W Crandall, Mayada Oudah, Fatimah Ishowo-Oloko, Sherief Abdallah, Jean-Franc ¸ois Bonnefon, Manuel Cebrian, Azim Shariff, Michael A Goodrich, Iyad Rahwan, et al. Cooperating with machines. Nature communications, 9(1):233, 2018.","title":"Cooperating with machines","authors":"Mayada Jacob W Crandall, Fatimah Oudah, Sherief Ishowo-Oloko, Jean-Franc Abdallah, Manuel ¸ois Bonnefon, Azim Cebrian, Michael A Shariff, Iyad Goodrich, Rahwan","year":"2018","venue":"Nature communications","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Devaine et al., 2014] Marie Devaine, Guillaume Hollard, and Jean Daunizeau. Theory of mind: did evolution fool us? PloS One, 9(2):e87619, 2014.","title":"Theory of mind: did evolution fool us?","authors":"Marie Devaine, Guillaume Hollard, Jean Daunizeau","year":"2014","venue":"PloS One","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gosavi, 2004] Abhijit Gosavi. Reinforcement learning for long-run average cost. European Journal of Operational Research, 155(3):654-674, 2004.","title":"Reinforcement learning for long-run average cost","authors":"Abhijit Gosavi","year":"2004","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Han et al., 2012] The Anh Han, Luis Moniz Pereira, and Francisco C Santos. Corpus-based intention recognition in cooperation dilemmas. Artificial Life, 18(4):365-383, 2012.","title":"The Anh Han, Luis Moniz Pereira, and Francisco C Santos. Corpus-based intention recognition in cooperation dilemmas","authors":"Han","year":"2012","venue":"Artificial Life","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hao et al., 2014] Dong Hao, Zhihai Rong, and Tao Zhou. Zero-determinant strategy: An underway revolution in game theory. Chinese Physics B, 23(7):078905, 2014.","title":"Zero-determinant strategy: An underway revolution in game theory","authors":"Dong Hao, Zhihai Rong, Tao Zhou","year":"2014","venue":"Chinese Physics B","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hao et al., 2015] Dong Hao, Zhihai Rong, and Tao Zhou. Extortion under uncertainty: Zero-determinant strategies in noisy games. Physical Review E, 91(5):052803, 2015.","title":"Extortion under uncertainty: Zero-determinant strategies in noisy games","authors":"Dong Hao, Zhihai Rong, Tao Zhou","year":"2015","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hilbe et al., 2013] Christian Hilbe, Martin A Nowak, and Karl Sigmund. Evolution of extortion in iterated prisoner's dilemma games. Proceedings of the National Academy of Sciences, 110(17):6913-6918, 2013.","title":"Evolution of extortion in iterated prisoner's dilemma games","authors":"Hilbe","year":"2013","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hilbe et al., 2014] Christian Hilbe, Bin Wu, Arne Traulsen, and Martin A Nowak. Cooperation and control in mul- tiplayer social dilemmas. Proceedings of the National Academy of Sciences, 111(46):16425-16430, 2014.","title":"Cooperation and control in multiplayer social dilemmas","authors":"Hilbe","year":"2014","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hilbe et al., 2015] Christian Hilbe, Arne Traulsen, and Karl Sigmund. Partners or rivals? strategies for the iterated pris- oner's dilemma. Games and Economic Behavior, 92:41- 52, 2015.","title":"Partners or rivals? strategies for the iterated prisoner's dilemma","authors":"Hilbe","year":"2015","venue":"Games and Economic Behavior","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kandori, 2002] Michihiro Kandori. Introduction to repeated games with private monitoring. Journal of Economic The- ory, 102(1):1-15, 2002.","title":"Introduction to repeated games with private monitoring","authors":"Michihiro Kandori, Kandori","year":"2002","venue":"Journal of Economic Theory","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li and Kendall, 2014] Jiawei Li and Graham Kendall. The effect of memory size on the evolutionary stability of strategies in iterated prisoner's dilemma. IEEE Transac- tions on Evolutionary Computation, 18(6):819-826, 2014.","title":"The effect of memory size on the evolutionary stability of strategies in iterated prisoner's dilemma","authors":"Kendall ; ] Jiawei Li, Graham Li, Kendall","year":"2014","venue":"IEEE Transactions on Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mailath and Samuelson, 2006] George J Mailath and Larry Samuelson. Repeated Games and Reputations: Long-Run Relationships. Oxford university press, 2006.","title":"Repeated Games and Reputations: Long-Run Relationships","authors":"J George, Larry Mailath, Samuelson","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mathieu and Delahaye, 2015] Philippe Mathieu and Jean- Paul Delahaye. New winning strategies for the iterated prisoner's dilemma. In Proceedings of the 2015 Interna- tional Conference on Autonomous Agents and Multiagent Systems, pages 1665-1666, 2015.","title":"New winning strategies for the iterated prisoner's dilemma","authors":"Delahaye Mathieu, Philippe Mathieu, Jean-Paul Delahaye","year":"2015","venue":"Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nowak et al., 1995] Martin A Nowak, Karl Sigmund, and Esam El-Sedy. Automata, repeated games and noise. Jour- nal of Mathematical Biology, 33(7):703-722, 1995.","title":"Automata, repeated games and noise","authors":"Martin A Nowak, Karl Sigmund, Esam El-Sedy","year":"1995","venue":"Journal of Mathematical Biology","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pan et al., 2015] Liming Pan, Dong Hao, Zhihai Rong, and Tao Zhou. Zero-determinant strategies in iterated public goods game. Scientific reports, 5:13096, 2015.","title":"Zero-determinant strategies in iterated public goods game","authors":"Liming Pan, Dong Hao, Zhihai Rong, Tao Zhou","year":"2015","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"William H Press and Freeman J Dyson. Iterated prisoners dilemma contains strategies that dominate any evolutionary opponent. Proceedings of the National Academy of Sciences, 109(26):10409-10413, 2012.","title":"Iterated prisoners dilemma contains strategies that dominate any evolutionary opponent","authors":"H William, Freeman J Press, Dyson","year":"2012","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Stewart and Plotkin, 2012] Alexander J Stewart and Joshua B Plotkin. Extortion and cooperation in the prisoner's dilemma. Proceedings of the National Academy of Sciences, 109(26):10134-10135, 2012.","title":"Extortion and cooperation in the prisoner's dilemma","authors":"J Alexander, Joshua B Stewart, Plotkin","year":"2012","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sutton and Barto, 1998] Richard S Sutton and Andrew G Barto. Reinforcement Learning: An Introduction. MIT press Cambridge, 1998.","title":"Reinforcement Learning: An Introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}