{"type":"ScholarlyArticle","title":"Irrational, but Adaptive and Goal Oriented: Humans Interacting with Autonomous Agents","authors":[{"name":"Amos Azaria","slug":"amos-azaria","given":"Amos","family":"Azaria","affiliation":"Computer Science Department , Ariel University , Israel"}],"abstract":"Autonomous agents that interact with humans are becoming more and more prominent.\r\nCurrently, such agents usually take one of the following approaches for considering human behavior. Some methods assume either a fully cooperative or a zero-sum setting; these assumptions entail that the human's goals are either identical to that of the agent, or their opposite. In both cases, the agent is not required to explicitly model the human’s goals and account for humans' adaptation nature. \r\nOther methods first compose a model of human behavior based on observing human actions, and then optimize the agent’s actions based on this model. Such methods do not account for how the human will react to the agent's actions and thus, suffer an overestimation bias.\r\nFinally, other methods, such as model free reinforcement learning, merely learn which actions the agent should take at which states. While such methods can, theoretically, account for human adaptation nature, since they require extensive interaction with humans, they usually run in simulation.\r\nBy not considering the human’s goals, autonomous agents act selfishly, lack generalization, require vast amounts of data, and cannot account for human’s strategic behavior. Therefore, we call for pursuing solution concepts for autonomous agents interacting with humans that consider the human’s goals and adaptive nature.","doi":"10.24963/ijcai.2022/813","url":"https://ijcai.org/proceedings/2022/813","pdf":"https://ijcai.org/proceedings/2022/0813.pdf","markdown":"https://ijcai.org/proceedings/2022/813.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/813","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Early Career","track":"Early Career","number":813,"pages":{"first":5798,"last":5802},"keywords":["EC: Human-Agent Interaction."],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987220","length_sec":93,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985363","length_sec":903,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Arabshahi et al., 2021] Forough Arabshahi, Jennifer Lee, Mikayla Gawarecki, Kathryn Mazaitis, Amos Azaria, and Tom Mitchell. Conversational neuro-symbolic common- sense reasoning. In AAAI, volume 35, pages 4902-4911, 2021.","title":"Conversational neuro-symbolic commonsense reasoning","authors":"Forough Arabshahi, Jennifer Lee, Mikayla Gawarecki, Kathryn Mazaitis, Amos Azaria, Tom Mitchell","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ariely et al., 2003] Dan Ariely, George Loewenstein, and Drazen Prelec. \"coherent arbitrariness\": Stable demand curves without stable preferences. The Quarterly Journal of Economics, 118(1):73-106, 2003.","title":"coherent arbitrariness\": Stable demand curves without stable preferences","authors":"Dan Ariely, George Loewenstein, Drazen Prelec","year":"2003","venue":"The Quarterly Journal of Economics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Azaria et al., 2012] Amos Azaria, Yonatan Aumann, and Sarit Kraus. Automated strategies for determining rewards for human work. In AAAI, 2012.","title":"Automated strategies for determining rewards for human work","authors":"Amos Azaria, Yonatan Aumann, Sarit Kraus","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Azaria et al., 2016] Amos Azaria, Ya'akov Gal, Sarit Kraus, and Claudia V Goldman. Strategic advice provision in repeated human-agent interactions. JAAMAS, 30(1):4- 29, 2016.","title":"Strategic advice provision in repeated human-agent interactions","authors":"Azaria","year":"2016","venue":"JAAMAS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Blanken et al., 2015] Irene Blanken, Niels van de Ven, and Marcel Zeelenberg. A meta-analytic review of moral licensing. Personality and Social Psychology Bulletin, 41(4):540-558, 2015.","title":"A meta-analytic review of moral licensing","authors":"Irene Blanken, Niels Van De Ven, Marcel Zeelenberg","year":"2015","venue":"Personality and Social Psychology Bulletin","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Camerer, 2003] C. F. Camerer. Behavioral Game Theory. Experiments in Strategic Interaction, chapter 2. Princeton University Press, 2003.","title":"Behavioral Game Theory. Experiments in Strategic Interaction","authors":"C F Camerer","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Carroll et al., 2019] Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, and Anca Dragan. On the utility of learning about humans for human-ai coordination. Advances in Neural Information Processing Systems, 32:5174-5185, 2019.","title":"On the utility of learning about humans for human-ai coordination","authors":"Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, Anca Dragan","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Freire et al., 2019] Ismael T Freire, Xerxes D Arsiwalla, Jordi-Ysard Puigbò, and Paul Verschure. Modeling the- ory of mind in multi-agent games using adaptive feedback control. arXiv preprint arXiv:1905.13225, 2019.","title":"Modeling theory of mind in multi-agent games using adaptive feedback control","authors":"Xerxes D Ismael T Freire, Jordi-Ysard Arsiwalla, Paul Puigbò, Verschure","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gal and Pfeffer, 2007] Ya'akov Gal and Avi Pfeffer. Model- ing reciprocal behavior in human bilateral negotiation. In NCAI, volume 22, page 815. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2007.","title":"Ya'akov Gal and Avi Pfeffer. Modeling reciprocal behavior in human bilateral negotiation","authors":"Pfeffer Gal","year":"1999","venue":"NCAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Grosz et al., 2004] Barbara Grosz, Sarit Kraus, Shavit Tal- man, Boaz Stossel, and Moti Havlin. The influence of social dependencies on decision-making: Initial investiga- tions with a new game. 2004.","title":"The influence of social dependencies on decision-making: Initial investigations with a new game","authors":"Barbara Grosz, Sarit Kraus, Shavit Talman, Boaz Stossel, Moti Havlin","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hsu et al., 1995] Feng-hsiung Hsu, Murray S Campbell, and A Joseph Hoane Jr. Deep blue system overview. In Supercomputing, pages 240-244, 1995.","title":"Deep blue system overview","authors":"Hsu","year":"1995","venue":"Supercomputing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jaques et al., 2019] Natasha Jaques, Angeliki Lazari- dou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, DJ Strouse, Joel Z Leibo, and Nando De Freitas. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In ICML, pages 3040-3049.","title":"Social influence as intrinsic motivation for multi-agent deep reinforcement learning","authors":"Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, Joel Z Strouse, Nando De Leibo, Freitas","year":null,"venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"PMLR, 2019.","title":null,"authors":"Pmlr","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Loewenstein, 2000] George Loewenstein. Willpower: A decision-theorist's perspective. Law and Philosophy, 19:51-76, 2000.","title":"Willpower: A decision-theorist's perspective","authors":"George Loewenstein","year":"2000","venue":"Law and Philosophy","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nay and Vorobeychik, 2016] John J Nay and Yevgeniy Vorobeychik. Predicting human cooperation. PloS one, 11(5):e0155656, 2016.","title":"Predicting human cooperation","authors":"J John, Yevgeniy Nay, Vorobeychik","year":"2016","venue":"PloS one","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Peled et al., 2011] Noam Peled, Ya'akov Kobi Gal, and Sarit Kraus. A study of computational and human strate- gies in revelation games. In AAMAS, pages 345-352, 2011.","title":"A study of computational and human strategies in revelation games","authors":"Peled","year":"2011","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ramachandran and Amir, 2007] Deepak Ramachandran and Eyal Amir. Bayesian inverse reinforcement learning. In IJCAI, volume 7, pages 2586-2591, 2007.","title":"Bayesian inverse reinforcement learning","authors":"Deepak Ramachandran, Eyal Amir","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/416"},{"position":18,"raw":"[Rosenfeld and Kraus, 2011] Avi Rosenfeld and Sarit Kraus. Using aspiration adaptation theory to improve learning. In AAMAS, pages 423-430, 2011.","title":"Using aspiration adaptation theory to improve learning","authors":"Kraus Rosenfeld, Avi Rosenfeld, Sarit Kraus","year":"2011","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sandholm and Crites, 1996] Tuomas W Sandholm and Robert H Crites. Multiagent reinforcement learning in the iterated prisoner's dilemma. Biosystems, 37(1-2):147- 166, 1996.","title":"Multiagent reinforcement learning in the iterated prisoner's dilemma","authors":"Crites ; ] Sandholm, W Tuomas, Robert H Sandholm, Crites","year":"1996","venue":"Biosystems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shapira and Azaria, 2022] Ido Shapira and Amos Azaria. Reinforcement learning agents for interacting with hu- mans. In CogSci, 2022.","title":"Reinforcement learning agents for interacting with humans","authors":"Azaria Shapira, Ido Shapira, Amos Azaria","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris J Maddi- son, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Pan- neershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484-489, 2016.","title":"Mastering the game of go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den, Marc Driessche, Lanctot","year":"2016","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Skinner and Walmsley, 2019] Geoff Skinner and Toby Walmsley. Artificial intelligence and deep learning in video games a brief review. In ICCCS, pages 404-408. IEEE, 2019.","title":"Artificial intelligence and deep learning in video games a brief review","authors":"Walmsley Skinner, Geoff Skinner, Toby Walmsley","year":"2019","venue":"ICCCS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Subrahmanian, 2000] Ventatramanan S Subrahmanian. Heterogeneous agent systems. MIT press, 2000.","title":"Heterogeneous agent systems","authors":"Ventatramanan S Subrahmanian","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Syll, 2018] Lars Pålsson Syll. Why game theory never will be anything but a footnote in the history of social science. Real-World Economics Review, 83:45-64, 2018.","title":"Why game theory never will be anything but a footnote in the history of social science","authors":"Lars Pålsson Syll, Syll","year":"2018","venue":"Real-World Economics Review","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tetlock, 1985] Philip E Tetlock. Accountability: A social check on the fundamental attribution error. Social psy- chology quarterly, pages 227-236, 1985.","title":"Accountability: A social check on the fundamental attribution error","authors":"Philip E Tetlock","year":"1985","venue":"Social psychology quarterly","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tversky and Kahneman, 1981] Amos Tversky and Danliel Kahneman. The framing of decisions and the psychology of choice. Science, 211(4481):453-458, 1981.","title":"The framing of decisions and the psychology of choice","authors":"Kahneman Tversky","year":"1981","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tversky and Kahneman, 1992] Amos Tversky and Daniel Kahneman. Advances in prospect theory: Cumulative rep- resentation of uncertainty. Risk and uncertainty, 5(4):297- 323, 1992.","title":"Advances in prospect theory: Cumulative representation of uncertainty","authors":"Kahneman Tversky, Amos Tversky, Daniel Kahneman","year":"1992","venue":"Risk and uncertainty","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2016] Zhijian Wang, Yanran Zhou, Jaimie W Lien, Jie Zheng, and Bin Xu. Extortion can outperform generosity in the iterated prisoner's dilemma. Nature com- munications, 7(1):1-7, 2016.","title":"Extortion can outperform generosity in the iterated prisoner's dilemma","authors":"Zhijian Wang, Yanran Zhou, Jaimie W Lien, Jie Zheng, Bin Xu","year":"2016","venue":"Nature communications","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}