{"type":"ScholarlyArticle","title":"Transparency, Detection and Imitation in Strategic Classification","authors":[{"name":"Flavia Barsotti","slug":"flavia-barsotti","given":"Flavia","family":"Barsotti","affiliation":"ING Analytics , Amsterdam , The Netherlands; Institute for Advanced Study , University of Amsterdam , The Netherlands; Delft Institute of Applied Mathematics , TU Delft , Delft , The Netherlands"},{"name":"Ruya Gokhan Kocer","slug":"ruya-gokhan-kocer","given":"Ruya Gokhan","family":"Kocer","affiliation":"ING Analytics , Amsterdam , The Netherlands"},{"name":"Fernando P. Santos","slug":"fernando-p-santos","given":"Fernando P.","family":"Santos","affiliation":"Informatics Institute , University of Amsterdam , The Netherlands"}],"abstract":"Given the ubiquity of AI-based decisions that affect individuals’ lives, providing transparent explanations about algorithms is ethically sound and often legally mandatory. How do individuals strategically adapt following explanations? What are the consequences of adaptation for algorithmic accuracy? We simulate the interplay between explanations shared by an Institution (e.g. a bank) and the dynamics of strategic adaptation by Individuals reacting to such feedback. Our model identifies key aspects related to strategic adaptation and the challenges that an institution could face as it attempts to provide explanations. Resorting to an agent-based approach, our model scrutinizes: i) the impact of transparency in explanations, ii) the interaction between faking behavior and detection capacity and iii) the role of behavior imitation. We find that the risks of transparent explanations are alleviated if effective methods to detect faking behaviors are in place. Furthermore, we observe that behavioral imitation --- as often happens across societies --- can alleviate malicious adaptation and contribute to accuracy, even after transparent explanations.","doi":"10.24963/ijcai.2022/10","url":"https://ijcai.org/proceedings/2022/10","pdf":"https://ijcai.org/proceedings/2022/0010.pdf","markdown":"https://ijcai.org/proceedings/2022/10.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/10","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":10,"pages":{"first":67,"last":73},"keywords":["Multidisciplinary Topics and Applications: Finance","Agent-based and Multi-agent Systems: Agent-Based Simulation and Emergence","AI Ethics, Trust, Fairness: Ethical, Legal and Societal Issues"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987053","length_sec":89,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985196","length_sec":718,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Akyol et al., 2016] Emrah Akyol, Cedric Langbort, and Tamer Basar. Price of transparency in strategic machine learning. arXiv preprint arXiv:1610.08210, 2016.","title":"Price of transparency in strategic machine learning","authors":"Emrah Akyol, Cedric Langbort, Tamer Basar","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bambauer and Zarsky, 2018] Jane Bambauer and Tal Zarsky. The algorithm game. Notre Dame L. Rev., 94:1, 2018.","title":"The algorithm game","authors":"Zarsky Bambauer, Jane Bambauer, Tal Zarsky","year":"2018","venue":"Notre Dame L. Rev","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Banerjee, 1992] Abhijit V Banerjee. A simple model of herd behavior. The Quarterly Journal of Economics, 107(3):797-817, 1992.","title":"A simple model of herd behavior","authors":"V Abhijit, Banerjee","year":"1992","venue":"The Quarterly Journal of Economics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Barreno et al., 2006] Marco Barreno, Blaine Nelson, Rus- sell Sears, Anthony D Joseph, and J Doug Tygar. Can machine learning be secure? In Proceedings of the 2006 ACM Symposium on Information, Computer and Commu- nications Security, pages 16-25, 2006.","title":"Can machine learning be secure?","authors":"Marco Barreno, Blaine Nelson, Russell Sears, Anthony D Joseph, Doug Tygar","year":"2006","venue":"Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Barsotti et al., 2022] Flavia Barsotti, Rüya Gökhan Koc ¸er, and Fernando P Santos. Can algorithms be explained with- out compromising efficiency? The benefits of detection and imitation in strategic classification. In Proc. of AA- MAS 2022, pages 1536-1538, 2022.","title":"Can algorithms be explained without compromising efficiency? The benefits of detection and imitation in strategic classification","authors":"Flavia Barsotti, Rüya Gökhan Koc ¸er, Fernando P Santos","year":"2022","venue":"Proc. of AA-MAS 2022","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bechavod et al., 2021] Yahav Bechavod, Chara Podimata, Zhiwei Steven Wu, and Juba Ziani. Information discrepancy in strategic learning. arXiv preprint arXiv:2103.01028, 2021.","title":"Information discrepancy in strategic learning","authors":"Yahav Bechavod, Chara Podimata, Zhiwei Steven Wu, Juba Ziani","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bikhchandani et al., 1992] Sushil Bikhchandani, David Hirshleifer, and Ivo Welch. A theory of fads, fashion, custom, and cultural change as informational cascades. Journal of Political Economy, 100(5):992-1026, 1992.","title":"A theory of fads, fashion, custom, and cultural change as informational cascades","authors":"Sushil Bikhchandani, David Hirshleifer, Ivo Welch","year":"1992","venue":"Journal of Political Economy","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chopra and Singh, 2018] Amit K Chopra and Munindar P Singh. Sociotechnical systems and ethics in the large. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 48-53, 2018.","title":"Sociotechnical systems and ethics in the large","authors":"K Amit, Munindar P Chopra, Singh","year":"2018","venue":"Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Citron and Pasquale, 2014] Danielle Keats Citron and Frank Pasquale. The scored society: Due process for automated predictions. Wash. L. Rev., 89:1, 2014.","title":"The scored society: Due process for automated predictions","authors":"Pasquale ; Citron, Danielle Keats, Citron, Frank Pasquale","year":"2014","venue":"Wash. L. Rev","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dalvi et al., 2004] Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, and Deepak Verma. Adversarial classification. In Proc. of the 10th ACM SIGKDD, pages 99-108, 2004.","title":"Adversarial classification","authors":"Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, Deepak Verma","year":"2004","venue":"Proc. of the 10th ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dekel et al., 2010] Ofer Dekel, Felix Fischer, and Ariel D Procaccia. Incentive compatible regression learning. J. Comput. Syst. Sci., 76(8):759-777, 2010.","title":"Incentive compatible regression learning","authors":"Ofer Dekel, Felix Fischer, Ariel D Procaccia","year":"2010","venue":"J. Comput. Syst. Sci","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Domingos et al., 2021] Elias Fernández Domingos, Jelena Grujić, Juan C Burguillo, Francisco C Santos, and Tom Lenaerts. Modeling behavioral experiments on uncer- tainty and cooperation with population-based reinforce- ment learning. Simul. Model. Pract. Theory., 109:102299, 2021.","title":"Modeling behavioral experiments on uncertainty and cooperation with population-based reinforcement learning","authors":"Elias Fernández Domingos, Jelena Grujić, Juan C Burguillo, Francisco C Santos, Tom Lenaerts","year":"2021","venue":"Simul. Model. Pract. Theory","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ghalme et al., 2021] Ganesh Ghalme, Vineet Nair, Itay Eilat, Inbal Talgam-Cohen, and Nir Rosenfeld. Strategic classification in the dark. In Proceedings of ICML 2021, volume 139, pages 3672-3681. PMLR, 18-24 Jul 2021.","title":"Strategic classification in the dark","authors":"Ganesh Ghalme, Vineet Nair, Itay Eilat, Inbal Talgam-Cohen, Nir Rosenfeld","year":"2021","venue":"Proceedings of ICML 2021","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Goodman and Flaxman, 2017] Bryce Goodman and Seth Flaxman. European union regulations on algorithmic decision-making and a \"right to explanation\". AI Maga- zine, 38(3):50-57, 2017.","title":"European union regulations on algorithmic decision-making and a \"right to explanation","authors":"Flaxman Goodman, Bryce Goodman, Seth Flaxman","year":"2017","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hardt et al., 2016] Moritz Hardt, Nimrod Megiddo, Chris- tos Papadimitriou, and Mary Wootters. Strategic classifi- cation. In Proc. of ITCS '16, pages 111-122, 2016.","title":"Strategic classification","authors":"Hardt","year":"2016","venue":"Proc. of ITCS '16","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hu et al., 2019] Lily Hu, Nicole Immorlica, and Jen- nifer Wortman Vaughan. The disparate effects of strate- gic manipulation. In Proc. of the Conference on Fairness, Accountability, and Transparency, pages 259-268, 2019.","title":"The disparate effects of strategic manipulation","authors":"Hu","year":"2019","venue":"Proc. of the Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jagadeesan et al., 2021] Meena Jagadeesan, Celestine Mendler-Dünner, and Moritz Hardt. Alternative micro- foundations for strategic classification. In Proc. of ICML 2021, pages 4687-4697. PMLR, 2021.","title":"Alternative microfoundations for strategic classification","authors":"Meena Jagadeesan, Celestine Mendler-Dünner, Moritz Hardt","year":"2021","venue":"Proc. of ICML 2021","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kleinberg and Raghavan, 2020] Jon Kleinberg and Manish Raghavan. How do classifiers induce agents to invest ef- fort strategically? ACM Transactions on Economics and Computation (TEAC), 8(4):1-23, 2020.","title":"How do classifiers induce agents to invest effort strategically","authors":"Raghavan Kleinberg, Jon Kleinberg, Manish Raghavan","year":"2020","venue":"ACM Transactions on Economics and Computation (TEAC)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Krishnaswamy et al., 2021] Anilesh K. Krishnaswamy, Haoming Li, David Rein, Hanrui Zhang, and Vincent Conitzer. Classification with strategically withheld data. Proceedings of AAAI 2021, 35(6):5514-5522, May 2021.","title":"Classification with strategically withheld data","authors":"K Anilesh, Haoming Krishnaswamy, David Li, Hanrui Rein, Vincent Zhang, Conitzer","year":"2021","venue":"Proceedings of AAAI 2021","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Meir et al., 2012] Reshef Meir, Ariel D Procaccia, and Jef- frey S Rosenschein. Algorithms for strategyproof classifi- cation. Artificial Intelligence, 186:123-156, 2012.","title":"Algorithms for strategyproof classification","authors":"Reshef Meir, Ariel D Procaccia, Jeffrey S Rosenschein","year":"2012","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Miller et al., 2020] John Miller, Smitha Milli, and Moritz Hardt. Strategic classification is causal modeling in dis- guise. In Proc. of ICML 2020, pages 6917-6926, 2020.","title":"Strategic classification is causal modeling in disguise","authors":"John Miller, Smitha Milli, Moritz Hardt","year":"2020","venue":"Proc. of ICML 2020","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Molnar, 2020] Christoph Molnar. Interpretable machine learning. https://christophm.github.io/interpretable-ml- book/, 2020.","title":"Interpretable machine learning","authors":"Christoph Molnar","year":null,"venue":null,"doi":null,"url":"https://christophm.github.io/interpretable-ml-book/,2020","internal_url":null},{"position":23,"raw":"[Murukannaiah et al., 2020] Pradeep K Murukannaiah, Ni- rav Ajmeri, Catholijn M Jonker, and Munindar P Singh. New foundations of ethical multiagent systems. In Pro- ceedings of AAMAS 2020, pages 1706-1710, 2020.","title":"New foundations of ethical multiagent systems","authors":"Nirav Pradeep K Murukannaiah, Catholijn M Ajmeri, Munindar P Jonker, Singh","year":"2020","venue":"Proceedings of AAMAS 2020","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Santos et al., 2019] Fernando P Santos, Jorge M Pacheco, Ana Paiva, and Francisco C Santos. Evolution of collective fairness in hybrid populations of humans and agents. In Proceedings of AAAI 2019, pages 6146-6153, 2019.","title":"Evolution of collective fairness in hybrid populations of humans and agents","authors":"Jorge M Fernando P Santos, Ana Pacheco, Francisco C Paiva, Santos","year":"2019","venue":"Proceedings of AAAI 2019","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Santos et al., 2021] Fernando P Santos, Simon A Levin, and Vítor V Vasconcelos. Biased perceptions explain col- lective action deadlocks and suggest new mechanisms to prompt cooperation. iScience, 24(4):102375, 2021.","title":"Biased perceptions explain collective action deadlocks and suggest new mechanisms to prompt cooperation","authors":"Simon A Fernando P Santos, Levin, Vítor, Vasconcelos","year":"2021","venue":"iScience","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sen and Airiau, 2007] Sandip Sen and Stephane Airiau. Emergence of norms through social learning. In Proceed- ings of IJCAI 2007, volume 1507, page 1512, 2007.","title":"Emergence of norms through social learning","authors":"Sandip Sen, Stephane Airiau","year":"2007","venue":"Proceedings of IJCAI 2007","doi":null,"url":null,"internal_url":"/proceedings/2007/243"},{"position":27,"raw":"[Shirado and Christakis, 2020] Hirokazu Shirado and Nicholas A Christakis. Network engineering using au- tonomous agents increases cooperation in human groups. iScience, 23(9):101438, 2020.","title":"Network engineering using autonomous agents increases cooperation in human groups","authors":"Hirokazu Shirado, Nicholas A Christakis","year":"2020","venue":"iScience","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wachter et al., 2017] Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations with- out opening the black box: Automated decisions and the GDPR. Harv. JL & Tech., 31:841, 2017.","title":"Counterfactual explanations without opening the black box: Automated decisions and the GDPR","authors":"Sandra Wachter, Brent Mittelstadt, Chris Russell","year":"2017","venue":"Harv. JL & Tech","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yu et al., 2013] Han Yu, Zhiqi Shen, Cyril Leung, Chunyan Miao, and Victor R Lesser. A survey of multi-agent trust management systems. IEEE Access, 1:35-50, 2013.","title":"A survey of multi-agent trust management systems","authors":"Han Yu, Zhiqi Shen, Cyril Leung, Chunyan Miao, Victor R Lesser","year":"2013","venue":"IEEE Access","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}