{"type":"ScholarlyArticle","title":"Reassessing Evaluation Functions in Algorithmic Recourse: An Empirical Study from a Human-Centered Perspective","authors":[{"name":"Tomu Tominaga","slug":"tomu-tominaga","given":"Tomu","family":"Tominaga","affiliation":"NTT Human Informatics Laboratories , NTT Corporation"},{"name":"Naomi Yamashita","slug":"naomi-yamashita","given":"Naomi","family":"Yamashita","affiliation":"NTT Communication Science Laboratories , NTT Corporation"},{"name":"Takeshi Kurashima","slug":"takeshi-kurashima","given":"Takeshi","family":"Kurashima","affiliation":"NTT Human Informatics Laboratories , NTT Corporation"}],"abstract":"In this study, we critically examine the foundational premise of algorithmic recourse - a process of generating counterfactual action plans (i.e., recourses) assisting individuals to reverse adverse decisions made by AI systems. The assumption underlying algorithmic recourse is that individuals accept and act on recourses that minimize the gap between their current and desired states. This assumption, however, remains empirically unverified. To address this issue, we conducted a user study with 362 participants and assessed whether minimizing the distance function, a metric of the gap between the current and desired states, indeed prompts them to accept and act upon suggested recourses. Our findings reveal a nuanced landscape: participants' acceptance of recourses did not correlate with the recourse distance. Moreover, participants' willingness to act upon recourses peaked at the minimal recourse distance but was otherwise constant. These findings cast doubt on the prevailing assumption of algorithmic recourse research and signal the need to rethink the evaluation functions to pave the way for human-centered recourse generation.","doi":"10.24963/ijcai.2024/876","url":"https://ijcai.org/proceedings/2024/876","pdf":"https://ijcai.org/proceedings/2024/0876.pdf","markdown":"https://ijcai.org/proceedings/2024/876.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/876","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Human-Centred AI","track":"Human-Centred AI","number":876,"pages":{"first":7913,"last":7921},"keywords":["Humans and AI: HAI: Applications","Humans and AI: HAI: Human-computer interaction","AI Ethics, Trust, Fairness: ETF: Explainability and interpretability","Humans and AI: HAI: Intelligent user interfaces"],"videos":[],"references":[{"position":1,"raw":"[Adadi and Berrada, 2018] Amina Adadi and Mohammed Berrada. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access, 6:52138-52160, 2018.","title":"Amina Adadi and Mohammed Berrada. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)","authors":"Berrada Adadi","year":"2018","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barocas et al., 2020] Solon Barocas, Andrew D. Selbst, and Manish Raghavan. The hidden assumptions behind coun- terfactual explanations and principal reasons. In Proceed- ings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 80-89, 2020.","title":"The hidden assumptions behind counterfactual explanations and principal reasons","authors":"Solon Barocas, Andrew D Selbst, Manish Raghavan","year":"2020","venue":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Becker and Kohavi, 1996] Barry Becker and Ronny Ko- havi. Adult. UCI Machine Learning Repository, 1996. DOI: https://doi.org/10.24432/C5XW20.","title":"Adult. UCI Machine Learning Repository","authors":"Kohavi Becker, Barry Becker, Ronny Kohavi","year":"1996","venue":null,"doi":"10.24432/c5xw20","url":"https://doi.org/10.24432/C5XW20","internal_url":null},{"position":4,"raw":"[Brown and Campion, 1994] Barbara K. Brown and Michael A. Campion. Biodata phenomenology: Re- cruiters' perceptions and use of biographical information in resume screening. Journal of Applied Psychology, 79(6):897-908, 1994.","title":"Biodata phenomenology: Recruiters' perceptions and use of biographical information in resume screening","authors":"Barbara K Brown, Michael A Campion","year":"1994","venue":"Journal of Applied Psychology","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cole et al., 2007] Michael S. Cole, Robert S. Rubin, Hu- bert S. Feild, and William F. Giles. Recruiters' Percep- tions and Use of Applicant Résumé Information: Screen- ing the Recent Graduate. Applied Psychology, 56(2):319- 343, 2007.","title":"Recruiters' Perceptions and Use of Applicant Résumé Information: Screening the Recent Graduate","authors":"Michael S Cole, Robert S Rubin, Hubert S Feild, William F Giles","year":"2007","venue":"Applied Psychology","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fogg, 1998] BJ Fogg. Persuasive computers: perspectives and research directions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, page 225-232, 1998.","title":"Persuasive computers: perspectives and research directions","authors":"Fogg","year":"1998","venue":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Förster et al., 2020] Maximilian Förster, Mathias Klier, Kil- ian Kluge, and Irina Sigler. Evaluating Explainable Artif- ical Intelligence -What Users Really Appreciate. In Eu- ropean Conference on Information Systems (ECIS), pages 1-18, 2020.","title":"Evaluating Explainable Artifical Intelligence -What Users Really Appreciate","authors":"Maximilian Förster, Mathias Klier, Kilian Kluge, Irina Sigler","year":"2020","venue":"European Conference on Information Systems (ECIS)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Förster et al., 2021] Maximilian Förster, Philipp Hühn, Mathias Klier, and Kilian Kluge. Capturing Users' Re- ality: A Novel Approach to Generate Coherent Coun- terfactual Explanations. In Proceedings of the Annual Hawaii International Conference on System Sciences, pages 1274-1283, 2021.","title":"Capturing Users' Reality: A Novel Approach to Generate Coherent Counterfactual Explanations","authors":"Maximilian Förster, Philipp Hühn, Mathias Klier, Kilian Kluge","year":"2021","venue":"Proceedings of the Annual Hawaii International Conference on System Sciences","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hofmann, 1994] Hans Hofmann. Statlog (German Credit Data). UCI Machine Learning Repository, 1994. DOI: https://doi.org/10.24432/C5NC77.","title":"Statlog (German Credit Data)","authors":"Hans Hofmann","year":"1994","venue":"UCI Machine Learning Repository","doi":"10.24432/c5nc77","url":"https://doi.org/10.24432/C5NC77","internal_url":null},{"position":10,"raw":"[Joshi et al., 2019] Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. To- wards Realistic Individual Recourse and Actionable Ex- planations in Black-Box Decision Making Systems. arXiv preprint arXiv:1907.09615, 2019.","title":"Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems","authors":"Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, Joydeep Ghosh","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kanamori et al., 2020] Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, and Hiroki Arimura. DACE: Distribution- Aware Counterfactual Explanation by Mixed-Integer Lin- ear Optimization Kentaro. In Proceedings of the Twenty- Ninth International Joint Conference on Artificial Intelli- gence, pages 2855-28, 2020.","title":"DACE: Distribution-Aware Counterfactual Explanation by Mixed-Integer Linear Optimization Kentaro","authors":"Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, Hiroki Arimura","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kanamori et al., 2021] Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, Yuichi Ike, Kento Uemura, and Hi- roki Arimura. Ordered Counterfactual Explanation by Mixed-Integer Linear Optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 35(13):11564- 11574, 2021.","title":"Ordered Counterfactual Explanation by Mixed-Integer Linear Optimization","authors":"Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, Yuichi Ike, Kento Uemura, Hiroki Arimura","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kanamori et al., 2022] Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, and Yuichi Ike. Counterfactual Expla- nation Trees: Transparent and Consistent Actionable Re- course with Decision Trees. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, pages 1846-1870, 2022.","title":"Counterfactual Explanation Trees: Transparent and Consistent Actionable Recourse with Decision Trees","authors":"Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, Yuichi Ike","year":"2022","venue":"Proceedings of the 25th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Karimi et al., 2020a] Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. Model-Agnostic Coun- terfactual Explanations for Consequential Decisions. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 895-905, 2020.","title":"Model-Agnostic Counterfactual Explanations for Consequential Decisions","authors":"Karimi","year":"2020","venue":"Proceedings of the International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Karimi et al., 2020b] Amir Hossein Karimi, Julius von Kügelgen, Bernhard Schölkopf, and Isabel Valera. Al- gorithmic recourse under imperfect causal knowledge: A probabilistic approach. In Advances in Neural Information Processing Systems, 2020.","title":"Algorithmic recourse under imperfect causal knowledge: A probabilistic approach","authors":"Karimi","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Karimi et al., 2021] Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. Algorithmic Recourse: from Counterfactual Explanations to Interventions. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 353-362, 2021.","title":"Algorithmic Recourse: from Counterfactual Explanations to Interventions","authors":"Amir-Hossein, Bernhard Karimi, Isabel Schölkopf, Valera","year":"2021","venue":"Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Karimi et al., 2022] Amir Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. A Survey of Al- gorithmic Recourse: Contrastive Explanations and Con- sequential Recommendations. ACM Computing Surveys, 55(5), 2022.","title":"A Survey of Algorithmic Recourse: Contrastive Explanations and Consequential Recommendations","authors":"Gilles Amir Hossein Karimi, Bernhard Barthe, Isabel Schölkopf, Valera","year":"2022","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Keane et al., 2021] Mark T Keane, Eoin M Kenny, Eoin De- laney, and Barry Smyth. If Only We Had Better Coun- terfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques. In Pro- ceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pages 4466-4474, 2021.","title":"If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques","authors":"Eoin M Mark T Keane, Eoin Kenny, Barry Delaney, Smyth","year":"2021","venue":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2021/609"},{"position":19,"raw":"[Kirfel and Liefgreen, 2021] Lara Kirfel and Alice Lief- green. What If (and How ...)? -Actionability Shapes Peo- ple's Perceptions of Counterfactual Explanations in Auto- mated Decision-Making. In ICML-21 Workshop on Algo- rithmic Recourse, 2021.","title":"What If (and How ...)? -Actionability Shapes People's Perceptions of Counterfactual Explanations in Automated Decision-Making","authors":"Lara Kirfel, Alice Liefgreen","year":"2021","venue":"ICML-21 Workshop on Algorithmic Recourse","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kuhl et al., 2022] Ulrike Kuhl, André Artelt, and Barbara Hammer. Keep Your Friends Close and Your Counter- factuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting. In 2022 ACM Conference on Fairness, Account- ability, and Transparency, pages 2125-2137, 2022.","title":"Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting","authors":"Ulrike Kuhl, André Artelt, Barbara Hammer","year":"2022","venue":"ACM Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Miller, 2019] Tim Miller. Explanation in artificial intelli- gence: Insights from the social sciences. Artificial Intelli- gence, 267:1-38, 2019.","title":"Explanation in artificial intelligence: Insights from the social sciences","authors":"Tim Miller, Miller","year":"2019","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mothilal et al., 2020] Ramaravind K. Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 607-617, 2020.","title":"Explaining machine learning classifiers through diverse counterfactual explanations","authors":"K Ramaravind, Amit Mothilal, Chenhao Sharma, Tan","year":"2020","venue":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Naumann and Ntoutsi, 2021] Philip Naumann and Eirini Ntoutsi. Consequence-Aware Sequential Counterfactual Generation. In Proceedings of the Joint European Con- ference on Machine Learning and Knowledge Discovery in Databases, pages 682-698, 2021.","title":"Consequence-Aware Sequential Counterfactual Generation","authors":"Philip Naumann, Eirini Ntoutsi","year":"2021","venue":"Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pawelczyk et al., 2020] Martin Pawelczyk, Klaus Broele- mann, and Gjergji. Kasneci. On counterfactual explana- tions under predictive multiplicity. In Jonas Peters and David Sontag, editors, Proceedings of the 36th Confer- ence on Uncertainty in Artificial Intelligence (UAI), vol- ume 124, pages 809-818. PMLR, 2020.","title":"On counterfactual explanations under predictive multiplicity","authors":"Martin Pawelczyk, Klaus Broelemann, Gjergji Kasneci","year":"2020","venue":"Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Poyiadzi et al., 2020] Rafael Poyiadzi, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. FACE: Feasible and Actionable Counterfactual Explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 344-350, 2020.","title":"FACE: Feasible and Actionable Counterfactual Explanations","authors":"Poyiadzi","year":"2020","venue":"Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rawal and Lakkaraju, 2020] Kaivalya Rawal and Himabindu Lakkaraju. Beyond individualized recourse: Interpretable and interactive summaries of actionable recourses. In Advances in Neural Information Processing Systems, 2020.","title":"Beyond individualized recourse: Interpretable and interactive summaries of actionable recourses","authors":"Kaivalya Rawal, Himabindu Lakkaraju","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Read and Marcus-Newhall, 1993] Stephen J. Read and Amy Marcus-Newhall. Explanatory coherence in social explanations: A parallel distributed processing account. Journal of Personality and Social Psychology, 65(3):429- 447, 1993.","title":"Explanatory coherence in social explanations: A parallel distributed processing account","authors":"Stephen J Read, Amy Marcus-Newhall","year":"1993","venue":"Journal of Personality and Social Psychology","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Russell, 2019] Chris Russell. Efficient Search for Diverse Coherent Explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 20- 28, 2019.","title":"Efficient Search for Diverse Coherent Explanations","authors":"Chris Russell","year":"2019","venue":"Proceedings of the Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Singh et al., 2023] Ronal Singh, Tim Miller, Henrietta Lyons, Liz Sonenberg, Eduardo Velloso, Frank Vetere, Piers Howe, and Paul Dourish. Directive Explanations for Actionable Explainability in Machine Learning Applica- tions. ACM Transactions on Interactive Intelligent Sys- tems, pages 1-25, 2023.","title":"Directive Explanations for Actionable Explainability in Machine Learning Applications","authors":"Ronal Singh, Tim Miller, Henrietta Lyons, Liz Sonenberg, Eduardo Velloso, Frank Vetere, Piers Howe, Paul Dourish","year":"2023","venue":"ACM Transactions on Interactive Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Thagard, 1989] Paul Thagard. Explanatory coherence. Be- havioral and Brain Sciences, 12(3):435-467, 1989.","title":"Paul Thagard. Explanatory coherence","authors":"Thagard","year":"1989","venue":"Behavioral and Brain Sciences","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Ustun et al., 2019] Berk Ustun, Alexander Spangher, and Yang Liu. Actionable Recourse in Linear Classification. In Proceedings of the Conference on Fairness, Account- ability, and Transparency, pages 10-19, 2019.","title":"Actionable Recourse in Linear Classification","authors":"Berk Ustun, Alexander Spangher, Yang Liu","year":"2019","venue":"Proceedings of the Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Van Looveren and Klaise, 2021] Arnaud Van Looveren and Janis Klaise. Interpretable Counterfactual Explanations Guided by Prototypes. In Joint European Confer- ence on Machine Learning and Knowledge Discovery in Databases, pages 650-665, 2021.","title":"Interpretable Counterfactual Explanations Guided by Prototypes","authors":"Klaise ; Arnaud Van Looveren, Janis Van Looveren, Klaise","year":"2021","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Venkatasubramanian and Alfano, 2020] Suresh Venkata- subramanian and Mark Alfano. The philosophical basis of algorithmic recourse. In Proceedings of the 2020 Con- ference on Fairness, Accountability, and Transparency, pages 284-293, 2020.","title":"The philosophical basis of algorithmic recourse","authors":"Suresh Venkatasubramanian, Mark Alfano","year":"2020","venue":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Verma et al., 2020] Sahil Verma, Varich Boonsanong, Minh Hoang, Keegan E. Hines, John P. Dickerson, and Chirag Shah. Counterfactual Explanations and Algorithmic Re- courses for Machine Learning: A Review. arXiv preprint arXiv:2010.10596, 2020.","title":"Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review","authors":"Sahil Verma, Varich Boonsanong, Minh Hoang, E Keegan, John P Hines, Chirag Dickerson, Shah","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wachter et al., 2018] Sandra Wachter, Brent Mittelstadt, and Chris Russel. Counterfactual Explanations With- out Opening the Black Box: Automated Decisions and thhe GDPR. Harvard Journal of Law & Technology, 20(3):842-887, 2018.","title":"Counterfactual Explanations Without Opening the Black Box: Automated Decisions and thhe GDPR","authors":"Sandra Wachter, Brent Mittelstadt, Chris Russel","year":"2018","venue":"Harvard Journal of Law & Technology","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wang et al., 2023] Zijie J. Wang, Jennifer Wortman Vaughan, Rich Caruana, and Duen Horng Chau. GAM Coach: Towards Interactive and User-centered Algo- rithmic Recourse. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-20, 2023.","title":"GAM Coach: Towards Interactive and User-centered Algorithmic Recourse","authors":"Wang","year":"2023","venue":"Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Warren et al., 2023] Greta Warren, Ruth M. J. Byrne, and Mark T. Keane. Categorical and Continuous Features in Counterfactual Explanations of AI Systems. In Proceed- ings of the 28th International Conference on Intelligent User Interfaces, pages 171-187, 2023.","title":"Categorical and Continuous Features in Counterfactual Explanations of AI Systems","authors":"Greta Warren, M J Ruth, Mark T Byrne, Keane","year":"2023","venue":"Proceedings of the 28th International Conference on Intelligent User Interfaces","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yacoby et al., 2022] Yaniv Yacoby, Ben Green, Christo- pher L. Griffin Jr., and Finale Doshi-Velez. \"If it didn't happen, why would I change my decision?\": How Judges Respond to Counterfactual Explanations for the Public Safety Assessment. In Proceedings of the AAAI Confer- ence on Human Computation and Crowdsourcing, pages 219-230, 2022.","title":"If it didn't happen, why would I change my decision?\": How Judges Respond to Counterfactual Explanations for the Public Safety Assessment","authors":"Yaniv Yacoby, Ben Green, Christopher L Griffin, Finale Doshi-Velez","year":"2022","venue":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yeh, 2016] I-Cheng Yeh. Default of Credit Card Clients. UCI Machine Learning Repository, 2016. DOI: https:// doi.org/10.24432/C55S3H.","title":"Default of Credit Card Clients","authors":"I-Cheng Yeh","year":"2016","venue":"UCI Machine Learning Repository","doi":"10.24432/c55s3h","url":"https://doi.org/10.24432/C55S3H","internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}