{"type":"ScholarlyArticle","title":"Explanation Perspectives from the Cognitive Sciences---A Survey","authors":[{"name":"Ramya Srinivasan","slug":"ramya-srinivasan","given":"Ramya","family":"Srinivasan","affiliation":"Fujitsu Laboratories of America"},{"name":"Ajay Chander","slug":"ajay-chander","given":"Ajay","family":"Chander","affiliation":"Fujitsu Laboratories of America"}],"abstract":"With growing adoption of AI across fields such as healthcare, finance, and the justice system, explaining an AI decision has become more important than ever before. Development of human-centric explainable AI (XAI) systems necessitates an understanding of the requirements of the human-in-the-loop seeking the explanation.  This includes the cognitive behavioral purpose that the explanation serves for its recipients, and the structure that the explanation uses to reach those ends.  An understanding of the psychological foundations of explanations is thus vital for the development of effective human-centric XAI systems. Towards this end, we survey papers from the cognitive science literature that address the following broad questions: (1) what is an explanation, (2) what are explanations for, and 3) what are the characteristics of good and bad explanations. We organize the insights gained therein by means of highlighting the advantages and shortcomings of various explanation structures and theories, discuss their applicability across different domains, and analyze their utility to various types of humans-in-the-loop.  We summarize the key takeaways for human-centric design of XAI systems, and recommend strategies to bridge the existing gap between XAI research and practical needs. We hope this work will spark the development of novel human-centric XAI systems.","doi":"10.24963/ijcai.2020/670","url":"https://ijcai.org/proceedings/2020/670","pdf":"https://ijcai.org/proceedings/2020/0670.pdf","markdown":"https://ijcai.org/proceedings/2020/670.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/670","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Survey track","track":"Survey track","number":670,"pages":{"first":4812,"last":4818},"keywords":["Safe, Explainable, and Trustworthy AI: general","Human aspects in AI: general"],"videos":[],"references":[{"position":1,"raw":"[Achinstein, 1983] Peter Achinstein. The nature of explana- tions. Oxford University Press, 1983.","title":"Peter Achinstein. The nature of explanations","authors":"Achinstein","year":"1983","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boyd, 1999] Richard Boyd. Homeostasis, species, and higher taxa. Species: New Interdisciplinary Studies-MIT Press, 1999.","title":"Homeostasis, species, and higher taxa. Species: New Interdisciplinary Studies","authors":"Richard Boyd","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chander and Srinivasan, 2018] Ajay Chander and Ramya Srinivasan. Cdmake workshop on make explainable ai. Evaluating Explanations by Cognitive Value, 2018.","title":"Cdmake workshop on make explainable ai","authors":"Srinivasan Chander, Ajay Chander, Ramya Srinivasan","year":"2018","venue":"Evaluating Explanations by Cognitive Value","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chander et al., 2018] Ajay Chander, Ramya Srinivasan, Suhas Chelian, Jun Wang, and Kanji Uchino. Working with beliefs: Ai transparency in the enterprise. Explain- able Smart Systems Workshop, Intelligent user Interfaces, 2018.","title":"Working with beliefs: Ai transparency in the enterprise","authors":"Ajay Chander, Ramya Srinivasan, Suhas Chelian, Jun Wang, Kanji Uchino","year":"2018","venue":"Explainable Smart Systems Workshop, Intelligent user Interfaces","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cummins, 2000] Robert Cummins. How does it work?\" versus \"what are the laws?\" two conceptions of psycho- logical explanation. Explanation and Cognition, 2000.","title":"How does it work?\" versus \"what are the laws?\" two conceptions of psychological explanation","authors":"Robert Cummins","year":"2000","venue":"Explanation and Cognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dennett, 1987] Daniel Dennett. The intentional stance. MIT Press, 1987.","title":"The intentional stance","authors":"Daniel Dennett","year":"1987","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Doshi and Kim, 2017] Finale Doshi and Been Kim. To- wards a rigorous science of interpretable machine learn- ing. ArXiv, 2017.","title":"Finale Doshi and Been Kim. Towards a rigorous science of interpretable machine learning","authors":"Kim Doshi","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Floyd and Aha, 2017] Michael Floyd and David Aha. Using explanations to provide transparency during trust-guided behavior adaptation. AI Communication, 2017.","title":"Using explanations to provide transparency during trust-guided behavior adaptation","authors":"Michael Floyd, David Aha","year":"2017","venue":"AI Communication","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Friedman, 1974] Micheal Friedman. Explanations and sci- entific understanding. Journal of philosophy, 1974.","title":"Explanations and scientific understanding","authors":"Micheal Friedman","year":"1974","venue":"Journal of philosophy","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[GDPR, 2017] GDPR. General data protection regulation. EUGDPR, 2017.","title":"GDPR. General data protection regulation","authors":"Gdpr","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gentner and Stevens, 1983] Dedre Gentner and Albert Stevens. Mental models. Hillsdale NJ, 1983.","title":"Mental models","authors":"Dedre Gentner, Albert Stevens","year":"1983","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gentner, 1983] Dedre Gentner. Structure-mapping: a theo- retical framework for analogy. Cognitive Science, 1983.","title":"Structure-mapping: a theoretical framework for analogy","authors":"Dedre Gentner, Gentner","year":"1983","venue":"Cognitive Science","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Goyal et al., 2019] Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and S. Lee. Counterfactual visual ex- planations. ICML, 2019.","title":"Counterfactual visual explanations","authors":"Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, S Lee","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gunning, 2017] Dave Gunning. Explainable artificial in- telligence (xai). Defense Advanced Research Projects Agency (DARPA), 2017.","title":"Explainable artificial intelligence (xai)","authors":"Dave Gunning","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Harman, 1965] Gilbert Harman. The inference to the best explanation. Philosophical Review, 1965.","title":"The inference to the best explanation","authors":"Gilbert Harman","year":"1965","venue":"Philosophical Review","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hempel and Oppenheim, 1948] Carl G. Hempel and Paul Oppenheim. Studies in the logic of explanation. Philoso- phy of Science, 15(2):135-175, April 1948.","title":"Studies in the logic of explanation","authors":"Carl G Hempel, Paul Oppenheim","year":"1948","venue":"Philosophy of Science","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Holland et al., 1986] John Holland, Holyoak Keith, Nisbett Richard, and Thagard Paul. Induction: Processes of in- ference, learning, and discovery. Cambridge University Press: MIT, 1986.","title":"Induction: Processes of inference, learning, and discovery","authors":"John Holland, Holyoak Keith, Nisbett Richard, Thagard Paul","year":"1986","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Johnson et al., 2017] Samuel Johnson, J. J. Valenti, and Frank C. Keil. Opponent uses of simplicity and complex- ity in causal explanation. CogSci, 2017.","title":"Opponent uses of simplicity and complexity in causal explanation","authors":"J J Samuel Johnson, Frank C Valenti, Keil","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Johnson-Laird, 1983] Philip Johnson-Laird. Mental mod- els. Harvard University Press, 1983.","title":"Mental models","authors":"Philip Johnson-Laird","year":"1983","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Joseph, 1988] Pitt Joseph. Theories of explanations. Oxford University Press, 1988.","title":"Pitt Joseph. Theories of explanations","authors":"Joseph","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kanehira and Harada, 2019] Atsushi Kanehira and Tatsuya Harada. Learning to explain with complemental examples. CVPR, 2019.","title":"Learning to explain with complemental examples","authors":"Harada Kanehira, Atsushi Kanehira, Tatsuya Harada","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Karpatne et al., 2017] Anuj Karpatne, Gowtham Atluri, James Faghmous, Michael Steinbach, Arindam Baner- jee, Aroop Ganguly, Shashi Shekhar, Nagiza Samatova, and Vipin Kumar. Theory-guided data science: A new paradigm for scientific discovery from data. IEEE Trans- actions on Knowledge and Data Engineering, 2017.","title":"Theory-guided data science: A new paradigm for scientific discovery from data","authors":"Anuj Karpatne, Gowtham Atluri, James Faghmous, Michael Steinbach, Arindam Banerjee, Aroop Ganguly, Shashi Shekhar, Nagiza Samatova, Vipin Kumar","year":"2017","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Keil, 2006] Frank Keil. Explanations and understanding. Annual Review of Psychology, 2006.","title":"Explanations and understanding","authors":"Frank Keil","year":"2006","venue":"Annual Review of Psychology","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Koh and Liang, 2017] Pang Wei Koh and Percy Liang. Un- derstanding black-box predictions via influence functions. International Conference on Machine Learning, 2017.","title":"Understanding black-box predictions via influence functions","authors":"Pang Wei, Koh, Percy Liang","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Kozhevnikov and Hegarty, 2001] Maria Kozhevnikov and Mary Hegarty. Impetus beliefs as default heuristics: dis- sociation between explicit and implicit knowledge about motion. Psychon Bull Rev., 2001.","title":"Impetus beliefs as default heuristics: dissociation between explicit and implicit knowledge about motion","authors":"Maria Kozhevnikov, Mary Hegarty","year":"2001","venue":"Psychon Bull Rev","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Kyndi, 2018] Kyndi. How explainability is driving the fu- ture of artificial intelligence. White Paper, 2018.","title":"Kyndi. How explainability is driving the future of artificial intelligence","authors":"Kyndi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[kyoung Ahn, 1998] Woo kyoung Ahn. Why are different features central for natural kinds and artifacts? the role of causal status in determining feature centrality. Cognition, 1998.","title":"Why are different features central for natural kinds and artifacts? the role of causal status in determining feature centrality","authors":"; Kyoung Ahn, Woo Kyoung Ahn","year":"1998","venue":"Cognition","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Lage et al., 2019] Isaac Lage, Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Greshman, and Finale Doshi. An evaluation of the human-interpretability of ex- planation. ArXiv, 2019.","title":"An evaluation of the human-interpretability of explanation","authors":"Isaac Lage, Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Greshman, Finale Doshi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Lipton, 2004] Peter Lipton. What good is an explanation. Explanations: Styles of Explanation in Science: Oxford University Press, 2004.","title":"What good is an explanation","authors":"Peter Lipton","year":"2004","venue":"Explanations: Styles of Explanation in Science","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Lombrozo, 2006] Tania Lombrozo. The structure and func- tions of explanation. Trends in Cognitive Science, 2006.","title":"The structure and functions of explanation","authors":"Tania Lombrozo","year":"2006","venue":"Trends in Cognitive Science","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Mayes, 2019] Randolph Mayes. Theories of explanation. Internet Encyclopedia of Philosophy, 2019.","title":"Theories of explanation","authors":"Randolph Mayes","year":"2019","venue":"Internet Encyclopedia of Philosophy","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Miller, 2018] Tim Miller. Explanation in artificial intelli- gence: Insights from the social sciences. ArXiv, 2018.","title":"Explanation in artificial intelligence: Insights from the social sciences","authors":"Tim Miller, Miller","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Narayanan et al., 2018] Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Greshman, and Finale Doshi and. How do humans understand explanations from ma- chine learning systems? an evaluation of the human- interpretability of explanation. ArXiv, 2018.","title":"How do humans understand explanations from machine learning systems? an evaluation of the humaninterpretability of explanation","authors":"Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Greshman, Finale Doshi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Park et al., 2018] Dong Park, Lisa Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. Multimodal explanations: Justifying decisions and pointing to the evidence. CoRRabs, 2018.","title":"Multimodal explanations: Justifying decisions and pointing to the evidence","authors":"Dong Park, Lisa Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, Marcus Rohrbach","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[PWC, 2018] PWC. Explainable ai driving business value through greater understanding. White Paper, Intelligent Digital, 2018.","title":"PWC. Explainable ai driving business value through greater understanding","authors":"Pwc","year":"2018","venue":"White Paper, Intelligent Digital","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ras et al., 2018] Gabrielle Ras, Marcel van Gerven, and Pim Haselager. Explanation methods in deep learning: Users, values, concerns and challenges. ArXiv, 2018.","title":"Explanation methods in deep learning: Users, values, concerns and challenges","authors":"Ras","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Rips, 2002] Lance Rips. Circular reasoning. Cogn. Science, 2002.","title":"Circular reasoning","authors":"Lance Rips","year":"2002","venue":"Cogn. Science","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Rosemary, 1999] Roberts Rosemary. What makes an expla- nation a good explanation? : adult learners' criteria for acceptance of a good explanation. Masters Thesis: Uni- versity of Newfoundland, 1999.","title":"What makes an explanation a good explanation? : adult learners' criteria for acceptance of a good explanation","authors":"Rosemary Roberts","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Ross et al., 1976] Lee Ross, David Greene, and Pamela House. The \"false consensus effect\": An egocentric bias in social perception and attribution processes. Journal of Experimental Psychology, 1976.","title":"The \"false consensus effect\": An egocentric bias in social perception and attribution processes","authors":"Lee Ross, David Greene, Pamela House","year":"1976","venue":"Journal of Experimental Psychology","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Ruben, 1990] David-Hillel Ruben. Explaining explanation. Routledge Publications, 1990. [Salmon, 1989] Salmon. Four decades of scientific explana- tion. Minneapolis University Press, pages 3-219, 1989.","title":"Explaining explanation","authors":"David-Hillel Ruben","year":"1989","venue":"Salmon. Four decades of scientific explanation","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Sober, 1984] Elliott Sober. Common cause explanation. Philos Sci., 1984.","title":"Common cause explanation","authors":"Elliott Sober","year":"1984","venue":"Philos Sci","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Srinivasan et al., 2018] Ramya Srinivasan, Ajay Chander, and Pouya Pezeshkpour. Generating user friendly expla- nations for loan denials using gans. NeurIPS Workshop, 2018.","title":"Generating user friendly explanations for loan denials using gans","authors":"Ramya Srinivasan, Ajay Chander, Pouya Pezeshkpour","year":"2018","venue":"NeurIPS Workshop","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Vigano and Magazzeni, 2018] Luca Vigano and Daniele Magazzeni. Explainable security. IJCAI XAI workshop, 2018.","title":"Explainable security","authors":"Luca Vigano, Daniele Magazzeni","year":"2018","venue":"IJCAI XAI workshop","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}