{"type":"ScholarlyArticle","title":"Choose your Data Wisely: A Framework for Semantic Counterfactuals","authors":[{"name":"Edmund Dervakos","slug":"edmund-dervakos","given":"Edmund","family":"Dervakos","affiliation":"National Technical University of Athens {eddiedervakos ,"},{"name":"Konstantinos Thomas","slug":"konstantinos-thomas","given":"Konstantinos","family":"Thomas","affiliation":"National Technical University of Athens {eddiedervakos ,"},{"name":"Giorgos Filandrianos","slug":"giorgos-filandrianos","given":"Giorgos","family":"Filandrianos","affiliation":"National Technical University of Athens {eddiedervakos ,"},{"name":"Giorgos Stamou","slug":"giorgos-stamou","given":"Giorgos","family":"Stamou","affiliation":"National Technical University of Athens {eddiedervakos ,"}],"abstract":"Counterfactual explanations have been argued to be one of the most intuitive forms of explanation. They are typically defined as a minimal set of edits on a given data sample that, when applied, changes the output of a model on that sample. However, a minimal set of edits is not always clear and understandable to an end-user, as it could constitute an adversarial example (which is indistinguishable from the original data sample to an end-user). Instead, there are recent ideas that the notion of minimality in the context of counterfactuals should refer to the semantics of the data sample, and not to the feature space. In this work, we build on these ideas, and propose a framework that provides counterfactual explanations in terms of knowledge graphs. We provide an algorithm for computing such explanations (given some assumptions about the underlying knowledge), and quantitatively evaluate the framework with a user study.","doi":"10.24963/ijcai.2023/43","url":"https://ijcai.org/proceedings/2023/43","pdf":"https://ijcai.org/proceedings/2023/0043.pdf","markdown":"https://ijcai.org/proceedings/2023/43.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/43","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":43,"pages":{"first":382,"last":390},"keywords":["Knowledge Representation and Reasoning: KRR: Applications","Machine Learning: ML: Explainable/Interpretable machine learning","AI Ethics, Trust, Fairness: ETF: Explainability and interpretability"],"videos":[],"references":[{"position":1,"raw":"[Abu-Aisheh et al., 2015] Zeina Abu-Aisheh, Romain Raveaux, Jean-Yves Ramel, and Patrick Martineau. An exact graph edit distance algorithm for solving pattern recognition problems. In 4th International Conference on Pattern Recognition Applications and Methods 2015, 2015.","title":"An exact graph edit distance algorithm for solving pattern recognition problems","authors":"Abu-Aisheh","year":"2015","venue":"4th International Conference on Pattern Recognition Applications and Methods","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Akula et al., 2020] Arjun Akula, Shuai Wang, and Song- Chun Zhu. Cocox: Generating conceptual and counter- factual explanations via fault-lines. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 2594-2601, 2020.","title":"Cocox: Generating conceptual and counterfactual explanations via fault-lines","authors":"Arjun Akula, Shuai Wang, Song-Chun Zhu","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Baader et al., 2003] Franz Baader, Diego Calvanese, Deb- orah L. McGuinness, Daniele Nardi, and Peter F. Patel- Schneider, editors. The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, 2003.","title":"The Description Logic Handbook: Theory, Implementation, and Applications","authors":"Franz Baader, Diego Calvanese, L Deborah","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Browne and Swift, 2020] Kieran Browne and Ben Swift. Semantics and explanation: why counterfactual explana- tions produce adversarial examples in deep neural net- works. arXiv preprint arXiv:2012.10076, 2020.","title":"Semantics and explanation: why counterfactual explanations produce adversarial examples in deep neural networks","authors":"Kieran Browne, Ben Swift","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cong et al., 2022] Yuren Cong, Michael Ying Yang, and Bodo Rosenhahn. Reltr: Relation transformer for scene graph generation. CoRR, abs/2201.11460, 2022.","title":"Reltr: Relation transformer for scene graph generation","authors":"Cong","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"['Amato et al., 2009] Claudia d'Amato, Nicola Fanizzi, and Floriana Esposito. A semantic similarity mea- sure for expressive description logics. arXiv preprint arXiv:0911.5043, 2009.","title":"A semantic similarity measure for expressive description logics","authors":"Amato","year":"2009","venue":"Claudia d","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Filandrianos et al., 2022] Giorgos Filandrianos, Konstanti- nos Thomas, Edmund Dervakos, and Giorgos Stamou. Conceptual edits as counterfactual explanations. AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering, 2022.","title":"Conceptual edits as counterfactual explanations","authors":"Giorgos Filandrianos, Konstantinos Thomas, Edmund Dervakos, Giorgos Stamou","year":"2022","venue":"AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goyal et al., 2019] Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. Counterfactual vi- sual explanations. In International Conference on Ma- chine Learning, pages 2376-2384. PMLR, 2019.","title":"Counterfactual visual explanations","authors":"Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, Stefan Lee","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hoffman et al., 2018] Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman. Metrics for explain- able ai: Challenges and prospects. arXiv preprint arXiv:1812.04608, 2018.","title":"Metrics for explainable ai: Challenges and prospects","authors":"Shane T Robert R Hoffman, Gary Mueller, Jordan Klein, Litman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hogan et al., 2021] Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d'Amato, Gerard De Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, et al. Knowledge graphs. ACM Computing Surveys (CSUR), 54(4):1-37, 2021.","title":"Knowledge graphs","authors":"Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia Amato, Gerard De Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra, Roberto Gayo, Sebastian Navigli, Neumaier","year":"2021","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Karp, 1980] Richard M Karp. An algorithm to solve the m× n assignment problem in expected time o (mn log n). Networks, 10(2):143-152, 1980.","title":"An algorithm to solve the m× n assignment problem in expected time o (mn log n)","authors":"M Richard, Karp","year":"1980","venue":"Networks","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krishna et al., 2017] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. Inter- national journal of computer vision, 123(1):32-73, 2017.","title":"Visual genome: Connecting language and vision using crowdsourced dense image annotations","authors":"Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma","year":"2017","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In European conference on com- puter vision, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Miller, 1995] George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39-41, 1995.","title":"Wordnet: a lexical database for english","authors":"George A Miller","year":"1995","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Poyiadzi et al., 2020] Rafael Poyiadzi, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. Face: fea- sible and actionable counterfactual explanations. In Pro- ceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 344-350, 2020.","title":"Face: feasible and actionable counterfactual explanations","authors":"Poyiadzi","year":"2020","venue":"Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rudin, 2019] Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206-215, 2019.","title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","authors":"Cynthia Rudin","year":"2019","venue":"Nature Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sanfeliu and Fu, 1983] Alberto Sanfeliu and King-Sun Fu. A distance measure between attributed relational graphs for pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13(3):353-362, 1983.","title":"A distance measure between attributed relational graphs for pattern recognition","authors":"Alberto Sanfeliu, King-Sun Fu","year":"1983","venue":"IEEE Transactions on Systems, Man, and Cybernetics, SMC","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sharma et al., 2020] Neeraj Sharma, Prashant Krishnan, Rohit Kumar, Shreyas Ramoji, Srikanth Raj Chetu- palli, Prasanta Kumar Ghosh, Sriram Ganapathy, et al. Coswara-a database of breathing, cough, and voice sounds for covid-19 diagnosis. arXiv preprint arXiv:2005.10548, 2020.","title":"Coswara-a database of breathing, cough, and voice sounds for covid-19 diagnosis","authors":"Neeraj Sharma, Prashant Krishnan, Rohit Kumar, Shreyas Ramoji, Raj Srikanth, Prasanta Chetupalli, Sriram Kumar Ghosh, Ganapathy","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Simonyan et al., 2013] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.","title":"Deep inside convolutional networks: Visualising image classification models and saliency maps","authors":"Karen Simonyan, Andrea Vedaldi, Andrew Zisserman","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vandenhende et al., 2022] Simon Vandenhende, Dhruv Ma- hajan, Filip Radenovic, and Deepti Ghadiyaram. Mak- ing heads or tails: Towards semantically consistent visual counterfactuals. arXiv preprint arXiv:2203.12892, 2022.","title":"Making heads or tails: Towards semantically consistent visual counterfactuals","authors":"Simon Vandenhende, Dhruv Mahajan, Filip Radenovic, Deepti Ghadiyaram","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Verma et al., 2020] Sahil Verma, John Dickerson, and Kee- gan Hines. Counterfactual explanations for machine learn- ing: A review. arXiv preprint arXiv:2010.10596, 2020.","title":"Counterfactual explanations for machine learning: A review","authors":"Sahil Verma, John Dickerson, Keegan Hines","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zeng et al., 2009] Zhiping Zeng, Anthony KH Tung, Jiany- ong Wang, Jianhua Feng, and Lizhu Zhou. Comparing stars: On approximating graph edit distance. Proceedings of the VLDB Endowment, 2(1):25-36, 2009.","title":"Comparing stars: On approximating graph edit distance","authors":"Zhiping Zeng, K H Anthony, Jianyong Tung, Jianhua Wang, Lizhu Feng, Zhou","year":"2009","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2018] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 40(6):1452-1464, 2018.","title":"Places: A 10 million image database for scene recognition","authors":"Zhou","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}