{"type":"ScholarlyArticle","title":"On Tackling Explanation Redundancy in Decision Trees (Extended Abstract)","authors":[{"name":"Yacine Izza","slug":"yacine-izza","given":"Yacine","family":"Izza","affiliation":"CREATE , NUS , Singapore"},{"name":"Alexey Ignatiev","slug":"alexey-ignatiev","given":"Alexey","family":"Ignatiev","affiliation":"Monash University , Australia"},{"name":"Joao Marques-Silva","slug":"joao-marques-silva","given":"Joao","family":"Marques-Silva","affiliation":"IRIT , CNRS , France"}],"abstract":"Claims about the interpretability of decision trees can be traced back to the origins of machine learning (ML). Indeed, given some input consistent with a decision tree's path, the explanation for  the resulting prediction consists of the features in that path.   Moreover, a growing number of works propose the use of decision trees, and of other so-called interpretable models, as a possible solution for deploying ML models in high-risk applications. This paper overviews recent theoretical and practical results which demonstrate that for most decision trees, tree paths exhibit so-called explanation redundancy, in that logically sound explanations can often be significantly more succinct than what the features in the path dictates.  More importantly, such decision tree explanations can be computed in polynomial-time, and so can be produced with essentially no effort other than traversing the decision tree. The experimental results, obtained on a large range of publicly available decision trees, support the paper's claims.","doi":"10.24963/ijcai.2023/779","url":"https://ijcai.org/proceedings/2023/779","pdf":"https://ijcai.org/proceedings/2023/0779.pdf","markdown":"https://ijcai.org/proceedings/2023/779.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/779","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Journal Track","track":"Journal Track","number":779,"pages":{"first":6900,"last":6904},"keywords":["Machine Learning: ML: Symbolic methods","Knowledge Representation and Reasoning: KRR: Diagnosis and abductive reasoning","AI Ethics, Trust, Fairness: ETF: Trustworthy AI","Machine Learning: ML: Explainable/Interpretable machine learning","Constraint Satisfaction and Optimization: CSO: Constraint satisfaction","Knowledge Representation and Reasoning: KRR: Automated reasoning and theorem proving","Constraint Satisfaction and Optimization: CSO: Satisfiabilty"],"videos":[],"references":[{"position":1,"raw":"[Amgoud, 2021] Leila Amgoud. Non-monotonic explana- tion functions. In ECSQARU, pages 19-31, 2021.","title":"Non-monotonic explanation functions","authors":"Leila Amgoud","year":"2021","venue":"ECSQARU","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arenas et al., 2022] Marcelo Arenas, Pablo Barceló, Miguel Romero, and Bernardo Subercaseaux. On com- puting probabilistic explanations for decision trees. In NeurIPS, 2022.","title":"On computing probabilistic explanations for decision trees","authors":"Marcelo Arenas, Pablo Barceló, Miguel Romero, Bernardo Subercaseaux","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Audemard et al., 2020] Gilles Audemard, Frédéric Koriche, and Pierre Marquis. On tractable XAI queries based on compiled representations. In KR, pages 838-849, 2020.","title":"On tractable XAI queries based on compiled representations","authors":"Gilles Audemard, Frédéric Koriche, Pierre Marquis","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Audemard et al., 2021] Gilles Audemard, Steve Bellart, Louenas Bounia, Frédéric Koriche, Jean-Marie Lagniez, and Pierre Marquis. On the computational intelligibility of boolean classifiers. In KR, pages 74-86, 2021.","title":"On the computational intelligibility of boolean classifiers","authors":"Gilles Audemard, Steve Bellart, Louenas Bounia, Frédéric Koriche, Jean-Marie Lagniez, Pierre Marquis","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Barceló et al., 2020] Pablo Barceló, Mikaël Monet, Jorge Pérez, and Bernardo Subercaseaux. Model interpretability through the lens of computational complexity. In NeurIPS, 2020.","title":"Model interpretability through the lens of computational complexity","authors":"Barceló","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bertsimas and Dunn, 2017] Dimitris Bertsimas and Jack Dunn. Optimal classification trees. Mach. Learn., 106(7):1039-1082, 2017.","title":"Optimal classification trees","authors":"Dimitris Bertsimas, Jack Dunn","year":"2017","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Breiman et al., 1984] Leo Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984.","title":"Classification and Regression Trees","authors":"J H Leo Breiman, R A Friedman, C J Olshen, Stone","year":"1984","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Breiman, 2001] Leo Breiman. Statistical modeling: The two cultures. Statistical science, 16(3):199-231, 2001.","title":"Statistical modeling: The two cultures","authors":"Leo Breiman, Breiman","year":"2001","venue":"Statistical science","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Darwiche and Hirth, 2020] Adnan Darwiche and Auguste Hirth. On the reasons behind decisions. In ECAI, pages 712-720, 2020.","title":"On the reasons behind decisions","authors":"Hirth Darwiche, Adnan Darwiche, Auguste Hirth","year":"2020","venue":"ECAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ghiasi et al., 2020] Mohammad M. Ghiasi, Sohrab Zende- hboudi, and Ali Asghar Mohsenipour. Decision tree-based diagnosis of coronary artery disease: CART model. Com- put. Methods Programs Biomed., 192:105400, 2020.","title":"Decision tree-based diagnosis of coronary artery disease: CART model","authors":"Mohammad M Ghiasi, Sohrab Zendehboudi, Ali Asghar, Mohsenipour","year":"2020","venue":"Comput. Methods Programs Biomed","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gorji and Rubin, 2022] Niku Gorji and Sasha Rubin. Suf- ficient reasons for classifier decisions in the presence of domain constraints. In AAAI, February 2022.","title":"Sufficient reasons for classifier decisions in the presence of domain constraints","authors":"Rubin ; Gorji, Niku Gorji, Sasha Rubin","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hu et al., 2019] Xiyang Hu, Cynthia Rudin, and Margo I. Seltzer. Optimal sparse decision trees. In NeurIPS, pages 7265-7273, 2019.","title":"Optimal sparse decision trees","authors":"Xiyang Hu, Cynthia Rudin, Margo I Seltzer","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2021] Xuanxiang Huang, Yacine Izza, Alexey Ignatiev, and Joao Marques-Silva. On efficiently explain- ing graph-based classifiers. In KR, pages 356-367, 2021.","title":"On efficiently explaining graph-based classifiers","authors":"Huang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang et al., 2022] Xuanxiang Huang, Yacine Izza, Alexey Ignatiev, Martin C. Cooper, Nicholas Asher, and Joao Marques-Silva. Tractable explanations for d-DNNF clas- sifiers. In AAAI, February 2022.","title":"Tractable explanations for d-DNNF classifiers","authors":"Huang","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ignatiev and Marques-Silva, 2021] Alexey Ignatiev and Joao Marques-Silva. SAT-based rigorous explanations for decision lists. In SAT, pages 251-269, 2021.","title":"SAT-based rigorous explanations for decision lists","authors":"Alexey Ignatiev, Joao Marques-Silva","year":"2021","venue":"SAT","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ignatiev et al., 2019] Alexey Ignatiev, Nina Narodytska, and Joao Marques-Silva. Abduction-based explanations for machine learning models. In AAAI, pages 1511-1519, 2019.","title":"Abduction-based explanations for machine learning models","authors":"Alexey Ignatiev, Nina Narodytska, Joao Marques-Silva","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ignatiev et al., 2020] Alexey Ignatiev, Nina Narodytska, Nicholas Asher, and Joao Marques-Silva. From con- trastive to abductive explanations and back again. In AIxIA, pages 335-355, 2020.","title":"From contrastive to abductive explanations and back again","authors":"Alexey Ignatiev, Nina Narodytska, Nicholas Asher, Joao Marques-Silva","year":"2020","venue":"AIxIA","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ignatiev et al., 2022] Alexey Ignatiev, Yacine Izza, Peter Stuckey, and Joao Marques-Silva. Using MaxSAT for ef- ficient explanations of tree ensembles. In AAAI, February 2022.","title":"Using MaxSAT for efficient explanations of tree ensembles","authors":"Alexey Ignatiev, Yacine Izza, Peter Stuckey, Joao Marques-Silva","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Izza and Marques-Silva, 2021] Yacine Izza and Joao Marques-Silva. On explaining random forests with SAT. In IJCAI, pages 2584-2591, 2021.","title":"On explaining random forests with SAT","authors":"Marques-Silva ; Yacine Izza, Joao Izza, Marques-Silva","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/356"},{"position":20,"raw":"[Izza et al., 2020] Yacine Izza, Alexey Ignatiev, and Joao Marques-Silva. On explaining decision trees. CoRR, abs/2010.11034, 2020.","title":"On explaining decision trees","authors":"Yacine Izza, Alexey Ignatiev, Joao Marques-Silva","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Izza et al., 2022] Yacine Izza, Alexey Ignatiev, and Joao Marques-Silva. On tackling explanation redundancy in de- cision trees. J. Artif. Intell. Res., 75:261-321, 2022.","title":"On tackling explanation redundancy in decision trees","authors":"Yacine Izza, Alexey Ignatiev, Joao Marques-Silva","year":"2022","venue":"J. Artif. Intell. Res","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lelis et al., 2020] Viviane Maria Lelis, Eduardo Guzmán, and María-Victoria Belmonte. Non-invasive meningitis diagnosis using decision trees. IEEE Access, 8:18394- 18407, 2020.","title":"Non-invasive meningitis diagnosis using decision trees","authors":"Maria Viviane, Eduardo Lelis, María-Victoria Guzmán, Belmonte","year":"2020","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lin et al., 2020] Jimmy Lin, Chudi Zhong, Diane Hu, Cyn- thia Rudin, and Margo I. Seltzer. Generalized and scalable optimal sparse decision trees. In ICML, pages 6150-6160, 2020.","title":"Generalized and scalable optimal sparse decision trees","authors":"Jimmy Lin, Chudi Zhong, Diane Hu, Cynthia Rudin, Margo I Seltzer","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu and Lorini, 2021] Xinghan Liu and Emiliano Lorini. A logic for binary classifiers and their explanation. In CLAR, 2021.","title":"Xinghan Liu and Emiliano Lorini. A logic for binary classifiers and their explanation","authors":"Lorini Liu","year":"2021","venue":"CLAR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Marques-Silva and Ignatiev, 2022] Joao Marques-Silva and Alexey Ignatiev. Delivering trustworthy AI through formal XAI. In AAAI, 2022.","title":"Delivering trustworthy AI through formal XAI","authors":"Joao Marques, - Silva, Alexey Ignatiev","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Marques-Silva, 2022] Joao Marques-Silva. Logic-based ex- plainability in machine learning. CoRR, abs/2211.00541, 2022.","title":"Logic-based explainability in machine learning","authors":"Joao Marques-Silva","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Molnar, 2020] Christoph Molnar. Interpretable Machine Learning. Leanpub, 2020. http://tiny.cc/6c76tz.","title":"Interpretable Machine Learning","authors":"Christoph Molnar","year":"2020","venue":null,"doi":null,"url":"http://tiny.cc/6c76tz","internal_url":null},{"position":28,"raw":"[Quinlan, 1986] J. Ross Quinlan. Induction of decision trees. Mach. Learn., 1(1):81-106, 1986.","title":"Induction of decision trees","authors":"J, Ross Quinlan","year":"1986","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Rudin, 2019] Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206-215, 2019.","title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","authors":"Cynthia Rudin","year":"2019","venue":"Nature Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Shih et al., 2018] Andy Shih, Arthur Choi, and Adnan Dar- wiche. A symbolic approach to explaining bayesian net- work classifiers. In IJCAI, pages 5103-5111, 2018.","title":"A symbolic approach to explaining bayesian network classifiers","authors":"Andy Shih, Arthur Choi, Adnan Darwiche","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/708"},{"position":31,"raw":"[Utgoff et al., 1997] Paul E. Utgoff, Neil C. Berkman, and Jeffery A. Clouse. Decision tree induction based on effi- cient tree restructuring. Mach. Learn., 29(1):5-44, 1997.","title":"Decision tree induction based on efficient tree restructuring","authors":"Paul E Utgoff, Neil C Berkman, Jeffery A Clouse","year":"1997","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Verwer and Zhang, 2019] Sicco Verwer and Yingqian Zhang. Learning optimal classification trees using a binary linear program formulation. In AAAI, pages 1625-1632, 2019.","title":"Learning optimal classification trees using a binary linear program formulation","authors":"Zhang Verwer, Sicco Verwer, Yingqian Zhang","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}