{"type":"ScholarlyArticle","title":"Learning Small Decision Trees with Large Domain","authors":[{"name":"Eduard Eiben","slug":"eduard-eiben","given":"Eduard","family":"Eiben","affiliation":"Royal Holloway University of London , UK"},{"name":"Sebastian Ordyniak","slug":"sebastian-ordyniak","given":"Sebastian","family":"Ordyniak","affiliation":"University of Leeds , UK"},{"name":"Giacomo Paesani","slug":"giacomo-paesani","given":"Giacomo","family":"Paesani","affiliation":"University of Leeds , UK"},{"name":"Stefan Szeider","slug":"stefan-szeider","given":"Stefan","family":"Szeider","affiliation":"Algorithms and Complexity Group , TU Wien , Vienna , Austria"}],"abstract":"One favors decision trees (DTs) of the smallest size or depth to facilitate explainability and interpretability. However, learning such an optimal DT from data is well-known to be NP-hard. To overcome this complexity barrier, Ordyniak and Szeider (AAAI 21) initiated the study of optimal DT learning under the parameterized complexity perspective. They showed that solution size (i.e., number of nodes or depth of the DT) is insufficient to obtain fixed-parameter tractability (FPT). Therefore, they proposed an FPT algorithm that utilizes two auxiliary parameters: the maximum difference (as a structural property of the data set) and maximum domain size. They left it as an open question of whether bounding the maximum domain size is necessary.\r\n\r\nThe main result of this paper answers this question. We present FPT algorithms for learning a smallest or  lowest-depth DT from data, with the only parameters solution size and maximum difference. Thus, our algorithm is significantly more potent than the one by Szeider and Ordyniak as it can handle problem inputs with features that range over unbounded domains. We also close several gaps concerning the quality of approximation one obtains by only considering DTs based on minimum support sets.","doi":"10.24963/ijcai.2023/355","url":"https://ijcai.org/proceedings/2023/355","pdf":"https://ijcai.org/proceedings/2023/0355.pdf","markdown":"https://ijcai.org/proceedings/2023/355.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/355","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":355,"pages":{"first":3184,"last":3192},"keywords":["Knowledge Representation and Reasoning: KRR: Computational complexity of reasoning","Machine Learning: ML: Explainable/Interpretable machine learning"],"videos":[],"references":[{"position":1,"raw":"[Aglin et al., 2020a] Gaël Aglin, Siegfried Nijssen, and Pierre Schaus. Learning optimal decision trees using caching branch-and-bound search. Proc. AAAI 2020, pages 3146-3153, 2020.","title":"Learning optimal decision trees using caching branch-and-bound search","authors":"Aglin","year":"2020","venue":"Proc. AAAI 2020","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aglin et al., 2020b] Gaël Aglin, Siegfried Nijssen, and Pierre Schaus. PyDL8.5: a library for learning optimal decision trees. Proc. IJCAI 2020, pages 5222-5224, 2020.","title":"PyDL8.5: a library for learning optimal decision trees","authors":"Aglin","year":"2020","venue":"Proc. IJCAI 2020","doi":null,"url":null,"internal_url":"/proceedings/2020/750"},{"position":3,"raw":"[Avellaneda, 2020] Florent Avellaneda. Efficient inference of optimal decision trees. Proc. AAAI 2020, pages 3195- 3202, 2020.","title":"Efficient inference of optimal decision trees","authors":"Florent Avellaneda","year":"2020","venue":"Proc. AAAI 2020","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bäckström et al., 2012] Christer Bäckström, Yue Chen, Pe- ter Jonsson, Sebastian Ordyniak, and Stefan Szeider. The complexity of planning revisited -a parameterized analy- sis. Proc. AAAI 2012, pages 1735-1741, 2012.","title":"The complexity of planning revisited -a parameterized analysis","authors":"Christer Bäckström, Yue Chen, Peter Jonsson, Sebastian Ordyniak, Stefan Szeider","year":"2012","venue":"Proc. AAAI 2012","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bertsimas and Dunn, 2017] Dimitris Bertsimas and Jack Dunn. Optimal classification trees. Machine Learning, 106(7):1039-1082, 2017.","title":"Optimal classification trees","authors":"Dimitris Bertsimas, Jack Dunn","year":"2017","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bessière et al., 2008] Christian Bessière, Emmanuel He- brard, Brahim Hnich, Zeynep Kiziltan, Claude-Guy Quim- per, and Toby Walsh. The parameterized complexity of global constraints. Proc. AAAI 2008, pages 235-240, 2008.","title":"The parameterized complexity of global constraints","authors":"Bessière","year":"2008","venue":"Proc. AAAI 2008","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bessiere et al., 2009] Christian Bessiere, Emmanuel He- brard, and Barry O'Sullivan. Minimising decision tree size as combinatorial optimisation. Proc. CP 2009, 5732:173- 187, 2009.","title":"Christian Bessiere, Emmanuel Hebrard, and Barry O'Sullivan. Minimising decision tree size as combinatorial optimisation","authors":"Bessiere","year":"2009","venue":"Proc. CP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bredereck et al., 2017] Robert Bredereck, Jiehua Chen, Rolf Niedermeier, and Toby Walsh. Parliamentary vot- ing procedures: Agenda control, manipulation, and uncer- tainty. Journal of Artificial Intelligence Research, 59:133- 173, 2017.","title":"Parliamentary voting procedures: Agenda control, manipulation, and uncertainty","authors":"Robert Bredereck, Jiehua Chen, Rolf Niedermeier, Toby Walsh","year":"2017","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cygan et al., 2015] Marek Cygan, Fedor V. Fomin, Łukasz Kowalik, Daniel Lokshtanov, Dániel Marx, Marcin Pilipczuk, Michał Pilipczuk, and Saket Saurabh. Parame- terized Algorithms. Springer, 2015.","title":"Parameterized Algorithms","authors":"Marek Cygan, V Fedor, Łukasz Fomin, Daniel Kowalik, Dániel Lokshtanov, Marcin Marx, Michał Pilipczuk, Saket Pilipczuk, Saurabh","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Darwiche and Hirth, 2023] Adnan Darwiche and Auguste Hirth. On the (complete) reasons behind decisions. Jour- nal of Logic, Language and Information, 32(1):63-88, 2023.","title":"On the (complete) reasons behind decisions","authors":"Darwiche, Hirth ;, Auguste Hirth","year":"2023","venue":"Language and Information","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Demirovic et al., 2022] Emir Demirovic, Anna Lukina, Emmanuel Hebrard, Jeffrey Chan, James Bailey, Christo- pher Leckie, Kotagiri Ramamohanarao, and Peter J. Stuckey. MurTree: Optimal decision trees via dynamic programming and search. Journal of Machine Learning Research, 23(26):1-47, 2022.","title":"MurTree: Optimal decision trees via dynamic programming and search","authors":"Demirovic","year":"2022","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Doshi-Velez and Kim, 2017] Finale Doshi-Velez and Been Kim. A roadmap for a rigorous science of interpretability. CoRR, abs/1702.08608, 2017.","title":"Finale Doshi-Velez and Been Kim. A roadmap for a rigorous science of interpretability","authors":"Kim Doshi-Velez","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Downey and Fellows, 2013] Rodney G. Downey and Michael R. Fellows. Fundamentals of Parameterized Complexity. Texts in Computer Science. Springer Verlag, 2013.","title":"Fundamentals of Parameterized Complexity","authors":"Rodney G Downey, Michael R Fellows","year":"2013","venue":"Texts in Computer Science","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Dvorák et al., 2012] Wolfgang Dvorák, Sebastian Ordy- niak, and Stefan Szeider. Augmenting tractable frag- ments of abstract argumentation. Artificial Intelligence, 186:157-173, 2012.","title":"Augmenting tractable fragments of abstract argumentation","authors":"Dvorák","year":"2012","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2011/177"},{"position":15,"raw":"[Ganian et al., 2018] Robert Ganian, Iyad Kanj, Sebastian Ordyniak, and Stefan Szeider. Parameterized algorithms for the matrix completion problem. Proc. ICML 2018, pages 1642-1651, 2018.","title":"Parameterized algorithms for the matrix completion problem","authors":"Robert Ganian, Iyad Kanj, Sebastian Ordyniak, Stefan Szeider","year":"2018","venue":"Proc. ICML 2018","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gaspers et al., 2017] Serge Gaspers, Neeldhara Misra, Se- bastian Ordyniak, Stefan Szeider, and Stanislav Zivný. Backdoors into heterogeneous classes of SAT and CSP. Journal of Computer and System Sciences, 85:38-56, 2017.","title":"Backdoors into heterogeneous classes of SAT and CSP","authors":"Serge Gaspers, Neeldhara Misra, Sebastian Ordyniak, Stefan Szeider, Stanislav Zivný","year":"2017","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Goodman and Flaxman, 2017] Bryce Goodman and Seth R. Flaxman. European union regulations on algorithmic decision-making and a \"right to explanation\". AI Maga- zine, 38(3):50-57, 2017.","title":"European union regulations on algorithmic decision-making and a \"right to explanation","authors":"Bryce Goodman, Seth R Flaxman","year":"2017","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Gottlob et al., 2002] Georg Gottlob, Francesco Scarcello, and Martha Sideri. Fixed-parameter complexity in AI and nonmonotonic reasoning. Artificial Intelligence, 138(1- 2):55-86, 2002.","title":"Fixed-parameter complexity in AI and nonmonotonic reasoning","authors":"Gottlob","year":"2002","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Hu et al., 2020] Hao Hu, Mohamed Siala, Emmanuel He- brard, and Marie-José Huguet. Learning optimal decision trees with MaxSAT and its integration in AdaBoost. Proc. IJCAI 2020, pages 1170-1176, 2020.","title":"Learning optimal decision trees with MaxSAT and its integration in AdaBoost","authors":"Hao Hu, Mohamed Siala, Emmanuel Hebrard, Marie-José Huguet","year":"2020","venue":"Proc. IJCAI 2020","doi":null,"url":null,"internal_url":"/proceedings/2020/163"},{"position":20,"raw":"[Hyafil and Rivest, 1976] Laurent Hyafil and Ronald L. Rivest. Constructing optimal binary decision trees is NP- complete. Information Processing Letters, 5(1):15-17, 1976.","title":"Constructing optimal binary decision trees is NPcomplete","authors":"Laurent Hyafil, Ronald L Rivest","year":"1976","venue":"Information Processing Letters","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ibaraki et al., 2011] Toshihide Ibaraki, Yves Crama, and Peter L. Hammer. Partially defined Boolean functions (in Boolean Functions: Theory, Algorithms, and Applica- tions). Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2011.","title":"Partially defined Boolean functions","authors":"Toshihide Ibaraki, Yves Crama, Peter L Hammer","year":"2011","venue":"Boolean Functions: Theory, Algorithms, and Applications). Encyclopedia of Mathematics and its Applications","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Janota and Morgado, 2020] Mikolás Janota and António Morgado. Sat-based encodings for optimal decision trees with explicit paths. Proc. SAT 2020, 12178:501-518, 2020.","title":"Mikolás Janota and António Morgado. Sat-based encodings for optimal decision trees with explicit paths","authors":"Morgado Janota","year":"2020","venue":"Proc. SAT","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Larose and Larose, 2014] Daniel T. Larose and Chantal D. Larose. Discovering Knowledge in Data: An Introduction to Data Mining. John Wiley & Sons, 2nd edition, 2014.","title":"Discovering Knowledge in Data: An Introduction to Data Mining","authors":"T Daniel, Chantal D Larose, Larose","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lipton, 2018] Zachary C. Lipton. The mythos of model in- terpretability. Communications of the ACM, 61(10):36-43, 2018.","title":"The mythos of model interpretability","authors":"Zachary C Lipton","year":"2018","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Monroe, 2018] Don Monroe. AI, explain yourself. AI Com- munications, 61(11):11-13, 2018.","title":"AI, explain yourself","authors":"Don Monroe","year":"2018","venue":"AI Communications","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Murthy, 1998] Sreerama K. Murthy. Automatic construc- tion of decision trees from data: A multi-disciplinary sur- vey. Data Mining and Knowledge Discovery, 2(4):345- 389, 1998.","title":"Automatic construction of decision trees from data: A multi-disciplinary survey","authors":"K Sreerama, Murthy","year":"1998","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Narodytska et al., 2018] Nina Narodytska, Alexey Ignatiev, Filipe Pereira, and Joao Marques-Silva. Learning optimal decision trees with SAT. Proc. IJCAI 2018, pages 1362- 1368, 2018.","title":"Learning optimal decision trees with SAT","authors":"Nina Narodytska, Alexey Ignatiev, Filipe Pereira, Joao Marques-Silva","year":"2018","venue":"Proc. IJCAI 2018","doi":null,"url":null,"internal_url":"/proceedings/2018/189"},{"position":28,"raw":"[Niedermeier, 2006] Rolf Niedermeier. Invitation to Fixed- Parameter Algorithms. Oxford Lecture Series in Mathe- matics and its Applications. Oxford University Press, Ox- ford, 2006.","title":"Invitation to Fixed-Parameter Algorithms","authors":"Rolf Niedermeier","year":"2006","venue":"Oxford Lecture Series in Mathematics and its Applications","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Ordyniak and Szeider, 2021] Sebastian Ordyniak and Ste- fan Szeider. Parameterized complexity of small decision tree learning. Proc. AAAI 2021, pages 6454-6462, 2021.","title":"Parameterized complexity of small decision tree learning","authors":"Sebastian Ordyniak, Stefan Szeider","year":"2021","venue":"Proc. AAAI 2021","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Quinlan, 1986] J. Ross Quinlan. Induction of decision trees. Machine Learning, 1(1):81-106, 1986.","title":"Induction of decision trees","authors":"J, Ross Quinlan","year":"1986","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Samer and Szeider, 2010] Marko Samer and Stefan Szeider. Constraint satisfaction with bounded treewidth revisited. Journal of Computer and System Sciences, 76(2):103-114, 2010.","title":"Constraint satisfaction with bounded treewidth revisited","authors":"Marko Samer, Stefan Szeider","year":"2010","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Schidler and Szeider, 2021] André Schidler and Stefan Szeider. SAT-based decision tree learning for large data sets. Proc. AAAI 2021, pages 3904-3912, 2021.","title":"SAT-based decision tree learning for large data sets","authors":"Szeider Schidler, André Schidler, Stefan Szeider","year":"2021","venue":"Proc. AAAI 2021","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Shati et al., 2021] Pouya Shati, Eldan Cohen, and Sheila A. McIlraith. SAT-based approach for learning optimal de- cision trees with non-binary features. Proc. CP 2021, 210(50):1-16, 2021.","title":"SAT-based approach for learning optimal decision trees with non-binary features","authors":"Pouya Shati, Eldan Cohen, Sheila A Mcilraith","year":"2021","venue":"Proc. CP","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Verhaeghe et al., 2020] Hélène Verhaeghe, Siegfried Ni- jssen, Gilles Pesant, Claude-Guy Quimper, and Pierre Schaus. Learning optimal decision trees using constraint programming. Constraints, 25(3-4):226-250, 2020.","title":"Learning optimal decision trees using constraint programming","authors":"Hélène Verhaeghe, Siegfried Nijssen, Gilles Pesant, Claude-Guy Quimper, Pierre Schaus","year":"2020","venue":"Constraints","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Verwer and Zhang, 2017] Sicco Verwer and Yingqian Zhang. Learning decision trees with flexible constraints and objectives using integer optimization. Proc. CPAIOR 2017, 10335:94-103, 2017.","title":"Learning decision trees with flexible constraints and objectives using integer optimization","authors":"Zhang Verwer, Sicco Verwer, Yingqian Zhang","year":"2017","venue":"Proc. CPAIOR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Verwer and Zhang, 2019] Sicco Verwer and Yingqian Zhang. Learning optimal classification trees using a binary linear program formulation. Proc. AAAI 2019, pages 1625-1632, 2019.","title":"Learning optimal classification trees using a binary linear program formulation","authors":"Zhang Verwer, Sicco Verwer, Yingqian Zhang","year":"2019","venue":"Proc. AAAI 2019","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhu et al., 2020] Haoran Zhu, Pavankumar Murali, Dzung T. Phan, Lam M. Nguyen, and Jayant Kalagnanam. A scalable MIP-based method for learning optimal multivariate decision trees. Proc. NeurIPS 2020, 2020.","title":"A scalable MIP-based method for learning optimal multivariate decision trees","authors":"Haoran Zhu, Pavankumar Murali, T Dzung, Phan, M Lam, Jayant Nguyen, Kalagnanam","year":"2020","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}