{"type":"ScholarlyArticle","title":"Learning Optimal Decision Trees using Constraint Programming (Extended Abstract)","authors":[{"name":"Hélène Verhaeghe","slug":"helene-verhaeghe","given":"Hélène","family":"Verhaeghe","affiliation":"UCLouvain , ICTEAM , Place Sainte Barbe 2 , 1348 Louvain-la-Neuve , Belgium"},{"name":"Siegfried Nijssen","slug":"siegfried-nijssen","given":"Siegfried","family":"Nijssen","affiliation":"UCLouvain , ICTEAM , Place Sainte Barbe 2 , 1348 Louvain-la-Neuve , Belgium"},{"name":"Gilles Pesant","slug":"gilles-pesant","given":"Gilles","family":"Pesant","affiliation":"Polytechnique Montréal , Montréal , Canada"},{"name":"Claude-Guy Quimper","slug":"claude-guy-quimper","given":"Claude-Guy","family":"Quimper","affiliation":"Université Laval , Québec , Canada"},{"name":"Pierre Schaus","slug":"pierre-schaus","given":"Pierre","family":"Schaus","affiliation":"UCLouvain , ICTEAM , Place Sainte Barbe 2 , 1348 Louvain-la-Neuve , Belgium"}],"abstract":"Decision trees are among the most popular classification models in machine learning. Traditionally, they are learned using greedy algorithms. However, such algorithms have their disadvantages: it is difficult to limit the size of the decision trees while maintaining a good classification accuracy, and it is hard to impose additional constraints on the models that are learned. For these reasons, there has been a recent interest in exact and flexible algorithms for learning decision trees. In this paper, we introduce a new approach to learn decision trees using constraint programming.\r\nCompared to earlier approaches, we show that our approach obtains better performance, while still being sufficiently flexible to allow for the inclusion of constraints. Our approach builds on three key building blocks: (1) the use of AND/OR search, (2) the use of caching, (3) the use of the CoverSize global constraint proposed recently for the problem of itemset mining. This allows our constraint programming approach to deal in a much more efficient way with the decompositions in the learning problem.","doi":"10.24963/ijcai.2020/662","url":"https://ijcai.org/proceedings/2020/662","pdf":"https://ijcai.org/proceedings/2020/0662.pdf","markdown":"https://ijcai.org/proceedings/2020/662.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/662","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Sister Conferences Best Papers","track":"Sister Conferences Best Papers","number":662,"pages":{"first":4765,"last":4769},"keywords":["Constraints and SAT: Constraints and Data Mining ; Constraints and Machine Learning","Constraints and SAT: Constraints: Modeling, Solvers, Applications","Constraints and SAT: Global Constraints","Constraints and SAT: Constraint Optimization"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25399/file_25399.m3u8","length_sec":247,"provider":"hls"}],"references":[{"position":1,"raw":"[Aghaei et al., 2019] Sina Aghaei, Mohammad Javad Azizi, and Phebe Vayanos. Learning optimal and fair decision trees for non-discriminative decision-making. 2019.","title":"Learning optimal and fair decision trees for non-discriminative decision-making","authors":"Aghaei","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aglin et al., 2020] Gaël Aglin, Siegfried Nijssen, and Pierre Schaus. Learning optimal decision trees using caching branch-and-bound search. In Thirty-Fourth AAAI Confer- ence on Artificial Intelligence, 2020.","title":"Learning optimal decision trees using caching branch-and-bound search","authors":"Siegfried Gaël Aglin, Pierre Nijssen, Schaus","year":"2020","venue":"Thirty-Fourth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Babaki et al., 2017] Behrouz Babaki, Tias Guns, and Luc De Raedt. Stochastic constraint programming with and- or branch-and-bound. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 539-545, 2017.","title":"Stochastic constraint programming with andor branch-and-bound","authors":"Behrouz Babaki, Tias Guns, Luc De, Raedt","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bertsimas and Dunn, 2017] Dimitris Bertsimas and Jack Dunn. Optimal classification trees. Machine Learning, 106(7):1039-1082, 2017.","title":"Optimal classification trees","authors":"Dimitris Bertsimas, Jack Dunn","year":"2017","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bessiere et al., 2009] Christian Bessiere, Emmanuel He- brard, and Barry O'Sullivan. Minimising decision tree size as combinatorial optimisation. In International Confer- ence on Principles and Practice of Constraint Program- ming, pages 173-187. Springer, 2009.","title":"Christian Bessiere, Emmanuel Hebrard, and Barry O'Sullivan. Minimising decision tree size as combinatorial optimisation","authors":"Bessiere","year":"2009","venue":"International Conference on Principles and Practice of Constraint Programming","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Breiman, 1984] Leo Breiman. Classification and regression trees. Routledge, 1984.","title":"Classification and regression trees","authors":"Leo Breiman, Breiman","year":"1984","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dechter and Mateescu, 2004] Rina Dechter and Robert Ma- teescu. The impact of AND/OR search spaces on con- straint satisfaction and counting. In Principles and Prac- tice of Constraint Programming -CP 2004, 10th Interna- tional Conference, CP 2004, Toronto, Canada, September 27 -October 1, 2004, Proceedings, pages 731-736, 2004.","title":"The impact of AND/OR search spaces on constraint satisfaction and counting","authors":"Rina Dechter, Robert Mateescu","year":"2004","venue":"Principles and Practice of Constraint Programming -CP 2004, 10th International Conference","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dechter and Mateescu, 2007] R. Dechter and R. Mateescu. And/or search spaces for graphical models. Artificial in- telligence, 171(2-3):73-106, 2007.","title":"And/or search spaces for graphical models","authors":"R Dechter, R Mateescu","year":"2007","venue":"Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Marinescu and Dechter, 2004] Radu Marinescu and Rina Dechter. And/or tree search for constraint optimization. In Proc. of the 6th International Workshop on Preferences and Soft Constraints. Citeseer, 2004.","title":"And/or tree search for constraint optimization","authors":"Radu Marinescu, Rina Dechter","year":"2004","venue":"Proc. of the 6th International Workshop on Preferences and Soft Constraints","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Narodytska et al., 2018] N. Narodytska, A. Ignatiev, F. Pereira, J. Marques-Silva, and IS RAS. Learning opti- mal decision trees with sat. In IJCAI, pages 1362-1368, 2018.","title":"Learning optimal decision trees with sat","authors":"N Narodytska, A Ignatiev, F Pereira, J Marques-Silva, I S Ras","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/189"},{"position":11,"raw":"[Nijssen and Fromont, 2007] S. Nijssen and E. Fromont. Mining optimal decision trees from itemset lattices. In Proceedings of the 13th ACM SIGKDD international con- ference on Knowledge discovery and data mining, pages 530-539. ACM, 2007.","title":"Mining optimal decision trees from itemset lattices","authors":"S Nijssen, E Fromont","year":"2007","venue":"Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Nijssen and Fromont, 2010] Siegfried Nijssen and Élisa Fromont. Optimal constraint-based decision tree induction from itemset lattices. Data Min. Knowl. Discov., 21(1):9- 51, 2010.","title":"Optimal constraint-based decision tree induction from itemset lattices","authors":"Fromont Nijssen, Siegfried Nijssen, Élisa Fromont","year":"2010","venue":"Data Min. Knowl. Discov","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Schaus et al., 2017] P. Schaus, J. Aoga, and T. Guns. Cover- size: a global constraint for frequency-based itemset min- ing. In International Conference on Principles and Prac- tice of Constraint Programming, pages 529-546. Springer, 2017.","title":"Coversize: a global constraint for frequency-based itemset mining","authors":"P Schaus, J Aoga, T Guns","year":"2017","venue":"International Conference on Principles and Practice of Constraint Programming","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Verhaeghe et al., 2019] Hélène Verhaeghe, Siegfried Ni- jssen, Gilles Pesant, Claude-Guy Quimper, and Pierre Schaus. Learning optimal decision trees using constraint programming. In The 25th International Conference on Principles and Practice of Constraint Programming (CP2019), 2019.","title":"Learning optimal decision trees using constraint programming","authors":"Verhaeghe","year":"2019","venue":"The 25th International Conference on Principles and Practice of Constraint Programming","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Verwer and Zhang, 2019] Sicco Verwer and Yingqian Zhang. Learning optimal classification trees using a binary linear program formulation. In 33rd AAAI Conference on Artificial Intelligence, 2019.","title":"Learning optimal classification trees using a binary linear program formulation","authors":"Zhang Verwer, Sicco Verwer, Yingqian Zhang","year":"2019","venue":"33rd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}