{"type":"ScholarlyArticle","title":"Learning Optimal Decision Trees with SAT","authors":[{"name":"Nina Narodytska","slug":"nina-narodytska","given":"Nina","family":"Narodytska","affiliation":"VMware Research , CA , USA"},{"name":"Alexey Ignatiev","slug":"alexey-ignatiev","given":"Alexey","family":"Ignatiev","affiliation":"LASIGE , Faculdade de Ciências , Universidade de Lisboa , Portugal; ISDCT SB RAS , Irkutsk , Russia"},{"name":"Filipe Pereira","slug":"filipe-pereira","given":"Filipe","family":"Pereira","affiliation":"LASIGE , Faculdade de Ciências , Universidade de Lisboa , Portugal"},{"name":"Joao Marques-Silva","slug":"joao-marques-silva","given":"Joao","family":"Marques-Silva","affiliation":"LASIGE , Faculdade de Ciências , Universidade de Lisboa , Portugal"}],"abstract":"Explanations of machine learning (ML) predictions are of fundamental importance in different settings. Moreover, explanations should be succinct, to enable easy understanding by humans.  Decision trees represent an often used approach for developing explainable ML models, motivated by the natural mapping between decision tree paths and rules. Clearly, smaller trees correlate well with smaller rules, and so one  challenge is to devise solutions for computing smallest size decision trees given training data. Although simple to formulate, the computation of smallest size decision trees turns out to be an extremely challenging computational problem, for which no practical solutions are known. This paper develops a SAT-based model for computing smallest-size decision trees given training data. In sharp contrast with past work, the proposed SAT model is shown to scale for publicly available datasets of practical interest.","doi":"10.24963/ijcai.2018/189","url":"https://ijcai.org/proceedings/2018/189","pdf":"https://ijcai.org/proceedings/2018/0189.pdf","markdown":"https://ijcai.org/proceedings/2018/189.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/189","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Constraints and SAT","track":"Main track","number":189,"pages":{"first":1362,"last":1368},"keywords":["Constraints and SAT: SAT","Constraints and SAT: Constraints and SAT","Constraints and SAT: SAT: Applications","Constraints and SAT: Constraints and Data Mining ; Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Angelino et al., 2017] Elaine Angelino, Nicholas Larus- Stone, Daniel Alabi, Margo Seltzer, and Cynthia Rudin. Learning certifiably optimal rule lists. In KDD, 2017.","title":"Learning certifiably optimal rule lists","authors":"Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, Cynthia Rudin","year":"2017","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Audemard and Simon, 2009] Gilles Audemard and Laurent Simon. Predicting learnt clauses quality in modern SAT solvers. In Craig Boutilier, editor, IJCAI, 2009.","title":"Predicting learnt clauses quality in modern SAT solvers","authors":"Simon ; Audemard, Gilles Audemard, Laurent Simon","year":"2009","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2009/74"},{"position":3,"raw":"[Bessiere et al., 2009] Christian Bessiere, Emmanuel He- brard, and Barry O'Sullivan. Minimising decision tree size as combinatorial optimisation. In CP, 2009.","title":"Christian Bessiere, Emmanuel Hebrard, and Barry O'Sullivan. Minimising decision tree size as combinatorial optimisation","authors":"Bessiere","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Biere et al., 2009] Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh, editors. Handbook of Satisfi- ability. IOS Press, 2009.","title":"Handbook of Satisfiability","authors":"Biere","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Biran and Cotton, 2017] Or Biran and Courtenay Cotton. Explanation and justification in machine learning: A sur- vey. In IJCAI-17 Workshop on Explainable AI, 2017.","title":"Explanation and justification in machine learning: A survey","authors":"Cotton Biran, Or Biran, Courtenay Cotton","year":"2017","venue":"IJCAI-17 Workshop on Explainable AI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Breiman et al., 1984] Leo Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984.","title":"Classification and Regression Trees","authors":"J H Leo Breiman, R A Friedman, C J Olshen, Stone","year":"1984","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cockett and Hierrera, 1990] J. Robin B. Cockett and J. A. Hierrera. Decision tree reduction. J. ACM, 37(4), 1990.","title":"Decision tree reduction","authors":"J, Robin B Cockett, J A Hierrera","year":"1990","venue":"J. ACM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[DARPA, 2016] DARPA. DARPA explainable Artificial Intelligence (XAI) program. https://www.darpa.mil/ program/explainable-artificial-intelligence, 2016.","title":"DARPA. DARPA explainable Artificial Intelligence (XAI) program","authors":null,"year":"2016","venue":null,"doi":null,"url":"https://www.darpa.mil/program/explainable-artificial-intelligence,2016","internal_url":null},{"position":9,"raw":"[Doshi-Velez and Kim, 2017] Finale Doshi-Velez and Been Kim. A roadmap for a rigorous science of interpretability. CoRR, abs/1702.08608, 2017.","title":"Finale Doshi-Velez and Been Kim. A roadmap for a rigorous science of interpretability","authors":"Kim Doshi-Velez","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dufour, 2014] David Dufour. Finding cost-efficient deci- sion trees. Master's thesis, University of Waterloo, 2014.","title":"Finding cost-efficient decision trees","authors":"David Dufour","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[EU Data Protection Regulation, 2016] EU Data Protection Regulation. Regulation (EU) 2016/679 of the European Parliament and of the Council, 2016.","title":"EU Data Protection Regulation","authors":null,"year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Goodman and Flaxman, 2017] Bryce Goodman and Seth R. Flaxman. European Union regulations on algorithmic decision-making and a \"right to explanation\". AI Maga- zine, 38(3):50-57, 2017.","title":"European Union regulations on algorithmic decision-making and a \"right to explanation","authors":"Bryce Goodman, Seth R Flaxman","year":"2017","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hall et al., 2009] Mark A. Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. The WEKA data mining software: an up- date. SIGKDD Explorations, 11(1):10-18, 2009.","title":"The WEKA data mining software: an update","authors":"Mark A Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, Ian H Witten","year":"2009","venue":"SIGKDD Explorations","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hancock et al., 1996] Thomas R. Hancock, Tao Jiang, Ming Li, and John Tromp. Lower bounds on learning decision lists and trees. Inf. Comput., 126(2):114-122, 1996.","title":"Lower bounds on learning decision lists and trees","authors":"R Thomas, Tao Hancock, Ming Jiang, John Li, Tromp","year":"1996","venue":"Inf. Comput","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hauser et al., 2010] John Hauser, Olivier Toubia, Theodoros Evgeniou, Rene Befurt, and Daria Dzyabura. Disjunctions of conjunctions, cognitive simplicity, and consideration sets. J. of Marketing Research, 47(3), 2010.","title":"Disjunctions of conjunctions, cognitive simplicity, and consideration sets","authors":"John Hauser, Olivier Toubia, Theodoros Evgeniou, Rene Befurt, Daria Dzyabura","year":"2010","venue":"J. of Marketing Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hyafil and Rivest, 1976] Laurent Hyafil and Ronald L. Rivest. Constructing optimal binary decision trees is NP- complete. Inf. Process. Lett., 5(1):15-17, 1976.","title":"Constructing optimal binary decision trees is NPcomplete","authors":"Laurent Hyafil, Ronald L Rivest","year":"1976","venue":"Inf. Process. Lett","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[ICML WHI Workshop, 2017] ICML WHI Workshop. ICML workshop on human interpretability in ML, 2017.","title":"ICML WHI Workshop. ICML workshop on human interpretability in ML","authors":null,"year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[IJCAI XAI Workshop, 2017] IJCAI XAI Workshop. IJCAI workshop on explainable artificial intelligence, 2017. [Lakkaraju et al., 2016] Himabindu Lakkaraju, Stephen Bach, and Jure Leskovec. Interpretable decision sets: A joint framework for description and prediction. In KDD, 2016.","title":"Interpretable decision sets: A joint framework for description and prediction","authors":"Lakkaraju","year":"2016","venue":"IJCAI XAI Workshop. IJCAI workshop on explainable artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lakkaraju et al., 2017] Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. Interpretable & ex- plorable approximations of black box models. CoRR, abs/1707.01154, 2017.","title":"Interpretable & explorable approximations of black box models","authors":"Lakkaraju","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Letham et al., 2015] Benjamin Letham, Cynthia Rudin, Tyler H McCormick, David Madigan, et al. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics, 9(3):1350-1371, 2015.","title":"Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model","authors":"Benjamin Letham, Cynthia Rudin, H Tyler, David Mccormick, Madigan","year":"2015","venue":"The Annals of Applied Statistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2018] Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. Deep learning for case-based reasoning through prototypes: A neural network that explains its pre- dictions. In AAAI, 2018.","title":"Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions","authors":"Oscar Li, Hao Liu, Chaofan Chen, Cynthia Rudin","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lou et al., 2012] Yin Lou, Rich Caruana, and Johannes Gehrke. Intelligible models for classification and regres- sion. In KDD, pages 150-158, 2012.","title":"Intelligible models for classification and regression","authors":"Yin Lou, Rich Caruana, Johannes Gehrke","year":"2012","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mitchell, 1997] Tom M. Mitchell. Machine learning. McGraw-Hill, 1997.","title":"Machine learning","authors":"Tom M Mitchell","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Murthy, 1998] Sreerama K. Murthy. Automatic construc- tion of decision trees from data: A multi-disciplinary sur- vey. Data Min. Knowl. Discov., 2(4):345-389, 1998.","title":"Automatic construction of decision trees from data: A multi-disciplinary survey","authors":"K Sreerama, Murthy","year":"1998","venue":"Data Min. Knowl. Discov","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[NIPS IML Symposium, 2017] NIPS IML Symposium. NIPS interpretable ML symposium, 2017.","title":"NIPS IML Symposium. NIPS interpretable ML symposium","authors":null,"year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Olson et al., 2017] Randal Olson, William La Cava, Pa- tryk Orzechowski, Ryan Urbanowicz, and Jason Moore. PMLB: a large benchmark suite for machine learning eval- uation and comparison. BioData Mining, 10(1):36, 2017.","title":"PMLB: a large benchmark suite for machine learning evaluation and comparison","authors":"Randal Olson, William La Cava, Patryk Orzechowski, Ryan Urbanowicz, Jason Moore","year":"2017","venue":"BioData Mining","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Pedregosa and et al., 2011] Fabian Pedregosa and et al. Scikit-learn: Machine learning in Python. Journal of Ma- chine Learning Research, 12:2825-2830, 2011.","title":"Scikit-learn: Machine learning in Python","authors":"Pedregosa","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Plaisted and Greenbaum, 1986] David A. Plaisted and Steven Greenbaum. A structure-preserving clause form translation. J. Symb. Comput., 2(3):293-304, 1986.","title":"A structure-preserving clause form translation","authors":"David A Plaisted, Steven Greenbaum","year":"1986","venue":"J. Symb. Comput","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Quinlan, 1986] J. Ross Quinlan. Induction of decision trees. Machine Learning, 1(1):81-106, 1986.","title":"Induction of decision trees","authors":"J, Ross Quinlan","year":"1986","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Quinlan, 1993] J. Ross Quinlan. C4.5: Programs for ma- chine learning. Morgan Kauffmann, 1993.","title":"C4.5: Programs for machine learning","authors":"J, Ross Quinlan","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Rokach and Maimon, 2015] Lior Rokach and Oded Mai- mon. Data Mining with Decision Trees. WorldScientific, 2015.","title":"Data Mining with Decision Trees","authors":"Maimon Rokach","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sinz, 2005] Carsten Sinz. Towards an optimal CNF encod- ing of boolean cardinality constraints. In CP, 2005.","title":"Towards an optimal CNF encoding of boolean cardinality constraints","authors":"Carsten Sinz, Sinz","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Utgoff et al., 1997a] Paul E. Utgoff, Neil C. Berkman, and Jeffery A. Clouse. Decision tree induction based on effi- cient tree restructuring. Machine Learning, 29(1), 1997.","title":"Decision tree induction based on efficient tree restructuring","authors":"Utgoff","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Utgoff et al., 1997b] Paul E. Utgoff, Neil C. Berkman, and Jeffery A. Clouse. Incremental decision tree induction. http://www-lrn.cs.umass.edu/iti/index.html, 1997.","title":"Incremental decision tree induction","authors":"Utgoff","year":"1997","venue":null,"doi":null,"url":"http://www-lrn.cs.umass.edu/iti/index.html","internal_url":null},{"position":35,"raw":"[Wang et al., 2015] Tong Wang, Cynthia Rudin, Finale Doshi-Velez, Yimin Liu, Erica Klampfl, and Perry Mac- Neille. Or's of And's for interpretable classification, with application to context-aware recommender systems. CoRR, abs/1504.07614, 2015.","title":"Or's of And's for interpretable classification, with application to context-aware recommender systems","authors":"Tong Wang, Cynthia Rudin, Finale Doshi-Velez, Yimin Liu, Erica Klampfl, Perry Mac-Neille","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Weller, 2017] Adrian Weller. Challenges for transparency. CoRR, abs/1708.01870, 2017.","title":"Challenges for transparency","authors":"Adrian Weller","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}