{"type":"ScholarlyArticle","title":"Optimal Decision Trees For Interpretable Clustering with Constraints","authors":[{"name":"Pouya Shati","slug":"pouya-shati","given":"Pouya","family":"Shati","affiliation":"Department of Computer Science , University of Toronto , Toronto , Canada; Vector Institute , Toronto , Canada"},{"name":"Eldan Cohen","slug":"eldan-cohen","given":"Eldan","family":"Cohen","affiliation":"Department of Mechanical and Industrial Engineering , University of Toronto , Toronto , Canada"},{"name":"Sheila McIlraith","slug":"sheila-mcilraith","given":"Sheila","family":"McIlraith","affiliation":"Department of Computer Science , University of Toronto , Toronto , Canada; Vector Institute , Toronto , Canada"}],"abstract":"Constrained clustering is a semi-supervised task that employs a limited amount of labelled data, formulated as constraints, to incorporate domain-specific knowledge and to significantly improve clustering accuracy. Previous work has considered exact optimization formulations that can guarantee optimal clustering while satisfying all constraints, however these approaches lack interpretability. Recently, decision trees have been used to produce inherently interpretable clustering solutions, however existing approaches do not support clustering constraints and do not provide strong theoretical guarantees on solution quality. In this work, we present a novel SAT-based framework for interpretable clustering that supports clustering constraints and that also provides strong theoretical guarantees on solution quality. We also present new insight into the trade-off between interpretability and satisfaction of such user-provided constraints. Our framework is the first approach for interpretable and constrained clustering. Experiments with a range of real-world and synthetic datasets demonstrate that our approach can produce high-quality and interpretable constrained clustering solutions.","doi":"10.24963/ijcai.2023/225","url":"https://ijcai.org/proceedings/2023/225","pdf":"https://ijcai.org/proceedings/2023/0225.pdf","markdown":"https://ijcai.org/proceedings/2023/225.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/225","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Constraint Satisfaction and Optimization","track":"Main Track","number":225,"pages":{"first":2022,"last":2030},"keywords":["Constraint Satisfaction and Optimization: CSO: Constraint optimization","Machine Learning: ML: Clustering","Constraint Satisfaction and Optimization: CSO: Satisfiabilty"],"videos":[],"references":[{"position":1,"raw":"[Babaki et al., 2014] Behrouz Babaki, Tias Guns, and Siegfried Nijssen. Constrained clustering using column generation. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pages 438-454. Springer, 2014.","title":"Constrained clustering using column generation","authors":"Behrouz Babaki, Tias Guns, Siegfried Nijssen","year":"2014","venue":"International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Berg and Järvisalo, 2017] Jeremias Berg and Matti Järvisalo. Cost-optimal constrained correlation clus- tering via weighted partial maximum satisfiability. Artificial Intelligence, 244:110-142, 2017.","title":"Cost-optimal constrained correlation clustering via weighted partial maximum satisfiability","authors":"Järvisalo Berg, Jeremias Berg, Matti Järvisalo","year":"2017","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berg et al., 2019] Jeremias Berg, Emir Demirović, and Pe- ter J Stuckey. Core-boosted linear search for incomplete MaxSAT. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research (CPAIOR), pages 39-56. Springer, 2019.","title":"Core-boosted linear search for incomplete MaxSAT","authors":"Jeremias Berg, Emir Demirović, Peter J Stuckey","year":"2019","venue":"International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research (CPAIOR)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bertsimas et al., 2021] Dimitris Bertsimas, Agni Or- fanoudaki, and Holly Wiberg. Interpretable clustering: an optimization approach. Machine Learning, 110(1):89- 138, 2021.","title":"Interpretable clustering: an optimization approach","authors":"Dimitris Bertsimas, Agni Orfanoudaki, Holly Wiberg","year":"2021","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Biere et al., 2009] Armin Biere, Marijn Heule, and Hans van Maaren. Handbook of satisfiability, volume 185. IOS press, 2009.","title":"Handbook of satisfiability","authors":"Biere","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bilenko et al., 2004] Mikhail Bilenko, Sugato Basu, and Raymond J Mooney. Integrating constraints and metric learning in semi-supervised clustering. In Proceedings of the twenty-first international conference on Machine learn- ing, page 11, 2004.","title":"Integrating constraints and metric learning in semi-supervised clustering","authors":"Mikhail Bilenko, Sugato Basu, Raymond J Mooney","year":"2004","venue":"Proceedings of the twenty-first international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cohen et al., 2020] Eldan Cohen, Arik Senderovich, and J Christopher Beck. An ising framework for constrained clustering on special purpose hardware. In International Conference on Integration of Constraint Programming, Ar- tificial Intelligence, and Operations Research, pages 130- 147. Springer, 2020.","title":"An ising framework for constrained clustering on special purpose hardware","authors":"Eldan Cohen, Arik Senderovich, J Christopher, Beck","year":"2020","venue":"International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dao et al., 2016] Thi-Bich-Hanh Dao, Christel Vrain, Khanh-Chuong Duong, and Ian Davidson. A framework for actionable clustering using constraint programming. In Proceedings of the Twenty-second European Conference on Artificial Intelligence, pages 453-461, 2016.","title":"A framework for actionable clustering using constraint programming","authors":"Thi-Bich-Hanh Dao, Christel Vrain, Khanh-Chuong Duong, Ian Davidson","year":"2016","venue":"Proceedings of the Twenty-second European Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dao et al., 2017] Thi-Bich-Hanh Dao, Khanh-Chuong Duong, and Christel Vrain. Constrained clustering by con- straint programming. Artificial Intelligence, 244:70-94, 2017.","title":"Constrained clustering by constraint programming","authors":"Thi-Bich-Hanh Dao, Khanh-Chuong Duong, Christel Vrain","year":"2017","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Davidson and Ravi, 2005] Ian Davidson and SS Ravi. Clus- tering with constraints: Feasibility issues and the k-means algorithm. In Proceedings of the 2005 SIAM international conference on data mining, pages 138-149. SIAM, 2005.","title":"Clustering with constraints: Feasibility issues and the k-means algorithm","authors":"Davidson, Ravi ;, Ian Davidson, S S Ravi","year":"2005","venue":"Proceedings of the 2005 SIAM international conference on data mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Davidson et al., 2010] Ian Davidson, SS Ravi, and Leonid Shamis. A sat-based framework for efficient constrained clustering. In Proceedings of the 2010 SIAM international conference on data mining, pages 94-105. SIAM, 2010.","title":"A sat-based framework for efficient constrained clustering","authors":"Ian Davidson, Leonid Ravi, Shamis","year":"2010","venue":"Proceedings of the 2010 SIAM international conference on data mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dua and Graff, 2017] Dheeru Dua and Casey Graff. UCI machine learning repository. http://archive.ics.uci.edu/ml, 2017. Accessed: 2022-12-01.","title":"Dheeru Dua and Casey Graff. UCI machine learning repository","authors":"Graff Dua","year":"2017","venue":null,"doi":null,"url":"http://archive.ics.uci.edu/ml,2017","internal_url":null},{"position":13,"raw":"[Frost et al., 2020] Nave Frost, Michal Moshkovitz, and Cyrus Rashtchian. Exkmc: Expanding explainable 𝑘- means clustering. arXiv preprint arXiv:2006.02399, 2020.","title":"Exkmc: Expanding explainable 𝑘means clustering","authors":"Nave Frost, Michal Moshkovitz, Cyrus Rashtchian","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gamlath et al., 2021] Buddhima Gamlath, Xinrui Jia, Adam Polak, and Ola Svensson. Nearly-tight and oblivious al- gorithms for explainable clustering. Advances in Neural Information Processing Systems, 34:28929-28939, 2021.","title":"Nearly-tight and oblivious algorithms for explainable clustering","authors":"Gamlath","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hubert and Arabie, 1985] Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of Classification, 2(1):193-218, 1985.","title":"Lawrence Hubert and Phipps Arabie","authors":"Arabie Hubert","year":"1985","venue":"Journal of Classification","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ignatiev et al., 2021] Alexey Ignatiev, Joao Marques-Silva, Nina Narodytska, and Peter J Stuckey. Reasoning-based learning of interpretable ml models. In IJCAI, pages 4458- 4465, 2021.","title":"Reasoning-based learning of interpretable ml models","authors":"Alexey Ignatiev, Joao Marques-Silva, Nina Narodytska, Peter J Stuckey","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/608"},{"position":17,"raw":"[Jabs et al., 2022] Christoph Jabs, Jeremias Berg, Andreas Niskanen, and Matti Järvisalo. Maxsat-based bi-objective boolean optimization. In 25th International Conference on Theory and Applications of Satisfiability Testing (SAT 2022). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2022.","title":"Maxsat-based bi-objective boolean optimization","authors":"Christoph Jabs, Jeremias Berg, Andreas Niskanen, Matti Järvisalo","year":"2022","venue":"25th International Conference on Theory and Applications of Satisfiability Testing (SAT 2022)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu and Fu, 2015] Hongfu Liu and Yun Fu. Clustering with partition level side information. In 2015 IEEE interna- tional conference on data mining, pages 877-882. IEEE, 2015.","title":"Clustering with partition level side information","authors":"Hongfu Liu, Yun Fu","year":"2015","venue":"IEEE international conference on data mining","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2017] Hongfu Liu, Zhiqiang Tao, and Yun Fu. Partition level constrained clustering. IEEE trans- actions on pattern analysis and machine intelligence, 40(10):2469-2483, 2017.","title":"Partition level constrained clustering","authors":"Hongfu Liu, Zhiqiang Tao, Yun Fu","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Moshkovitz et al., 2020] Michal Moshkovitz, Sanjoy Das- gupta, Cyrus Rashtchian, and Nave Frost. Explainable k- means and k-medians clustering. In International Con- ference on Machine Learning, pages 7055-7065. PMLR, 2020.","title":"Explainable kmeans and k-medians clustering","authors":"Michal Moshkovitz, Sanjoy Dasgupta, Cyrus Rashtchian, Nave Frost","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pelleg and Baras, 2007] Dan Pelleg and Dorit Baras. K- means with large and noisy constraint sets. In Euro- pean Conference on Machine Learning, pages 674-682. Springer, 2007.","title":"Kmeans with large and noisy constraint sets","authors":"Baras Pelleg, Dan Pelleg, Dorit Baras","year":"2007","venue":"European Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shati et al., 2021] Pouya Shati, Eldan Cohen, and Sheila McIlraith. Sat-based approach for learning optimal deci- sion trees with non-binary features. In 27th International Conference on Principles and Practice of Constraint Pro- gramming (CP 2021). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2021.","title":"Sat-based approach for learning optimal decision trees with non-binary features","authors":"Pouya Shati, Eldan Cohen, Sheila Mcilraith","year":"2021","venue":"27th International Conference on Principles and Practice of Constraint Programming (CP 2021)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shati et al., 2023] Pouya Shati, Eldan Cohen, and Sheila McIlraith. Optimal decision trees for interpretable clus- tering with constraints (extended version). arXiv preprint arXiv:2301.12671, 2023.","title":"Optimal decision trees for interpretable clustering with constraints","authors":"Pouya Shati, Eldan Cohen, Sheila Mcilraith","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Strehl and Ghosh, 2002] Alexander Strehl and Joydeep Ghosh. Cluster ensembles-a knowledge reuse frame- work for combining multiple partitions. Journal of Machine Learning Research, 3:583-617, 2002.","title":"Cluster ensembles-a knowledge reuse framework for combining multiple partitions","authors":"Ghosh Strehl, Alexander Strehl, Joydeep Ghosh","year":"2002","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Thalamuthu et al., 2006] Anbupalam Thalamuthu, Indranil Mukhopadhyay, Xiaojing Zheng, and George C Tseng. Evaluation and comparison of gene clustering methods in microarray analysis. Bioinformatics, 22(19):2405-2412, 2006.","title":"Evaluation and comparison of gene clustering methods in microarray analysis","authors":"Thalamuthu","year":"2006","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ultsch and Lötsch, 2020] Alfred Ultsch and Jörn Lötsch. The fundamental clustering and projection suite (fcps): A dataset collection to test the performance of clustering and data projection algorithms. Data, 5(1):13, 2020.","title":"The fundamental clustering and projection suite (fcps): A dataset collection to test the performance of clustering and data projection algorithms","authors":"Lötsch Ultsch, Alfred Ultsch, Jörn Lötsch","year":"2020","venue":"Data","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wagstaff and Cardie, 2000] Kiri Wagstaff and Claire Cardie. Clustering with instance-level constraints. In Proceedings of the Seventeenth International Conference on Machine Learning, pages 1103-1110. Citeseer, 2000.","title":"Clustering with instance-level constraints","authors":"Cardie Wagstaff, Kiri Wagstaff, Claire Cardie","year":"2000","venue":"Proceedings of the Seventeenth International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wagstaff et al., 2001] Kiri Wagstaff, Claire Cardie, Seth Rogers, and Stefan Schroedl. Constrained k-means clus- tering with background knowledge. In Proceedings of the Eighteenth International Conference on Machine Learn- ing, pages 577-584, 2001.","title":"Constrained k-means clustering with background knowledge","authors":"Kiri Wagstaff, Claire Cardie, Seth Rogers, Stefan Schroedl","year":"2001","venue":"Proceedings of the Eighteenth International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wu and Chou, 2011] Roung-Shiunn Wu and Po-Hsuan Chou. Customer segmentation of multiple category data in e-commerce using a soft-clustering approach. Electronic Commerce Research and Applications, 10(3):331-341, 2011.","title":"Roung-Shiunn Wu and Po-Hsuan Chou. Customer segmentation of multiple category data in e-commerce using a soft-clustering approach","authors":"Chou Wu","year":"2011","venue":"Electronic Commerce Research and Applications","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}