{"type":"ScholarlyArticle","title":"The Sparse MinMax k-Means Algorithm for High-Dimensional Clustering","authors":[{"name":"Sayak Dey","slug":"sayak-dey","given":"Sayak","family":"Dey","affiliation":"Samsung Research Institute , Bangalore , 560037 , India"},{"name":"Swagatam Das","slug":"swagatam-das","given":"Swagatam","family":"Das","affiliation":"Indian Statistical Institute , Kolkata , 700108 , India"},{"name":"Rammohan Mallipeddi","slug":"rammohan-mallipeddi","given":"Rammohan","family":"Mallipeddi","affiliation":"Kyungpook National University , Daegu , 41566 , Republic of Korea"}],"abstract":"Classical clustering methods usually face tough challenges when we have a larger set of features compared to the number of items to be partitioned. We propose a Sparse MinMax k-Means Clustering approach by reformulating the objective of the MinMax k-Means algorithm (a variation of classical k-Means that minimizes the maximum intra-cluster variance instead of the sum of intra-cluster variances), into a new weighted between-cluster sum of squares (BCSS) form. We impose sparse regularization on these weights to make it suitable for high-dimensional clustering. We seek to use the advantages of the MinMax k-Means algorithm in the high-dimensional space to generate good quality clusters. The efficacy of the proposal is showcased through comparison against a few representative clustering methods over several real world datasets.","doi":"10.24963/ijcai.2020/291","url":"https://ijcai.org/proceedings/2020/291","pdf":"https://ijcai.org/proceedings/2020/0291.pdf","markdown":"https://ijcai.org/proceedings/2020/291.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/291","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":291,"pages":{"first":2103,"last":2110},"keywords":["Machine Learning: Clustering","Data Mining: Clustering, Unsupervised Learning","Machine Learning: Feature Selection; Learning Sparse Models"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24966/file_24966.m3u8","length_sec":300,"provider":"hls"}],"references":[{"position":1,"raw":"[Arthur and Vassilvitskii, 2007] David Arthur and Sergei Vas- silvitskii. K-means++: The advantages of careful seed- ing. In Proceedings of the Eighteenth Annual ACM- SIAM Symposium on Discrete Algorithms, SODA '07, page 1027-1035, USA, 2007. Society for Industrial and Applied Mathematics.","title":"K-means++: The advantages of careful seeding","authors":"David Arthur, Sergei Vassilvitskii","year":"2007","venue":"Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '07","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bache and Lichman, 2013] K. Bache and M. Lichman. Uci machine learning repository, 2013.","title":"Uci machine learning repository","authors":"K Bache, M Lichman","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Belhumeur et al., 1997] Peter N. Belhumeur, João P. Hes- panha, and David J. Kriegman. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711-720, 1997.","title":"Eigenfaces vs. fisherfaces: recognition using class specific linear projection","authors":"N Peter, João P Belhumeur, David J Hespanha, Kriegman","year":"1997","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2007] Deng Cai, Xiaofei He, Yuxiao Hu, Jiawei Han, and Thomas Huang. Learning a spatially smooth subspace for face recognition. In Proc. IEEE Conf. Com- puter Vision and Pattern Recognition Machine Learning (CVPR'07), 2007.","title":"Learning a spatially smooth subspace for face recognition","authors":"Deng Cai, Xiaofei He, Yuxiao Hu, Jiawei Han, Thomas Huang","year":"2007","venue":"Proc. IEEE Conf. Computer Vision and Pattern Recognition Machine Learning (CVPR'07)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Celebi et al., 2013] M. Emre Celebi, Hassan A. Kingravi, and Patricio A. Vela. A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Systems with Applications, 40(1):200-210, 2013.","title":"A comparative study of efficient initialization methods for the k-means clustering algorithm","authors":"M Emre Celebi, Hassan A Kingravi, Patricio A Vela","year":"2013","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chakraborty and Das, 2017] Saptarshi Chakraborty and Swagatam Das. k-means clustering with a new divergence- based distance metric: Convergence and performance analysis. Pattern Recognition Letters, 100:67-73, 2017.","title":"Saptarshi Chakraborty and Swagatam Das. k-means clustering with a new divergencebased distance metric: Convergence and performance analysis","authors":"Das Chakraborty","year":"2017","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chang et al., 2017] Xiangyu Chang, Qingnan Wang, Yuewen Liu, and Yu Wang. Sparse regularization in fuzzy c -means for high-dimensional data clustering. IEEE Transactions on Cybernetics, 47(9):2616-2627, 2017.","title":"Sparse regularization in fuzzy c -means for high-dimensional data clustering","authors":"Xiangyu Chang, Qingnan Wang, Yuewen Liu, Yu Wang","year":"2017","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dunn, 1974] J. C. Dunn. Well-separated clusters and opti- mal fuzzy partitions. Journal of Cybernetics, 4(1):95-104, 1974.","title":"Well-separated clusters and optimal fuzzy partitions","authors":"J C Dunn","year":"1974","venue":"Journal of Cybernetics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jin and Wang, 2014] Jiashun Jin and Wanjie Wang. Gene microarray data sets, 2014.","title":"Gene microarray data sets","authors":"Jin, Wang ;, Jiashun Jin, Wanjie Wang","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jin and Wang, 2016] Jiashun Jin and Wanjie Wang. Influ- ential features pca for high dimensional clustering. The Annals of Statistics, 44(6):2323-2359, 2016.","title":"Influential features pca for high dimensional clustering","authors":"Jin, Wang ;, Jiashun Jin, Wanjie Wang","year":"2016","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kriegel et al., 2008] Hans-Peter Kriegel, Peer Kröger, and Arthur Zimek. Detecting clusters in moderate-to-high di- mensional data: subspace clustering, pattern-based cluster- ing, and correlation clustering. Proceedings of the VLDB Endowment, 1(2):1528-1529, 2008.","title":"Detecting clusters in moderate-to-high dimensional data: subspace clustering, pattern-based clustering, and correlation clustering","authors":"Kriegel","year":"2008","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kriegel et al., 2009] Hans-Peter Kriegel, Peer Kröger, and Arthur Zimek. Clustering high-dimensional data: A sur- vey on subspace clustering, pattern-based clustering, and correlation clustering. ACM Transactions on Knowledge Discovery from Data (TKDD), 3(1):1, 2009.","title":"Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering, and correlation clustering","authors":"Kriegel","year":"2009","venue":"ACM Transactions on Knowledge Discovery from Data (TKDD)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lee et al., 2010] Ann B. Lee, Diana Luca, and Kathryn Roeder. A spectral graph approach to discovering genetic ancestry. Ann. Appl. Stat., 4(1):179-202, 03 2010.","title":"A spectral graph approach to discovering genetic ancestry","authors":"Ann B Lee, Diana Luca, Kathryn Roeder","year":"2010","venue":"Ann. Appl. Stat","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2018] Rongjian Li, Xiangyu Chang, Yu Wang, and Zongben Xu. Sparse k-means with ℓ ∞ /ℓ 0 penalty for high- dimensional data clustering. Statistica Sinica, 2018.","title":"Sparse k-means with ℓ ∞ /ℓ 0 penalty for highdimensional data clustering","authors":"Li","year":"2018","venue":"Statistica Sinica","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lloyd, 1982] S. Lloyd. Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129-137, 1982.","title":"Least squares quantization in pcm","authors":"S Lloyd","year":"1982","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pandove et al., 2018] Divya Pandove, Shivan Goel, and Rinkl Rani. Systematic review of clustering high- dimensional and large datasets. ACM Transactions on Knowledge Discovery from Data (TKDD), 12(2):16, 2018.","title":"Systematic review of clustering highdimensional and large datasets","authors":"Divya Pandove, Shivan Goel, Rinkl Rani","year":"2018","venue":"ACM Transactions on Knowledge Discovery from Data (TKDD)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Peña et al., 1999] J.m Peña, J.a Lozano, and P Larrañaga. An empirical comparison of four initialization methods for the k-means algorithm. Pattern Recognition Letters, 20(10):1027-1040, 1999.","title":"An empirical comparison of four initialization methods for the k-means algorithm","authors":"J Peña, J Lozano, Larrañaga","year":"1999","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pontes et al., 2015] Beatriz Pontes, Raúl Giráldez, and Jesús S. Aguilar-Ruiz. Biclustering on expression data: A review. Journal of Biomedical Informatics, 57:163 -180, 2015.","title":"Biclustering on expression data: A review","authors":"Beatriz Pontes, Raúl Giráldez, Jesús S Aguilar-Ruiz","year":"2015","venue":"Journal of Biomedical Informatics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sarkar and Ghosh, 2019] Soham Sarkar and Anil K. Ghosh. On perfect clustering of high dimension, low sample size data. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1-1, 2019.","title":"On perfect clustering of high dimension, low sample size data","authors":"Ghosh Sarkar, Soham Sarkar, Anil K Ghosh","year":"2019","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tibshirani et al., 2001] Robert Tibshirani, Guenther Walther, and Trevor Hastie. Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(2):411-423, 2001.","title":"Estimating the number of clusters in a data set via the gap statistic","authors":"Robert Tibshirani, Walther Guenther, Trevor Hastie","year":"2001","venue":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tzortzis and Likas, 2014] Grigorios Tzortzis and Aristidis Likas. The minmax k-means clustering algorithm. Pattern Recognition, 47(7):2505-2516, 2014.","title":"The minmax k-means clustering algorithm","authors":"Likas Tzortzis, Grigorios Tzortzis, Aristidis Likas","year":"2014","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[van der Maaten and Hinton, 2008] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Jour- nal of Machine Learning Research, 9:2579-2605, 2008.","title":"Visualizing data using t-SNE","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2019] Xiao-Dong Wang, Rung-Ching Chen, Fei Yan, Zhi-Qiang Zeng, and Chao-Qun Hong. Fast adaptive k- means subspace clustering for high-dimensional data. IEEE Access, 7:42639-42651, March 2019.","title":"Fast adaptive kmeans subspace clustering for high-dimensional data","authors":"Xiao-Dong Wang, Rung-Ching Chen, Fei Yan, Zhi-Qiang Zeng, Chao-Qun Hong","year":"2019","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Witten and Tibshirani, 2010] Daniela M. Witten and Robert Tibshirani. A framework for feature selection in clus- tering. Journal of the American Statistical Association, 105(490):713-726, 2010.","title":"A framework for feature selection in clustering","authors":"Daniela M Witten, Robert Tibshirani","year":"2010","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}