{"type":"ScholarlyArticle","title":"Locally Normalized Soft Contrastive Clustering for Compact Clusters","authors":[{"name":"Xin Ma","slug":"xin-ma","given":"Xin","family":"Ma","affiliation":"Computer Science and Engineering , University of Texas at Arlington , USA"},{"name":"Won Hwa Kim","slug":"won-hwa-kim","given":"Won Hwa","family":"Kim","affiliation":"Computer Science and Engineering , University of Texas at Arlington , USA; Computer Science and Engineering , POSTECH , South Korea; Graduate School of AI , POSTECH , South Korea"}],"abstract":"Recent deep clustering algorithms take advantage of self-supervised learning and self-training techniques to map the original data into a latent space, where the data embedding and clustering assignment can be jointly optimized. However, as many recent datasets are enormous and noisy, getting a clear boundary between different clusters is challenging with existing methods that mainly focus on contracting similar samples together and overlooking samples near boundary of clusters in the latent space. In this regard, we propose an end-to-end deep clustering algorithm, i.e., Locally Normalized Soft Contrastive Clustering (LNSCC). It takes advantage of similarities among each sample's local neighborhood and globally disconnected samples to leverage positiveness and negativeness of sample pairs in a contrastive way to separate different clusters. Experimental results on various datasets illustrate that our proposed approach achieves outstanding clustering performance over most of the state-of-the-art clustering methods for both image and non-image data even without convolution.","doi":"10.24963/ijcai.2022/460","url":"https://ijcai.org/proceedings/2022/460","pdf":"https://ijcai.org/proceedings/2022/0460.pdf","markdown":"https://ijcai.org/proceedings/2022/460.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/460","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":460,"pages":{"first":3313,"last":3320},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Clustering","Computer Vision: Representation Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986748","length_sec":82,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984891","length_sec":301,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bolin et al., 2014] Jocelyn H Bolin, Julianne M Edwards, W Holmes Finch, et al. Applications of cluster analysis to the creation of perfectionism profiles: a comparison of two clustering approaches. Frontiers in psychology, 5:343, 2014.","title":"Applications of cluster analysis to the creation of perfectionism profiles: a comparison of two clustering approaches","authors":"Jocelyn H Bolin, Julianne M Edwards, W Holmes Finch","year":"2014","venue":"Frontiers in psychology","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caron et al., 2018] Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In ECCV, pages 132-149, 2018.","title":"Deep clustering for unsupervised learning of visual features","authors":"Mathilde Caron, Piotr Bojanowski, Armand Joulin, Matthijs Douze","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Caron et al., 2020] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsuper- vised learning of visual features by contrasting cluster assign- ments. NeurIPS, 33:9912-9924, 2020.","title":"Unsupervised learning of visual features by contrasting cluster assignments","authors":"Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, Armand Joulin","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chang et al., 2017] Jianlong Chang, Lingfeng Wang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. Deep adaptive im- age clustering. In ICCV, pages 5879-5887, 2017.","title":"Deep adaptive image clustering","authors":"Jianlong Chang, Lingfeng Wang, Gaofeng Meng, Shiming Xiang, Chunhong Pan","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen and Cai, 2011] Xinlei Chen and Deng Cai. Large scale spec- tral clustering with landmark-based representation. In AAAI. Citeseer, 2011.","title":"Large scale spectral clustering with landmark-based representation","authors":"Chen, Cai, Xinlei Chen, Deng Cai","year":"2011","venue":"AAAI. Citeseer","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for con- trastive learning of visual representations. In ICML, pages 1597- 1607. PMLR, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Coates et al., 2011] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In AISTATS, pages 215-223, 2011.","title":"An analysis of single-layer networks in unsupervised feature learning","authors":"Adam Coates, Andrew Ng, Honglak Lee","year":"2011","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Costa and Ortale, 2020] Gianni Costa and Riccardo Ortale. Docu- ment clustering meets topic modeling with word embeddings. In ICDM, pages 244-252. SIAM, 2020.","title":"Document clustering meets topic modeling with word embeddings","authors":"Gianni Costa, Riccardo Ortale","year":"2020","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cubuk et al., 2020] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data aug- mentation with a reduced search space. In CVPR Workshop, pages 702-703, 2020.","title":"Randaugment: Practical automated data augmentation with a reduced search space","authors":"Barret Ekin D Cubuk, Jonathon Zoph, Quoc V Shlens, Le","year":"2020","venue":"CVPR Workshop","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ester et al., 1996] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD, volume 96 (34), pages 226-231, 1996.","title":"A density-based algorithm for discovering clusters in large spatial databases with noise","authors":"Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu","year":"1996","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fard et al., 2020] Mazar Moradi Fard, Thibaut Thonet, and Eric Gaussier. Seed-guided deep document clustering. In ECIR, pages 3-16. Springer, 2020.","title":"Seed-guided deep document clustering","authors":"Mazar Moradi Fard, Thibaut Thonet, Eric Gaussier","year":"2020","venue":"ECIR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ghasedi Dizaji et al., 2017] Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, and Heng Huang. Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In ICCV, pages 5736-5745, 2017.","title":"Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization","authors":"Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, Heng Huang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guo et al., 2017] Xifeng Guo, Long Gao, Xinwang Liu, and Jian- ping Yin. Improved deep embedded clustering with local struc- ture preservation. In IJCAI, pages 1753-1759, 2017.","title":"Improved deep embedded clustering with local structure preservation","authors":"Xifeng Guo, Long Gao, Xinwang Liu, Jianping Yin","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/243"},{"position":14,"raw":"[Guo et al., 2019] Xifeng Guo, Xinwang Liu, En Zhu, Xinzhong Zhu, Miaomiao Li, Xin Xu, and Jianping Yin. Adaptive self-paced deep clustering with data augmentation. TKDE, 32(9):1680-1693, 2019.","title":"Adaptive self-paced deep clustering with data augmentation","authors":"Xifeng Guo, Xinwang Liu, En Zhu, Xinzhong Zhu, Miaomiao Li, Xin Xu, Jianping Yin","year":"2019","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Huang et al., 2020] Jiabo Huang, Shaogang Gong, and Xiatian Zhu. Deep semantic clustering by partition confidence maximi- sation. In CVPR, pages 8849-8858, 2020.","title":"Deep semantic clustering by partition confidence maximisation","authors":"Huang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ji et al., 2019] Xu Ji, João F Henriques, and Andrea Vedaldi. In- variant information clustering for unsupervised image classifica- tion and segmentation. In ICCV, pages 9865-9874, 2019.","title":"Invariant information clustering for unsupervised image classification and segmentation","authors":"Ji Xu, João F Henriques, Andrea Vedaldi","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kanezaki, 2018] Asako Kanezaki. Unsupervised image segmenta- tion by backpropagation. In ICASSP, pages 1543-1547. IEEE, 2018.","title":"Unsupervised image segmentation by backpropagation","authors":"Asako Kanezaki","year":"2018","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. technical report, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Krizhevsky","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[MacQueen and others, 1967] James MacQueen et al. Some meth- ods for classification and analysis of multivariate observations. In Berkeley symposium on mathematical statistics and probability, volume 1(14), pages 281-297.","title":"Some methods for classification and analysis of multivariate observations","authors":"James Macqueen, Macqueen","year":"1967","venue":"Berkeley symposium on mathematical statistics and probability","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Petegrosso et al., 2020] Raphael Petegrosso, Zhuliu Li, and Rui Kuang. Machine learning and statistical methods for cluster- ing single-cell rna-sequencing data. Briefings in bioinformatics, 21(4):1209-1223, 2020.","title":"Machine learning and statistical methods for clustering single-cell rna-sequencing data","authors":"Raphael Petegrosso, Zhuliu Li, Rui Kuang","year":"2020","venue":"Briefings in bioinformatics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ren et al., 2020] Yazhou Ren, Ni Wang, Mingxia Li, and Zenglin Xu. Deep density-based image clustering. Knowledge-Based Sys- tems, page 105841, 2020.","title":"Deep density-based image clustering","authors":"Ni Yazhou Ren, Mingxia Wang, Zenglin Li, Xu","year":"2020","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rodriguez and Laio, 2014] Alex Rodriguez and Alessandro Laio. Clustering by fast search and find of density peaks. Science, 344(6191):1492-1496, 2014.","title":"Clustering by fast search and find of density peaks","authors":"Alex Rodriguez, Alessandro Laio","year":"2014","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale im- age recognition. In ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tsai et al., 2020] Tsung Wei Tsai, Chongxuan Li, and Jun Zhu. Mice: Mixture of contrastive experts for unsupervised image clustering. In ICLR, 2020.","title":"Mice: Mixture of contrastive experts for unsupervised image clustering","authors":"Tsung Wei Tsai, Chongxuan Li, Jun Zhu","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Van Gansbeke et al., 2020] Wouter Van Gansbeke, Simon Van- denhende, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool. Scan: Learning to classify images without labels. In ECCV, pages 268-285. Springer, 2020.","title":"Scan: Learning to classify images without labels","authors":"Van Gansbeke","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2016] Junyuan Xie, Ross Girshick, and Ali Farhadi. Un- supervised deep embedding for clustering analysis. In ICML, pages 478-487, 2016.","title":"Unsupervised deep embedding for clustering analysis","authors":"Junyuan Xie, Ross Girshick, Ali Farhadi","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2003] Wei Xu, Xin Liu, and Yihong Gong. Document clustering based on non-negative matrix factorization. In SIGIR, pages 267-273, 2003.","title":"Document clustering based on non-negative matrix factorization","authors":"Wei Xu, Xin Liu, Yihong Gong","year":"2003","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2016] Jianwei Yang, Devi Parikh, and Dhruv Batra. Joint unsupervised learning of deep representations and image clusters. In CVPR, pages 5147-5156, 2016.","title":"Joint unsupervised learning of deep representations and image clusters","authors":"Jianwei Yang, Devi Parikh, Dhruv Batra","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang et al., 2019] Xu Yang, Cheng Deng, Feng Zheng, Junchi Yan, and Wei Liu. Deep spectral clustering using dual autoen- coder network. In CVPR, pages 4066-4075, 2019.","title":"Deep spectral clustering using dual autoencoder network","authors":"Xu Yang, Cheng Deng, Feng Zheng, Junchi Yan, Wei Liu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zelnik-Manor and Perona, 2005] Lihi Zelnik-Manor and Pietro Perona. Self-tuning spectral clustering. In NeurIPS, pages 1601- 1608, 2005.","title":"Self-tuning spectral clustering","authors":"Lihi Zelnik, -Manor, Pietro Perona","year":"2005","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhong et al., 2021] Huasong Zhong, Jianlong Wu, Chong Chen, Jianqiang Huang, Minghua Deng, Liqiang Nie, Zhouchen Lin, and Xian-Sheng Hua. Graph contrastive clustering. In ICCV, pages 9224-9233, October 2021.","title":"Graph contrastive clustering","authors":"Huasong Zhong, Jianlong Wu, Chong Chen, Jianqiang Huang, Minghua Deng, Liqiang Nie, Zhouchen Lin, Xian-Sheng Hua","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zou et al., 2020] Quan Zou, Gang Lin, Xingpeng Jiang, Xian- grong Liu, and Xiangxiang Zeng. Sequence clustering in bioin- formatics: an empirical study. Briefings in bioinformatics, 21(1):1-10, 2020.","title":"Sequence clustering in bioinformatics: an empirical study","authors":"Quan Zou, Gang Lin, Xingpeng Jiang, Xiangrong Liu, Xiangxiang Zeng","year":"2020","venue":"Briefings in bioinformatics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}