{"type":"ScholarlyArticle","title":"Deep Embedding Clustering Driven by Sample Stability","authors":[{"name":"Zhanwen Cheng","slug":"zhanwen-cheng","given":"Zhanwen","family":"Cheng","affiliation":"Institute of Big Data Science and Industry , Shanxi University , Taiyuan 030006 , China."},{"name":"Feijiang Li","slug":"feijiang-li","given":"Feijiang","family":"Li","affiliation":"Institute of Big Data Science and Industry , Shanxi University , Taiyuan 030006 , China."},{"name":"Jieting Wang","slug":"jieting-wang","given":"Jieting","family":"Wang","affiliation":"Institute of Big Data Science and Industry , Shanxi University , Taiyuan 030006 , China."},{"name":"Yuhua Qian","slug":"yuhua-qian","given":"Yuhua","family":"Qian","affiliation":"Institute of Big Data Science and Industry , Shanxi University , Taiyuan 030006 , China."}],"abstract":"Deep clustering methods improve the performance of clustering tasks by jointly optimizing deep representation learning and clustering.  While numerous deep clustering algorithms have been proposed, most of them rely on artificially constructed pseudo targets for performing clustering.  This construction process requires some prior knowledge, and it is challenging to determine a suitable pseudo target for clustering.  To address this issue, we propose a deep embedding clustering algorithm driven by sample stability (DECS), which eliminates the requirement of pseudo targets.  Specifically, we start by constructing the initial feature space with an autoencoder and then learn the cluster-oriented embedding feature constrained by sample stability.  The sample stability aims to explore the deterministic relationship between samples and all cluster centroids, pulling samples to their respective clusters and keeping them away from other clusters with high determinacy.  We analyzed the convergence of the loss using Lipschitz continuity in theory, which verifies the validity of the model.  The experimental results on five datasets illustrate that the proposed method achieves superior performance compared to state-of-the-art clustering approaches.","doi":"10.24963/ijcai.2024/426","url":"https://ijcai.org/proceedings/2024/426","pdf":"https://ijcai.org/proceedings/2024/0426.pdf","markdown":"https://ijcai.org/proceedings/2024/426.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/426","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":426,"pages":{"first":3854,"last":3862},"keywords":["Machine Learning: ML: Deep learning architectures","Machine Learning: ML: Clustering","Machine Learning: ML: Convolutional networks","Machine Learning: ML: Unsupervised learning"],"videos":[],"references":[{"position":1,"raw":"[Bishop and Nasrabadi, 2006] Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.","title":"Pattern recognition and machine learning","authors":"M Christopher, Bishop, M Nasser, Nasrabadi","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen and Cai, 2011] Xinlei Chen and Deng Cai. Large scale spectral clustering with landmark-based representa- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 25, pages 313-318, 2011.","title":"Large scale spectral clustering with landmark-based representation","authors":"Chen, Cai, Xinlei Chen, Deng Cai","year":"2011","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ghasedi Dizaji et al., 2017] Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, and Heng Huang. Deep clustering via joint convolutional au- toencoder embedding and relative entropy minimization. In Proceedings of the IEEE international conference on computer vision, pages 5736-5745, 2017.","title":"Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization","authors":"Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, Heng Huang","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Guo et al., 2017a] Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. Improved deep embedded clustering with local structure preservation. In Ijcai, volume 17, pages 1753-1759, 2017.","title":"Improved deep embedded clustering with local structure preservation","authors":"Guo","year":"2017","venue":"Ijcai","doi":null,"url":null,"internal_url":"/proceedings/2017/243"},{"position":5,"raw":"[Guo et al., 2017b] Xifeng Guo, Xinwang Liu, En Zhu, and Jianping Yin. Deep clustering with convolutional autoen- coders. In Neural Information Processing: 24th Inter- national Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part II 24, pages 373-382. Springer, 2017.","title":"Deep clustering with convolutional autoencoders","authors":"Guo","year":"2017","venue":"Neural Information Processing: 24th International Conference, ICONIP 2017","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guo et al., 2018] Xifeng Guo, En Zhu, Xinwang Liu, and Jianping Yin. Deep embedded clustering with data aug- mentation. In Asian conference on machine learning, pages 550-565. PMLR, 2018.","title":"Deep embedded clustering with data augmentation","authors":"Xifeng Guo, En Zhu, Xinwang Liu, Jianping Yin","year":"2018","venue":"Asian conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guo et al., 2019] Xifeng Guo, Xinwang Liu, En Zhu, Xinzhong Zhu, Miaomiao Li, Xin Xu, and Jianping Yin. Adaptive self-paced deep clustering with data augmenta- tion. IEEE Transactions on Knowledge and Data Engi- neering, 32(9):1680-1693, 2019.","title":"Adaptive self-paced deep clustering with data augmentation","authors":"Xifeng Guo, Xinwang Liu, En Zhu, Xinzhong Zhu, Miaomiao Li, Xin Xu, Jianping Yin","year":"2019","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jabi et al., 2019] Mohammed Jabi, Marco Pedersoli, Amar Mitiche, and Ismail Ben Ayed. Deep clustering: On the link between discriminative models and k-means. IEEE transactions on pattern analysis and machine intelligence, 43(6):1887-1896, 2019.","title":"Deep clustering: On the link between discriminative models and k-means","authors":"Mohammed Jabi, Marco Pedersoli, Amar Mitiche, Ismail Ben, Ayed","year":"2019","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jaiswal et al., 2020] Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. A survey on contrastive self-supervised learn- ing. Technologies, 9(1):2, 2020.","title":"A survey on contrastive self-supervised learning","authors":"Ashish Jaiswal, Ramesh Ashwin, Mohammad Zaki Babu, Debapriya Zadeh, Fillia Banerjee, Makedon","year":"2020","venue":"Technologies","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2016] Zhuxi Jiang, Yin Zheng, Huachun Tan, Bangsheng Tang, and Hanning Zhou. Variational deep embedding: An unsupervised and generative approach to clustering. arXiv preprint arXiv:1611.05148, 2016.","title":"Variational deep embedding: An unsupervised and generative approach to clustering","authors":"Jiang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/273"},{"position":11,"raw":"[Jing and Tian, 2020] Longlong Jing and Yingli Tian. Self- supervised visual feature learning with deep neural net- works: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(11):4037-4058, 2020.","title":"Longlong Jing and Yingli Tian. Selfsupervised visual feature learning with deep neural networks: A survey","authors":"Tian Jing","year":"2020","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Johnson, 1967] Stephen C Johnson. Hierarchical clustering schemes. Psychometrika, 32(3):241-254, 1967.","title":"Hierarchical clustering schemes","authors":"Johnson Stephen","year":"1967","venue":"Psychometrika","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Koga et al., 2007] Hisashi Koga, Tetsuo Ishibashi, and Toshinori Watanabe. Fast agglomerative hierarchical clus- tering algorithm using locality-sensitive hashing. Knowl- edge and Information Systems, 12:25-53, 2007.","title":"Fast agglomerative hierarchical clustering algorithm using locality-sensitive hashing","authors":"Hisashi Koga, Tetsuo Ishibashi, Toshinori Watanabe","year":"2007","venue":"Knowledge and Information Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2018] Fengfu Li, Hong Qiao, and Bo Zhang. Dis- criminatively boosted image clustering with fully convo- lutional auto-encoders. Pattern Recognition, 83:161-173, 2018.","title":"Discriminatively boosted image clustering with fully convolutional auto-encoders","authors":"Li","year":"2018","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2019] Feijiang Li, Yuhua Qian, Jieting Wang, Chuangyin Dang, and Liping Jing. Clustering ensemble based on sample's stability. Artificial Intelligence, 273:37- 55, 2019.","title":"Clustering ensemble based on sample's stability","authors":"Feijiang Li, Yuhua Qian, Jieting Wang, Chuangyin Dang, Liping Jing","year":"2019","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2020] F Li, Y Qian, J Wang, J Liang, and W Wang. Clustering method based on samples stability. Sci. Sin. Inf., 50(8):1239-1254, 2020.","title":"Clustering method based on samples stability","authors":"Y Li, J Qian, J Wang, Liang, Wang","year":"2020","venue":"Sci. Sin. Inf","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2021] Xuelong Li, Hongyuan Zhang, and Rui Zhang. Adaptive graph auto-encoder for general data clus- tering. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 44(12):9725-9732, 2021.","title":"Adaptive graph auto-encoder for general data clustering","authors":"Xuelong Li, Hongyuan Zhang, Rui Zhang","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lu et al., 2022] Hu Lu, Chao Chen, Hui Wei, Zhongchen Ma, Ke Jiang, and Yingquan Wang. Improved deep con- volutional embedded clustering with re-selectable sample training. Pattern Recognition, 127:108611, 2022.","title":"Improved deep convolutional embedded clustering with re-selectable sample training","authors":"Hu Lu, Chao Chen, Hui Wei, Zhongchen Ma, Ke Jiang, Yingquan Wang","year":"2022","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[MacQueen, 1967a] J MacQueen. Classification and analy- sis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Prob- ability, pages 281-297, 1967.","title":"Classification and analysis of multivariate observations","authors":"; J Macqueen, Macqueen","year":"1967","venue":"Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[MacQueen, 1967b] J. MacQueen. Some methods for classi- fication and analysis of multivariate observations. In Pro- ceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281-297.","title":"Some methods for classification and analysis of multivariate observations","authors":"; J Macqueen, Macqueen","year":"1967","venue":"Proceedings of the fifth Berkeley symposium on mathematical statistics and probability","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mrabah et al., 2020] Nairouz Mrabah, Naimul Mefraz Khan, Riadh Ksantini, and Zied Lachiri. Deep clustering with a dynamic autoencoder: From reconstruction towards centroids construction. Neural Networks, 130:206-228, 2020.","title":"Deep clustering with a dynamic autoencoder: From reconstruction towards centroids construction","authors":"Mrabah","year":"2020","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Nesterov, 1998] Yurii Nesterov. Introductory lectures on convex programming volume i: Basic course. Lecture notes, 3(4):5, 1998.","title":"Introductory lectures on convex programming volume i: Basic course","authors":"Yurii Nesterov","year":"1998","venue":"Lecture notes","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ng et al., 2001] Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 14, 2001.","title":"On spectral clustering: Analysis and an algorithm","authors":"Andrew Ng, Michael Jordan, Yair Weiss","year":"2001","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Niu et al., 2020] Chuang Niu, Jun Zhang, Ge Wang, and Jimin Liang. Gatcluster: Self-supervised gaussian- attention network for image clustering. In Computer Vision-ECCV 2020: 16th European Conference, Glas- gow, UK, August 23-28, 2020, Proceedings, Part XXV 16, pages 735-751. Springer, 2020.","title":"Gatcluster: Self-supervised gaussianattention network for image clustering","authors":"Chuang Niu, Jun Zhang, Ge Wang, Jimin Liang","year":"2020","venue":"Computer Vision-ECCV 2020: 16th European Conference","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Niu et al., 2022] Chuang Niu, Hongming Shan, and Ge Wang. Spice: Semantic pseudo-labeling for image clustering. IEEE Transactions on Image Processing, 31:7264-7278, 2022.","title":"Spice: Semantic pseudo-labeling for image clustering","authors":"Chuang Niu, Hongming Shan, Ge Wang","year":"2022","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Peng et al., 2022] Xi Peng, Yunfan Li, Ivor W Tsang, Hongyuan Zhu, Jiancheng Lv, and Joey Tianyi Zhou. Xai beyond classification: Interpretable neural clustering. The Journal of Machine Learning Research, 23(1):227-254, 2022.","title":"Xai beyond classification: Interpretable neural clustering","authors":"Xi Peng, Yunfan Li, Ivor W Tsang, Hongyuan Zhu, Jiancheng Lv, Joey Tianyi Zhou","year":"2022","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ren et al., 2019] Yazhou Ren, Kangrong Hu, Xinyi Dai, Lili Pan, Steven CH Hoi, and Zenglin Xu. Semi- supervised deep embedded clustering. Neurocomputing, 325:121-130, 2019.","title":"Semisupervised deep embedded clustering","authors":"Kangrong Yazhou Ren, Xinyi Hu, Lili Dai, Pan, C H Steven, Zenglin Hoi, Xu","year":"2019","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Reynolds, 2009] Douglas Reynolds. Gaussian Mixture Models, pages 659-663. Springer US, Boston, MA, 2009.","title":"Gaussian Mixture Models","authors":"Douglas Reynolds","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Rezaei et al., 2021] Mina Rezaei, Emilio Dorigatti, David Ruegamer, and Bernd Bischl. Learning statistical rep- resentation with joint deep embedded clustering. arXiv preprint arXiv:2109.05232, 1, 2021.","title":"Learning statistical representation with joint deep embedded clustering","authors":"Mina Rezaei, Emilio Dorigatti, David Ruegamer, Bernd Bischl","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ronen et al., 2022] Meitar Ronen, Shahaf E Finder, and Oren Freifeld. Deepdpm: Deep clustering with an unknown number of clusters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9861-9870, 2022.","title":"Deepdpm: Deep clustering with an unknown number of clusters","authors":"Shahaf E Meitar Ronen, Oren Finder, Freifeld","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Shi and Malik, 2000] Jianbo Shi and Jitendra Malik. Nor- malized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence, 22(8):888- 905, 2000.","title":"Normalized cuts and image segmentation","authors":"Jianbo Shi, Jitendra Malik","year":"2000","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sneath and Sokal, 1962] Peter HA Sneath and Robert R Sokal. Numerical taxonomy. Nature, 193:855-860, 1962.","title":"Numerical taxonomy","authors":"H A Peter, Robert R Sneath, Sokal","year":"1962","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xie et al., 2016] Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In International conference on machine learn- ing, pages 478-487. PMLR, 2016.","title":"Unsupervised deep embedding for clustering analysis","authors":"Junyuan Xie, Ross Girshick, Ali Farhadi","year":"2016","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xu and Wunsch, 2005] Rui Xu and Donald Wunsch. Sur- vey of clustering algorithms. IEEE Transactions on neural networks, 16(3):645-678, 2005.","title":"Survey of clustering algorithms","authors":"Rui Xu, Donald Wunsch","year":"2005","venue":"IEEE Transactions on neural networks","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yang et al., 2016] Jianwei Yang, Devi Parikh, and Dhruv Batra. Joint unsupervised learning of deep representations and image clusters. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5147- 5156, 2016.","title":"Joint unsupervised learning of deep representations and image clusters","authors":"Jianwei Yang, Devi Parikh, Dhruv Batra","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yang et al., 2018] Xu Yang, Cheng Deng, Xianglong Liu, and Feiping Nie. New l2, 1-norm relaxation of multi-way graph cut for clustering. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 32, 2018.","title":"New l2, 1-norm relaxation of multi-way graph cut for clustering","authors":"Xu Yang, Cheng Deng, Xianglong Liu, Feiping Nie","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yang et al., 2019] Xu Yang, Cheng Deng, Feng Zheng, Junchi Yan, and Wei Liu. Deep spectral clustering us- ing dual autoencoder network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4066-4075, 2019.","title":"Deep spectral clustering using dual autoencoder network","authors":"Xu Yang, Cheng Deng, Feng Zheng, Junchi Yan, Wei Liu","year":"2019","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhang et al., 2013] Wei Zhang, Deli Zhao, and Xiaogang Wang. Agglomerative clustering via maximum incremen- tal path integral. Pattern Recognition, 46(11):3056-3065, 2013.","title":"Agglomerative clustering via maximum incremental path integral","authors":"Wei Zhang, Deli Zhao, Xiaogang Wang","year":"2013","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhang et al., 2023] Ruilin Zhang, Haiyang Zheng, and Hongpeng Wang. Tdec: Deep embedded image cluster- ing with transformer and distribution information. In Pro- ceedings of the 2023 ACM International Conference on Multimedia Retrieval, pages 280-288, 2023.","title":"Tdec: Deep embedded image clustering with transformer and distribution information","authors":"Ruilin Zhang, Haiyang Zheng, Hongpeng Wang","year":"2023","venue":"Proceedings of the 2023 ACM International Conference on Multimedia Retrieval","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}