{"type":"ScholarlyArticle","title":"Contrastive Multi-view Hyperbolic Hierarchical Clustering","authors":[{"name":"Fangfei Lin","slug":"fangfei-lin","given":"Fangfei","family":"Lin","affiliation":"University of Electronic Science and Technology of China , Chengdu , China; Tencent Security Big Data Lab , Tencent Inc. , China"},{"name":"Bing Bai","slug":"bing-bai","given":"Bing","family":"Bai","affiliation":"Tencent Security Big Data Lab , Tencent Inc. , China"},{"name":"Kun Bai","slug":"kun-bai","given":"Kun","family":"Bai","affiliation":"Tencent Security Big Data Lab , Tencent Inc. , China"},{"name":"Yazhou Ren","slug":"yazhou-ren","given":"Yazhou","family":"Ren","affiliation":"University of Electronic Science and Technology of China , Chengdu , China"},{"name":"Peng Zhao","slug":"peng-zhao","given":"Peng","family":"Zhao","affiliation":"University of Electronic Science and Technology of China , Chengdu , China"},{"name":"Zenglin Xu","slug":"zenglin-xu","given":"Zenglin","family":"Xu","affiliation":"Harbin Institute of Technology , Shenzhen , China; Department of Network Intelligence , Peng Cheng National Lab , Shenzhen , China"}],"abstract":"Hierarchical clustering recursively partitions data at an increasingly finer granularity. In real-world applications, multi-view data have become increasingly important. This raises a less investigated problem, i.e., multi-view hierarchical clustering, to better understand the hierarchical structure of multi-view data. To this end, we propose a novel neural network-based model, namely Contrastive Multi-view Hyperbolic Hierarchical Clustering(CMHHC). It consists of three components, i.e., multi-view alignment learning, aligned feature similarity learning, and continuous hyperbolic hierarchical clustering. First, we align sample-level representations across multiple views in a contrastive way to capture the view-invariance information. Next, we utilize both the manifold and Euclidean similarities to improve the metric property. Then, we embed the representations into a hyperbolic space and optimize the hyperbolic embeddings via a continuous relaxation of hierarchical clustering loss. Finally, a binary clustering tree is decoded from optimized hyperbolic embeddings. Experimental results on five real-world datasets demonstrate the effectiveness of the proposed method and its components.","doi":"10.24963/ijcai.2022/451","url":"https://ijcai.org/proceedings/2022/451","pdf":"https://ijcai.org/proceedings/2022/0451.pdf","markdown":"https://ijcai.org/proceedings/2022/451.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/451","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":451,"pages":{"first":3250,"last":3256},"keywords":["Machine Learning: Clustering","Machine Learning: Multi-view learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986480","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984623","length_sec":341,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In ICML, pages 1247-1255, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Jeff Bilmes, Karen Livescu","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2013] Xiao Cai, Feiping Nie, and Heng Huang. Multi-view k-means clustering on big data. In IJCAI, pages 2598-2604, 2013.","title":"Multi-view k-means clustering on big data","authors":"Xiao Cai, Feiping Nie, Heng Huang","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/383"},{"position":3,"raw":"[Chami et al., 2020] Ines Chami, Albert Gu, Vaggos Chatzi- afratis, and Christopher Ré. From trees to continuous em- beddings and back: Hyperbolic hierarchical clustering. In NeurIPS, pages 15065-15076, 2020.","title":"From trees to continuous embeddings and back: Hyperbolic hierarchical clustering","authors":"Ines Chami, Albert Gu, Vaggos Chatziafratis, Christopher Ré","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chaudhuri et al., 2009] Kamalika Chaudhuri, Sham M Kakade, Karen Livescu, and Karthik Sridharan. Multi- view clustering via canonical correlation analysis. In ICML, pages 129-136, 2009.","title":"Multiview clustering via canonical correlation analysis","authors":"Kamalika Chaudhuri, M Sham, Karen Kakade, Karthik Livescu, Sridharan","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597-1607, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chierchia and Perret, 2020] Giovanni Chierchia and Ben- jamin Perret. Ultrametric fitting by gradient descent. Jour- nal of Statistical Mechanics: Theory and Experiment, 2020(12):124004, 2020.","title":"Giovanni Chierchia and Benjamin Perret. Ultrametric fitting by gradient descent","authors":"Perret Chierchia","year":"2020","venue":"Journal of Statistical Mechanics: Theory and Experiment","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dasgupta, 2016] Sanjoy Dasgupta. A cost function for similarity-based hierarchical clustering. In STOC, pages 118-127, 2016.","title":"A cost function for similarity-based hierarchical clustering","authors":"Sanjoy Dasgupta","year":"2016","venue":"STOC","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dueck and Frey, 2007] Delbert Dueck and Brendan J Frey. Non-metric affinity propagation for unsupervised image categorization. In ICCV, pages 1-8, 2007.","title":"Non-metric affinity propagation for unsupervised image categorization","authors":"Frey Dueck, Delbert Dueck, Brendan J Frey","year":"2007","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hinton and Salakhutdinov, 2006] Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, 2006.","title":"Reducing the dimensionality of data with neural networks","authors":"Geoffrey E Hinton, Ruslan R Salakhutdinov","year":"2006","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Iscen et al., 2017] Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, Teddy Furon, and Ondrej Chum. Efficient diffu- sion on region manifolds: Recovering small objects with compact cnn representations. In CVPR, pages 2077-2086, 2017.","title":"Efficient diffusion on region manifolds: Recovering small objects with compact cnn representations","authors":"Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, Teddy Furon, Ondrej Chum","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Iscen et al., 2018] Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondřej Chum. Mining on manifolds: Met- ric learning without labels. In CVPR, pages 7642-7651, 2018.","title":"Mining on manifolds: Metric learning without labels","authors":"Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, Ondřej Chum","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jain et al., 1999] Anil K Jain, M Narasimha Murty, and Patrick J Flynn. Data clustering: a review. ACM com- puting surveys (CSUR), 31(3):264-323, 1999.","title":"Data clustering: a review","authors":"M Anil K Jain, Patrick J Narasimha Murty, Flynn","year":"1999","venue":"ACM computing surveys","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kang et al., 2020] Zhao Kang, Guoxin Shi, Shudong Huang, Wenyu Chen, Xiaorong Pu, Joey Tianyi Zhou, and Zenglin Xu. Multi-graph fusion for multi-view spectral clustering. Knowledge-Based Systems, 189:105102, 2020.","title":"Multi-graph fusion for multi-view spectral clustering","authors":"Zhao Kang, Guoxin Shi, Shudong Huang, Wenyu Chen, Xiaorong Pu, Joey Tianyi Zhou, Zenglin Xu","year":"2020","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kobren et al., 2017] Ari Kobren, Nicholas Monath, Akshay Krishnamurthy, and Andrew McCallum. A hierarchical algorithm for extreme clustering. In SIGKDD, pages 255- 264, 2017.","title":"A hierarchical algorithm for extreme clustering","authors":"Ari Kobren, Nicholas Monath, Akshay Krishnamurthy, Andrew Mccallum","year":"2017","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kochurov et al., 2020] Max Kochurov, Rasul Karimov, and Serge Kozlukov. Geoopt: Riemannian optimization in Py- Torch. arXiv preprint arXiv:2005.02819, 2020.","title":"Geoopt: Riemannian optimization in Py-Torch","authors":"Kochurov","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2018] Yingming Li, Ming Yang, and Zhongfei Zhang. A survey of multi-view representation learning. TKDE, 31(10):1863-1883, 2018.","title":"A survey of multi-view representation learning","authors":"Li","year":"2018","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2019a] Ruihuang Li, Changqing Zhang, Huazhu Fu, Xi Peng, Tianyi Zhou, and Qinghua Hu. Reciprocal multi-layer subspace learning for multi-view clustering. In ICCV, pages 8172-8180, 2019.","title":"Reciprocal multi-layer subspace learning for multi-view clustering","authors":"Li","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2019b] Zhaoyang Li, Qianqian Wang, Zhiqiang Tao, Quanxue Gao, and Zhaohua Yang. Deep adversar- ial multi-view clustering network. In IJCAI, pages 2952- 2958, 2019.","title":"Deep adversarial multi-view clustering network","authors":"Li","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/409"},{"position":19,"raw":"[Monath et al., 2019] Nicholas Monath, Manzil Zaheer, Daniel Silva, Andrew McCallum, and Amr Ahmed. Gradient-based hierarchical clustering using continuous representations of trees in hyperbolic space. In SIGKDD, pages 714-722, 2019.","title":"Gradient-based hierarchical clustering using continuous representations of trees in hyperbolic space","authors":"Nicholas Monath, Manzil Zaheer, Daniel Silva, Andrew Mccallum, Amr Ahmed","year":"2019","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nickel and Kiela, 2017] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical rep- resentations. In NeurIPS, pages 6338-6347, 2017.","title":"Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations","authors":"Kiela Nickel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Peng et al., 2019] Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, and Joey Tianyi Zhou. Comic: Multi-view clustering without parameter selection. In ICML, pages 5092-5101, 2019.","title":"Comic: Multi-view clustering without parameter selection","authors":"Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, Joey Tianyi Zhou","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sala et al., 2018] Frederic Sala, Chris De Sa, Albert Gu, and Christopher Ré. Representation tradeoffs for hyperbolic embeddings. In ICML, pages 4460-4469, 2018.","title":"Representation tradeoffs for hyperbolic embeddings","authors":"Frederic Sala, Chris De Sa, Albert Gu, Christopher Ré","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Trosten et al., 2021] Daniel J Trosten, Sigurd Lokse, Robert Jenssen, and Michael Kampffmeyer. Reconsidering repre- sentation alignment for multi-view clustering. In CVPR, pages 1255-1265, 2021.","title":"Reconsidering representation alignment for multi-view clustering","authors":"Sigurd Daniel J Trosten, Robert Lokse, Michael Jenssen, Kampffmeyer","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2015] Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multi-view represen- tation learning. In ICML, pages 1083-1092, 2015.","title":"On deep multi-view representation learning","authors":"Wang","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu et al., 2021] Jie Xu, Yazhou Ren, Huayi Tang, Xiaorong Pu, Xiaofeng Zhu, Ming Zeng, and Lifang He. Multi- VAE: Learning disentangled view-common and view- peculiar visual representations for multi-view clustering. In ICCV, pages 9234-9243, 2021.","title":"Multi-VAE: Learning disentangled view-common and viewpeculiar visual representations for multi-view clustering","authors":"Jie Xu, Yazhou Ren, Huayi Tang, Xiaorong Pu, Xiaofeng Zhu, Ming Zeng, Lifang He","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2022] Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. Multi-level feature learning for contrastive multi-view clustering. In CVPR, 2022.","title":"Multi-level feature learning for contrastive multi-view clustering","authors":"Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, Lifang He","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yan et al., 2021] Jiexi Yan, Lei Luo, Cheng Deng, and Heng Huang. Unsupervised hyperbolic metric learning. In CVPR, pages 12465-12474, 2021.","title":"Unsupervised hyperbolic metric learning","authors":"Yan","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zheng et al., 2020] Qinghai Zheng, Jihua Zhu, and Shuangxun Ma. Multi-view hierarchical clustering. arXiv preprint arXiv:2010.07573, 2020.","title":"Multi-view hierarchical clustering","authors":"Zheng","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou and Shen, 2020] Runwu Zhou and Yi-Dong Shen. End-to-end adversarial-attention network for multi-modal clustering. In CVPR, pages 14619-14628, 2020.","title":"End-to-end adversarial-attention network for multi-modal clustering","authors":"Shen Zhou, Runwu Zhou, Yi-Dong Shen","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}