{"type":"ScholarlyArticle","title":"Incomplete Multi-view Clustering via Prototype-based Imputation","authors":[{"name":"Haobin Li","slug":"haobin-li","given":"Haobin","family":"Li","affiliation":"College of Computer Science , Sichuan University"},{"name":"Yunfan Li","slug":"yunfan-li","given":"Yunfan","family":"Li","affiliation":"College of Computer Science , Sichuan University"},{"name":"Mouxing Yang","slug":"mouxing-yang","given":"Mouxing","family":"Yang","affiliation":"College of Computer Science , Sichuan University"},{"name":"Peng Hu","slug":"peng-hu","given":"Peng","family":"Hu","affiliation":"College of Computer Science , Sichuan University"},{"name":"Dezhong Peng","slug":"dezhong-peng","given":"Dezhong","family":"Peng","affiliation":"College of Computer Science , Sichuan University"},{"name":"Xi Peng","slug":"xi-peng","given":"Xi","family":"Peng","affiliation":"College of Computer Science , Sichuan University"}],"abstract":"In this paper, we study how to achieve two characteristics highly-expected by incomplete multi-view clustering (IMvC). Namely, i) instance commonality refers to that within-cluster instances should share a common pattern, and ii) view versatility refers to that cross-view samples should own view-specific patterns. To this end, we design a novel dual-stream model which employs a dual attention layer and a dual contrastive learning loss to learn view-specific prototypes and model the sample-prototype relationship. When the view is missed, our model performs data recovery using the prototypes in the missing view and the sample-prototype relationship inherited from the observed view. Thanks to our dual-stream model, both cluster- and view-specific information could be captured, and thus the instance commonality and view versatility could be preserved to facilitate IMvC. Extensive experiments demonstrate the superiority of our method on five challenging benchmarks compared with 11 approaches. The code could be accessed from https://pengxi.me.","doi":"10.24963/ijcai.2023/435","url":"https://ijcai.org/proceedings/2023/435","pdf":"https://ijcai.org/proceedings/2023/0435.pdf","markdown":"https://ijcai.org/proceedings/2023/435.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/435","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":435,"pages":{"first":3911,"last":3919},"keywords":["Machine Learning: ML: Clustering","Machine Learning: ML: Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Amini et al., 2009] Massih R Amini, Nicolas Usunier, and Cyril Goutte. Learning from multiple partially observed views-an application to multilingual text categorization. NeurIPS, 2009.","title":"Learning from multiple partially observed views-an application to multilingual text categorization","authors":"Nicolas Massih R Amini, Cyril Usunier, Goutte","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In ICML, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Jeff Bilmes, Karen Livescu","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bach and Jordan, 2002] Francis R Bach and Michael I Jor- dan. Kernel independent component analysis. Journal of Machine Learning Research, 2002.","title":"Kernel independent component analysis","authors":"R Francis, Michael I Bach, Jordan","year":"2002","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fei-Fei and Perona, 2005] Li Fei-Fei and Pietro Perona. A bayesian hierarchical model for learning natural scene cat- egories. In CVPR, 2005.","title":"A bayesian hierarchical model for learning natural scene categories","authors":"Li Fei, -Fei, Pietro Perona","year":"2005","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hu and Chen, 2019] Menglei Hu and Songcan Chen. Doubly aligned incomplete multi-view clustering. arXiv:1903.02785, 2019.","title":"Doubly aligned incomplete multi-view clustering","authors":"Chen ; Hu, Menglei Hu, Songcan Chen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2018/313"},{"position":6,"raw":"[Hu et al., 2019] Peng Hu, Dezhong Peng, Yongsheng Sang, and Yong Xiang. Multi-view linear discriminant analysis network. IEEE Transactions on Image Processing, 2019.","title":"Multi-view linear discriminant analysis network","authors":"Peng Hu, Dezhong Peng, Yongsheng Sang, Yong Xiang","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2019] Zhenyu Huang, Joey Tianyi Zhou, Xi Peng, Changqing Zhang, Hongyuan Zhu, and Jiancheng Lv. Multi-view spectral clustering network. In IJCAI, 2019.","title":"Multi-view spectral clustering network","authors":"Zhenyu Huang, Joey Tianyi Zhou, Xi Peng, Changqing Zhang, Hongyuan Zhu, Jiancheng Lv","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/356"},{"position":8,"raw":"[Jiang et al., 2019] Yangbangyan Jiang, Qianqian Xu, Zhiy- ong Yang, Xiaochun Cao, and Qingming Huang. Dm2c: Deep mixed-modal clustering. NeurIPS, 2019.","title":"Dm2c: Deep mixed-modal clustering","authors":"Yangbangyan Jiang, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, Qingming Huang","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2014] Shao-Yuan Li, Yuan Jiang, and Zhi-Hua Zhou. Partial multi-view clustering. In AAAI, 2014.","title":"Partial multi-view clustering","authors":"Shao-Yuan, Yuan Li, Zhi-Hua Jiang, Zhou","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2021] Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive clus- tering. In AAAI, 2021.","title":"Contrastive clustering","authors":"Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, Xi Peng","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2022] Yunfan Li, Mouxing Yang, Dezhong Peng, Taihao Li, Jiantao Huang, and Xi Peng. Twin contrastive learning for online clustering. International Journal of Computer Vision, 2022.","title":"Twin contrastive learning for online clustering","authors":"Li","year":"2022","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2021] Yijie Lin, Yuanbiao Gou, Zitao Liu, Boyun Li, Jiancheng Lv, and Xi Peng. Completer: Incomplete multi-view clustering via contrastive prediction. In CVPR, 2021.","title":"Completer: Incomplete multi-view clustering via contrastive prediction","authors":"Yijie Lin, Yuanbiao Gou, Zitao Liu, Boyun Li, Jiancheng Lv, Xi Peng","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2022] Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. Dual contrastive prediction for incomplete multi-view representation learn- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.","title":"Dual contrastive prediction for incomplete multi-view representation learning","authors":"Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, Xi Peng","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2019] Xinwang Liu, Xinzhong Zhu, Miaomiao Li, Lei Wang, En Zhu, Tongliang Liu, Marius Kloft, Ding- gang Shen, Jianping Yin, and Wen Gao. Multiple kernel k k-means with incomplete kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.","title":"Multiple kernel k k-means with incomplete kernels","authors":"Liu","year":"2019","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2020] Xinwang Liu, Miaomiao Li, Chang Tang, Jingyuan Xia, Jian Xiong, Li Liu, Marius Kloft, and En Zhu. Efficient and effective regularized incomplete multi-view clustering. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 2020.","title":"Efficient and effective regularized incomplete multi-view clustering","authors":"Liu","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Luo et al., 2020] Keyang Luo, Tao Guan, Lili Ju, Yuesong Wang, Zhuo Chen, and Yawei Luo. Attention-aware multi- view stereo. In CVPR, 2020.","title":"Attention-aware multiview stereo","authors":"Keyang Luo, Tao Guan, Lili Ju, Yuesong Wang, Zhuo Chen, Yawei Luo","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Peng et al., 2019] Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, and Joey Tianyi Zhou. Comic: Multi-view clustering without parameter selection. In ICML, 2019.","title":"Comic: Multi-view clustering without parameter selection","authors":"Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, Joey Tianyi Zhou","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qu et al., 2017] Meng Qu, Jian Tang, Jingbo Shang, Xiang Ren, Ming Zhang, and Jiawei Han. An attention-based collaboration framework for multi-view network represen- tation learning. In CIKM, 2017.","title":"An attention-based collaboration framework for multi-view network representation learning","authors":"Meng Qu, Jian Tang, Jingbo Shang, Xiang Ren, Ming Zhang, Jiawei Han","year":"2017","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rush et al., 2015] Alexander M Rush, Sumit Chopra, and Jason Weston. A neural attention model for abstractive sentence summarization. arXiv:1509.00685, 2015.","title":"A neural attention model for abstractive sentence summarization","authors":"Sumit Alexander M Rush, Jason Chopra, Weston","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shao et al., 2015] Weixiang Shao, Lifang He, and Philip S Yu. Multiple incomplete views clustering via weighted nonnegative matrix factorization with l 2,1 regularization. In ECML PKDD, 2015.","title":"Multiple incomplete views clustering via weighted nonnegative matrix factorization with l 2,1 regularization","authors":"Weixiang Shao, Lifang He, Philip S Yu","year":"2015","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tang and Liu, 2022] Huayi Tang and Yong Liu. Deep safe incomplete multi-view clustering: Theorem and algo- rithm. In ICML, 2022.","title":"Deep safe incomplete multi-view clustering: Theorem and algorithm","authors":"Liu ; ] Huayi Tang, Yong Tang, Liu","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tao et al., 2017] Zhiqiang Tao, Hongfu Liu, Sheng Li, Zhengming Ding, and Yun Fu. From ensemble clustering to multi-view clustering. In IJCAI, 2017.","title":"From ensemble clustering to multi-view clustering","authors":"Zhiqiang Tao, Hongfu Liu, Sheng Li, Zhengming Ding, Yun Fu","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/396"},{"position":23,"raw":"[Van Gansbeke et al., 2020] Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool. Scan: Learning to classify images without labels. In ECCV, 2020.","title":"Scan: Learning to classify images without labels","authors":"Van Gansbeke","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2015] Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multi-view represen- tation learning. In ICML, 2015.","title":"On deep multi-view representation learning","authors":"Wang","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2018] Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, and Yun Fu. Partial multi-view clustering via consistent gan. In ICDM, 2018.","title":"Partial multi-view clustering via consistent gan","authors":"Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, Yun Fu","year":"2018","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2019] Hao Wang, Linlin Zong, Bing Liu, Yan Yang, and Wei Zhou. Spectral perturbation meets incom- plete multi-view data. arXiv:1906.00098, 2019.","title":"Spectral perturbation meets incomplete multi-view data","authors":"Hao Wang, Linlin Zong, Bing Liu, Yan Yang, Wei Zhou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2019/510"},{"position":28,"raw":"[Wen et al., 2019] Jie Wen, Zheng Zhang, Yong Xu, Bob Zhang, Lunke Fei, and Hong Liu. Unified embedding alignment with missing views inferring for incomplete multi-view clustering. In AAAI, 2019.","title":"Unified embedding alignment with missing views inferring for incomplete multi-view clustering","authors":"Jie Wen, Zheng Zhang, Yong Xu, Bob Zhang, Lunke Fei, Hong Liu","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Woo et al., 2018] Sanghyun Woo, Jongchan Park, Joon- Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In ECCV, 2018.","title":"Cbam: Convolutional block attention module","authors":"Sanghyun Woo, Jongchan Park, Joon-Young Lee, In So Kweon","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yan et al., 2022] Shen Yan, Xuehan Xiong, Anurag Arnab, Zhichao Lu, Mi Zhang, Chen Sun, and Cordelia Schmid. Multiview transformers for video recognition. In CVPR, 2022.","title":"Multiview transformers for video recognition","authors":"Xuehan Shen Yan, Anurag Xiong, Zhichao Arnab, Mi Lu, Chen Zhang, Cordelia Sun, Schmid","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2021a] Mouxing Yang, Yunfan Li, Zhenyu Huang, Zitao Liu, Peng Hu, and Xi Peng. Partially view- aligned representation learning with noise-robust con- trastive loss. In CVPR, 2021.","title":"Partially viewaligned representation learning with noise-robust contrastive loss","authors":"Yang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2021b] Xu Yang, Cheng Deng, Zhiyuan Dang, and Dacheng Tao. Deep multiview collaborative cluster- ing. IEEE Transactions on Neural Networks and Learning Systems, 2021.","title":"Deep multiview collaborative clustering","authors":"Yang","year":"2021","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yang et al., 2022a] Mouxing Yang, Zhenyu Huang, Peng Hu, Taihao Li, Jiancheng Lv, and Xi Peng. Learn- ing with twin noisy labels for visible-infrared person re- identification. In CVPR, 2022.","title":"Learning with twin noisy labels for visible-infrared person reidentification","authors":"Yang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2022b] Mouxing Yang, Yunfan Li, Peng Hu, Jinfeng Bai, Jian Cheng Lv, and Xi Peng. Robust multi- view clustering with incomplete information. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2022.","title":"Robust multiview clustering with incomplete information","authors":"Yang","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yu et al., 2018] Zhou Yu, Jun Yu, Chenchao Xiang, Zhou Zhao, Qi Tian, and Dacheng Tao. Rethinking diversified and discriminative proposal generation for visual ground- ing. arXiv:1805.03508, 2018.","title":"Rethinking diversified and discriminative proposal generation for visual grounding","authors":"Zhou Yu, Jun Yu, Chenchao Xiang, Zhou Zhao, Qi Tian, Dacheng Tao","year":"2018","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2018/155"},{"position":36,"raw":"[Zhang et al., 2018] Zheng Zhang, Li Liu, Fumin Shen, Heng Tao Shen, and Ling Shao. Binary multi-view cluster- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018.","title":"Binary multi-view clustering","authors":"Zheng Zhang, Li Liu, Fumin Shen, Heng Tao Shen, Ling Shao","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2019a] Changqing Zhang, Zongbo Han, Huazhu Fu, Joey Tianyi Zhou, Qinghua Hu, et al. Cpm-nets: Cross partial multi-view networks. NIPS, 2019.","title":"Cpm-nets: Cross partial multi-view networks","authors":"Zhang","year":"2019","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhang et al., 2019b] Changqing Zhang, Yeqing Liu, and Huazhu Fu. Ae2-nets: Autoencoder in autoencoder net- works. In CVPR, 2019.","title":"Ae2-nets: Autoencoder in autoencoder networks","authors":"Zhang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhang et al., 2020] Changqing Zhang, Yajie Cui, Zongbo Han, Joey Tianyi Zhou, Huazhu Fu, and Qinghua Hu. Deep partial multi-view learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.","title":"Deep partial multi-view learning","authors":"Changqing Zhang, Yajie Cui, Zongbo Han, Joey Tianyi Zhou, Huazhu Fu, Qinghua Hu","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhao et al., 2016] Handong Zhao, Hongfu Liu, and Yun Fu. Incomplete multi-modal visual data grouping. In IJCAI, 2016.","title":"Incomplete multi-modal visual data grouping","authors":"Handong Zhao, Hongfu Liu, Yun Fu","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/341"},{"position":41,"raw":"[Zhou and Shao, 2018] Yi Zhou and Ling Shao. Aware at- tentive multi-view inference for vehicle re-identification. In CVPR, 2018.","title":"Aware attentive multi-view inference for vehicle re-identification","authors":"Yi Zhou, Ling Shao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}