{"type":"ScholarlyArticle","title":"Hypergraph Self-supervised Learning with Sampling-efficient Signals","authors":[{"name":"Fan Li","slug":"fan-li","given":"Fan","family":"Li","affiliation":"University of New South Wales , Sydney , Australia"},{"name":"Xiaoyang Wang","slug":"xiaoyang-wang","given":"Xiaoyang","family":"Wang","affiliation":"University of New South Wales , Sydney , Australia"},{"name":"Dawei Cheng","slug":"dawei-cheng","given":"Dawei","family":"Cheng","affiliation":"Tongji University , Shanghai , China"},{"name":"Wenjie Zhang","slug":"wenjie-zhang","given":"Wenjie","family":"Zhang","affiliation":"University of New South Wales , Sydney , Australia"},{"name":"Ying Zhang","slug":"ying-zhang","given":"Ying","family":"Zhang","affiliation":"University of Technology Sydney , Sydney , Australia"},{"name":"Xuemin Lin","slug":"xuemin-lin","given":"Xuemin","family":"Lin","affiliation":"Shanghai Jiaotong University , Shanghai , China"}],"abstract":"Self-supervised learning (SSL) provides a promising alternative for representation learning on hypergraphs without costly labels. However, existing hypergraph SSL models are mostly based on contrastive methods with the instance-level discrimination strategy, suffering from two significant limitations: (1) They select negative samples arbitrarily, which is unreliable in deciding similar and dissimilar pairs, causing training bias. (2) They often require a large number of negative samples, resulting in expensive computational costs. To address the above issues, we propose SE-HSSL, a hypergraph SSL framework with three sampling-efficient self-supervised signals. Specifically, we introduce two sampling-free objectives leveraging the canonical correlation analysis as the node-level and group-level self-supervised signals. Additionally, we develop a novel hierarchical membership-level contrast objective motivated by the cascading overlap relationship in hypergraphs, which can further reduce membership sampling bias and improve the efficiency of sample utilization. Through comprehensive experiments on 7 real-world hypergraphs, we demonstrate the superiority of our approach over the state-of-the-art method in terms of both effectiveness and efficiency.","doi":"10.24963/ijcai.2024/486","url":"https://ijcai.org/proceedings/2024/486","pdf":"https://ijcai.org/proceedings/2024/0486.pdf","markdown":"https://ijcai.org/proceedings/2024/486.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/486","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":486,"pages":{"first":4398,"last":4406},"keywords":["Data Mining: DM: Mining graphs","Machine Learning: ML: Self-supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In International conference on machine learn- ing, pages 1247-1255. PMLR, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Jeff Bilmes, Karen Livescu","year":"2013","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Antelmi et al., 2023] Alessia Antelmi, Gennaro Cordasco, Mirko Polato, Vittorio Scarano, Carmine Spagnuolo, and Dingqi Yang. A survey on hypergraph representation learning. ACM Computing Surveys, 56(1):1-38, 2023.","title":"A survey on hypergraph representation learning","authors":"Alessia Antelmi, Gennaro Cordasco, Mirko Polato, Vittorio Scarano, Carmine Spagnuolo, Dingqi Yang","year":"2023","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2022] Derun Cai, Moxian Song, Chenxi Sun, Baofeng Zhang, Shenda Hong, and Hongyan Li. Hyper- graph structure learning for hypergraph neural networks. In Proceedings of the Thirty-First International Joint Con- ference on Artificial Intelligence, IJCAI-22, pages 1923- 1929, 2022.","title":"Hypergraph structure learning for hypergraph neural networks","authors":"Cai","year":"2022","venue":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22","doi":null,"url":null,"internal_url":"/proceedings/2022/267"},{"position":4,"raw":"[Chang et al., 2018] Xiaobin Chang, Tao Xiang, and Timo- thy M Hospedales. Scalable and effective deep cca via soft decorrelation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1488- 1497, 2018.","title":"Scalable and effective deep cca via soft decorrelation","authors":"Xiaobin Chang, Tao Xiang, Timothy M Hospedales","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chien et al., 2021] Eli Chien, Chao Pan, Jianhao Peng, and Olgica Milenkovic. You are allset: A multiset function framework for hypergraph neural networks. In Interna- tional Conference on Learning Representations, 2021.","title":"You are allset: A multiset function framework for hypergraph neural networks","authors":"Eli Chien, Chao Pan, Jianhao Peng, Olgica Milenkovic","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feng et al., 2019] Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural net- works. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 3558-3565, 2019.","title":"Hypergraph neural networks","authors":"Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, Yue Gao","year":"2019","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross Girshick. Momentum contrast for unsu- pervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-9738, 2020.","title":"Momentum contrast for unsupervised visual representation learning","authors":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick","year":"2020","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hein et al., 2013] Matthias Hein, Simon Setzer, Leonardo Jost, and Syama Sundar Rangapuram. The total vari- ation on hypergraphs-learning on hypergraphs revisited. Advances in Neural Information Processing Systems, 26, 2013.","title":"The total variation on hypergraphs-learning on hypergraphs revisited","authors":"Matthias Hein, Simon Setzer, Leonardo Jost, Syama Sundar, Rangapuram","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hua et al., 2021] Tianyu Hua, Wenxiao Wang, Zihui Xue, Sucheng Ren, Yue Wang, and Hang Zhao. On feature decorrelation in self-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9598-9608, 2021.","title":"On feature decorrelation in self-supervised learning","authors":"Tianyu Hua, Wenxiao Wang, Zihui Xue, Sucheng Ren, Yue Wang, Hang Zhao","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jia et al., 2021] Renqi Jia, Xiaofei Zhou, Linhua Dong, and Shirui Pan. Hypergraph convolutional network for group recommendation. In 2021 IEEE International Conference on Data Mining (ICDM), pages 260-269. IEEE, 2021.","title":"Hypergraph convolutional network for group recommendation","authors":"Jia","year":"2021","venue":"2021 IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jing et al., 2014] Xiao-Yuan Jing, Rui-Min Hu, Yang-Ping Zhu, Shan-Shan Wu, Chao Liang, and Jing-Yu Yang. Intra- view and inter-view supervised correlation analysis for multi-view feature learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.","title":"Intraview and inter-view supervised correlation analysis for multi-view feature learning","authors":"Xiao-Yuan Jing, Rui-Min Hu, Yang-Ping Zhu, Shan-Shan Wu, Chao Liang, Jing-Yu Yang","year":"2014","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim et al., 2020] Eun-Sol Kim, Woo Young Kang, Kyoung-Woon On, Yu-Jung Heo, and Byoung-Tak Zhang. Hypergraph attention networks for multimodal learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14581-14590, 2020.","title":"Hypergraph attention networks for multimodal learning","authors":"Eun-Sol Kim, Young Woo, Kyoung-Woon Kang, Yu-Jung On, Byoung-Tak Heo, Zhang","year":"2020","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lee and Shin, 2023] Dongjin Lee and Kijung Shin. I'm me, we're us, and i'm us: Tri-directional contrastive learning on hypergraphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8456-8464, 2023.","title":"I'm me, we're us, and i'm us: Tri-directional contrastive learning on hypergraphs","authors":"Lee, Shin ;, Dongjin Lee, Kijung Shin","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liang et al., 2023] Yongquan Liang, Qiuyu Song, Zhongy- ing Zhao, Hui Zhou, and Maoguo Gong. Ba- gnn: Behavior-aware graph neural network for session- based recommendation. Frontiers of Computer Science, 17(6):176613, 2023.","title":"Bagnn: Behavior-aware graph neural network for sessionbased recommendation","authors":"Yongquan Liang, Qiuyu Song, Zhongying Zhao, Hui Zhou, Maoguo Gong","year":"2023","venue":"Frontiers of Computer Science","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Miech et al., 2020] Antoine Miech, Jean-Baptiste Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, and Andrew Zis- serman. End-to-end learning of visual representations from uncurated instructional videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 9879-9889, 2020.","title":"End-to-end learning of visual representations from uncurated instructional videos","authors":"Antoine Miech, Jean-Baptiste Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, Andrew Zisserman","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Thakoor et al., 2021] Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Rémi Munos, Petar Veličković, and Michal Valko. Bootstrapped representa- tion learning on graphs. In ICLR 2021 Workshop on Geo- metrical and Topological Representation Learning, 2021.","title":"Bootstrapped representation learning on graphs","authors":"Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Rémi Munos, Petar Veličković, Michal Valko","year":"2021","venue":"ICLR 2021 Workshop on Geometrical and Topological Representation Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Veličković et al., 2018] Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2018.","title":"Deep graph infomax","authors":"Petar Veličković, William Fedus, Pietro William L Hamilton, Yoshua Liò, Devon Bengio, Hjelm","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wei et al., 2022] Tianxin Wei, Yuning You, Tianlong Chen, Yang Shen, Jingrui He, and Zhangyang Wang. Augmenta- tions in hypergraph contrastive learning: Fabricated and generative. Advances in neural information processing systems, 35:1909-1922, 2022.","title":"Augmentations in hypergraph contrastive learning: Fabricated and generative","authors":"Tianxin Wei, Yuning You, Tianlong Chen, Yang Shen, Jingrui He, Zhangyang Wang","year":"2022","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xia et al., 2022a] Lianghao Xia, Chao Huang, Yong Xu, Ji- ashu Zhao, Dawei Yin, and Jimmy Huang. Hypergraph contrastive collaborative filtering. In Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval, pages 70-79, 2022.","title":"Hypergraph contrastive collaborative filtering","authors":"Xia","year":"2022","venue":"Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xia et al., 2022b] Lianghao Xia, Chao Huang, and Chuxu Zhang. Self-supervised hypergraph transformer for rec- ommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2100-2109, 2022.","title":"Self-supervised hypergraph transformer for recommender systems","authors":"Xia","year":"2022","venue":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xiao et al., 2019] Li Xiao, Junqi Wang, Peyman H Kas- sani, Yipu Zhang, Yuntong Bai, Julia M Stephen, Tony W Wilson, Vince D Calhoun, and Yu-Ping Wang. Multi- hypergraph learning-based brain functional connectivity analysis in fmri data. IEEE transactions on medical imag- ing, 39(5):1746-1758, 2019.","title":"Multihypergraph learning-based brain functional connectivity analysis in fmri data","authors":"Li Xiao, Junqi Wang, H Peyman, Yipu Kassani, Yuntong Zhang, Julia M Bai, Tony W Stephen, Vince D Wilson, Yu-Ping Calhoun, Wang","year":"2019","venue":"IEEE transactions on medical imaging","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yadati et al., 2019] Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergcn: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019.","title":"Hypergcn: A new method for training graph convolutional networks on hypergraphs","authors":"Yadati","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2021] Haoran Yang, Hongxu Chen, Lin Li, S Yu Philip, and Guandong Xu. Hyper meta-path contrastive learning for multi-behavior recommendation. In 2021 IEEE International Conference on Data Mining (ICDM), pages 787-796. IEEE, 2021.","title":"Hyper meta-path contrastive learning for multi-behavior recommendation","authors":"Haoran Yang, Hongxu Chen, Lin Li, Yu Philip, Guandong Xu","year":"2021","venue":"2021 IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2022] Chaoqi Yang, Ruijie Wang, Shuochao Yao, and Tarek Abdelzaher. Semi-supervised hypergraph node classification on hypergraph line expansion. In Pro- ceedings of the 31st ACM International Conference on In- formation & Knowledge Management, pages 2352-2361, 2022.","title":"Semi-supervised hypergraph node classification on hypergraph line expansion","authors":"Yang","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yi and Park, 2020] Jaehyuk Yi and Jinkyoo Park. Hyper- graph convolutional recurrent neural network. In Proceed- ings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 3366-3376, 2020.","title":"Jaehyuk Yi and Jinkyoo Park. Hypergraph convolutional recurrent neural network","authors":"Park Yi","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph con- trastive learning with augmentations. Advances in neural information processing systems, 33:5812-5823, 2020.","title":"Graph contrastive learning with augmentations","authors":"Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2012] Jun Yu, Dacheng Tao, and Meng Wang. Adaptive hypergraph learning and its application in image classification. IEEE Transactions on Image Processing, 21(7):3262-3272, 2012.","title":"Adaptive hypergraph learning and its application in image classification","authors":"Yu","year":"2012","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2021a] Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf, and Philip S Yu. From canonical correla- tion analysis to self-supervised graph neural networks. Ad- vances in Neural Information Processing Systems, 34:76- 89, 2021.","title":"From canonical correlation analysis to self-supervised graph neural networks","authors":"Zhang","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2021b] Junwei Zhang, Min Gao, Junliang Yu, Lei Guo, Jundong Li, and Hongzhi Yin. Double-scale self-supervised hypergraph learning for group recommen- dation. In Proceedings of the 30th ACM international con- ference on information & knowledge management, pages 2557-2567, 2021.","title":"Double-scale self-supervised hypergraph learning for group recommendation","authors":"Zhang","year":"2021","venue":"Proceedings of the 30th ACM international conference on information & knowledge management","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2022] Yifei Zhang, Hao Zhu, Zixing Song, Pi- otr Koniusz, and Irwin King. Costa: covariance-preserving feature augmentation for graph contrastive learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2524- 2534, 2022.","title":"Costa: covariance-preserving feature augmentation for graph contrastive learning","authors":"Yifei Zhang, Hao Zhu, Zixing Song, Piotr Koniusz, Irwin King","year":"2022","venue":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2023] Yifei Zhang, Hao Zhu, Zixing Song, Pi- otr Koniusz, and Irwin King. Spectral feature augmen- tation for graph contrastive learning and beyond. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11289-11297, 2023.","title":"Spectral feature augmen-tation for graph contrastive learning and beyond","authors":"Yifei Zhang, Hao Zhu, Zixing Song, Piotr Koniusz, Irwin King","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhao et al., 2021] Han Zhao, Xu Yang, Zhenru Wang, Erkun Yang, and Cheng Deng. Graph debiased contrastive learning with joint representation clustering. In IJCAI, pages 3434-3440, 2021.","title":"Graph debiased contrastive learning with joint representation clustering","authors":"Han Zhao, Xu Yang, Zhenru Wang, Erkun Yang, Cheng Deng","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/473"},{"position":33,"raw":"[Zhou et al., 2006] Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Clus- tering, classification, and embedding. Advances in neural information processing systems, 19, 2006.","title":"Learning with hypergraphs: Clustering, classification, and embedding","authors":"Zhou","year":"2006","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhu et al., 2020] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131, 2020.","title":"Deep graph contrastive representation learning","authors":"Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, Liang Wang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhu et al., 2021] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, pages 2069-2080, 2021.","title":"Graph contrastive learning with adaptive augmentation","authors":"Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, Liang Wang","year":"2021","venue":"Proceedings of the Web Conference 2021","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}