{"type":"ScholarlyArticle","title":"Hypergraph Structure Learning for Hypergraph Neural Networks","authors":[{"name":"Derun Cai","slug":"derun-cai","given":"Derun","family":"Cai","affiliation":"Key Laboratory of Machine Perception (Ministry of Education) , Peking University , Beijing , China.; School of Electronics Engineering and Computer Science , Peking University , Beijing , China."},{"name":"Moxian Song","slug":"moxian-song","given":"Moxian","family":"Song","affiliation":"Key Laboratory of Machine Perception (Ministry of Education) , Peking University , Beijing , China.; School of Electronics Engineering and Computer Science , Peking University , Beijing , China."},{"name":"Chenxi Sun","slug":"chenxi-sun","given":"Chenxi","family":"Sun","affiliation":"Key Laboratory of Machine Perception (Ministry of Education) , Peking University , Beijing , China.; School of Electronics Engineering and Computer Science , Peking University , Beijing , China."},{"name":"Baofeng Zhang","slug":"baofeng-zhang","given":"Baofeng","family":"Zhang","affiliation":"Key Laboratory of Machine Perception (Ministry of Education) , Peking University , Beijing , China.; School of Electronics Engineering and Computer Science , Peking University , Beijing , China."},{"name":"Shenda Hong","slug":"shenda-hong","given":"Shenda","family":"Hong","affiliation":"National Institute of Health Data Science , Peking University , Beijing , China.; Institute of Medical Technology , Health Science Center of Peking University , Beijing , China."},{"name":"Hongyan Li","slug":"hongyan-li","given":"Hongyan","family":"Li","affiliation":"Key Laboratory of Machine Perception (Ministry of Education) , Peking University , Beijing , China.; School of Electronics Engineering and Computer Science , Peking University , Beijing , China."}],"abstract":"Hypergraphs are natural and expressive modeling tools to encode high-order relationships among entities. Several variations of Hypergraph Neural Networks (HGNNs) are proposed to learn the node representations and complex relationships in the hypergraphs. Most current approaches assume that the input hypergraph structure accurately depicts the relations in the hypergraphs. However, the input hypergraph structure inevitably contains noise, task-irrelevant information, or false-negative connections.\r\nTreating the input hypergraph structure as ground-truth information unavoidably leads to sub-optimal performance. In this paper, we propose a Hypergraph Structure Learning (HSL) framework, which optimizes the hypergraph structure and the HGNNs simultaneously in an end-to-end way. \r\nHSL learns an informative and concise hypergraph structure that is optimized for downstream tasks. To efficiently learn the hypergraph structure, HSL adopts a two-stage sampling process: hyperedge sampling for pruning redundant hyperedges and incident node sampling for pruning irrelevant incident nodes and discovering potential implicit connections. \r\nThe consistency between the optimized structure and the original structure is maintained by the intra-hyperedge contrastive learning module.\r\nThe sampling processes are jointly optimized with HGNNs towards the objective of the downstream tasks. Experiments conducted on 7 datasets show shat HSL outperforms the state-of-the-art baselines while adaptively sparsifying hypergraph structures.","doi":"10.24963/ijcai.2022/267","url":"https://ijcai.org/proceedings/2022/267","pdf":"https://ijcai.org/proceedings/2022/0267.pdf","markdown":"https://ijcai.org/proceedings/2022/267.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/267","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":267,"pages":{"first":1923,"last":1929},"keywords":["Data Mining: Mining Graphs","Machine Learning: Geometric Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986889","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985032","length_sec":327,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bai et al., 2021] Song Bai, Feihu Zhang, and Philip H. S. Torr. Hypergraph convolution and hypergraph attention. Pattern Recognit., 110:107637, 2021.","title":"Hypergraph convolution and hypergraph attention","authors":"Song Bai, Feihu Zhang, Philip H S Torr","year":"2021","venue":"Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2003] Ding-Yun Chen, Xiao-Pei Tian, Edward Yu-Te Shen, and Ming Ouhyoung. On visual similarity based 3d model retrieval. Computer Graphics Forum, 22, 2003.","title":"On visual similarity based 3d model retrieval","authors":"Ding-Yun Chen, Xiao-Pei Tian, Edward Yu-Te Shen, Ming Ouhyoung","year":"2003","venue":"Computer Graphics Forum","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2020a] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey E. Hinton. A simple frame- work for contrastive learning of visual representations. ArXiv, abs/2002.05709, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020b] Yu Chen, Lingfei Wu, and Mo- hammed J. Zaki. Iterative deep graph learning for graph neural networks: Better and robust node embeddings. ArXiv, abs/2006.13009, 2020.","title":"Iterative deep graph learning for graph neural networks: Better and robust node embeddings","authors":"Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chien et al., 2021] Eli Chien, Chao Pan, Jianhao Peng, and Olgica Milenkovic. You are allset: A multiset func- tion framework for hypergraph neural networks. ArXiv, abs/2106.13264, 2021.","title":"You are allset: A multiset function framework for hypergraph neural networks","authors":"Eli Chien, Chao Pan, Jianhao Peng, Olgica Milenkovic","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feng et al., 2019] Yifan Feng, Haoxuan You, Zizhao Zhang, R. Ji, and Yue Gao. Hypergraph neural networks. ArXiv, abs/1809.09401, 2019.","title":"Hypergraph neural networks","authors":"Yifan Feng, Haoxuan You, Zizhao Zhang, R Ji, Yue Gao","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fey and Lenssen, 2019] Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. ArXiv, abs/1903.02428, 2019.","title":"Fast graph representation learning with pytorch geometric","authors":"Fey, Lenssen, Matthias Fey, Jan Eric Lenssen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Franceschi et al., 2019] Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. Learning discrete structures for graph neural networks. In ICML, 2019.","title":"Learning discrete structures for graph neural networks","authors":"Luca Franceschi, Mathias Niepert, Massimiliano Pontil, Xiao He","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao et al., 2020] Yue Gao, Zizhao Zhang, Haojie Lin, Xibin Zhao, S. Du, and Changqing Zou. Hypergraph learning: Methods and practices. IEEE transactions on pattern analysis and machine intelligence, PP, 2020.","title":"Hypergraph learning: Methods and practices","authors":"Yue Gao, Zizhao Zhang, Haojie Lin, Xibin Zhao, S Du, Changqing Zou","year":"2020","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gutmann and Hyvärinen, 2010] Michael U. Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new es- timation principle for unnormalized statistical models. In AISTATS, 2010.","title":"Noise-contrastive estimation: A new estimation principle for unnormalized statistical models","authors":"U Michael, Aapo Gutmann, Hyvärinen","year":"2010","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hamilton et al., 2017] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, 2017.","title":"Inductive representation learning on large graphs","authors":"William L Hamilton, Zhitao Ying, Jure Leskovec","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang and Yang, 2021] Jing Huang and Jie Yang. Unignn: a unified framework for graph and hypergraph neural net- works. In IJCAI, 2021.","title":"Unignn: a unified framework for graph and hypergraph neural networks","authors":"Jing Huang, Jie Yang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/353"},{"position":13,"raw":"[Jang et al., 2017] Eric Jang, Shixiang Shane Gu, and Ben Poole. Categorical reparameterization with gumbel- softmax. ArXiv, abs/1611.01144, 2017.","title":"Categorical reparameterization with gumbelsoftmax","authors":"Eric Jang, Shixiang Shane Gu, Ben Poole","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jiang et al., 2019] Jianwen Jiang, Yuxuan Wei, Yifan Feng, Jingxuan Cao, and Yue Gao. Dynamic hypergraph neural networks. In IJCAI, 2019.","title":"Dynamic hypergraph neural networks","authors":"Jianwen Jiang, Yuxuan Wei, Yifan Feng, Jingxuan Cao, Yue Gao","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/366"},{"position":15,"raw":"[Kipf and Welling, 2017] Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ArXiv, abs/1609.02907, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas Kipf, Max Welling","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2018] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, 2018.","title":"Deeper insights into graph convolutional networks for semi-supervised learning","authors":"Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Luo et al., 2021] Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, and Xiang Zhang. Learning to drop: Robust graph neural network via topological denoising. Proceedings of the 14th ACM International Conference on Web Search and Data Min- ing, 2021.","title":"Learning to drop: Robust graph neural network via topological denoising","authors":"Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, Xiang Zhang","year":"2021","venue":"Proceedings of the 14th ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Maddison et al., 2017] Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A con- tinuous relaxation of discrete random variables. ArXiv, abs/1611.00712, 2017.","title":"The concrete distribution: A continuous relaxation of discrete random variables","authors":"Chris J Maddison, Andriy Mnih, Yee Whye Teh","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rossi and Ahmed, 2015] Ryan A. Rossi and Nesreen Ahmed. The network data repository with interactive graph analytics and visualization. In AAAI, 2015.","title":"The network data repository with interactive graph analytics and visualization","authors":"Ryan A Rossi, Nesreen Ahmed","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, and Tina Eliassi- Rad. Collective classification in network data. AI Mag., 29:93-106, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, Tina Eliassi-Rad","year":"2008","venue":"AI Mag","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Velickovic et al., 2018] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio', and Yoshua Bengio. Graph attention networks. ArXiv, abs/1710.10903, 2018.","title":"Graph attention networks","authors":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, ', Yoshua Bengio","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yadati et al., 2019] Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Pratim Talukdar. Hypergcn: A new method for training graph convolutional networks on hypergraphs. In NeurIPS, 2019.","title":"Hypergcn: A new method for training graph convolutional networks on hypergraphs","authors":"Yadati","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu et al., 2021] Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoc Viet Hung, and Xiangliang Zhang. Self-supervised multi-channel hypergraph convo- lutional network for social recommendation. Proceedings of the Web Conference 2021, 2021.","title":"Self-supervised multi-channel hypergraph convolutional network for social recommendation","authors":"Yu","year":"2021","venue":"Proceedings of the Web Conference 2021","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2018] Zizhao Zhang, Haojie Lin, and Yue Gao. Dynamic hypergraph structure learning. In IJCAI, 2018.","title":"Dynamic hypergraph structure learning","authors":"Zhang","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/439"},{"position":25,"raw":"[Zhang et al., 2021] Jiying Zhang, Yuzhao Chen, Xiong Xiao, Runiu Lu, and Shutao Xia. Learnable hypergraph laplacian for hypergraph learning. ArXiv, abs/2106.06666, 2021.","title":"Learnable hypergraph laplacian for hypergraph learning","authors":"Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zheng et al., 2020] Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. Robust graph representation learning via neural sparsification. In ICML, 2020.","title":"Robust graph representation learning via neural sparsification","authors":"Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, Wei Wang","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2006] Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Clus- tering, classification, and embedding. In NIPS, 2006.","title":"Learning with hypergraphs: Clustering, classification, and embedding","authors":"Zhou","year":"2006","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu et al., 2021] Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. Deep graph struc- ture learning for robust representations: A survey. ArXiv, abs/2103.03036, 2021.","title":"Deep graph structure learning for robust representations: A survey","authors":"Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, Liang Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}