{"type":"ScholarlyArticle","title":"Ensemble Multi-Relational Graph Neural Networks","authors":[{"name":"Yuling Wang","slug":"yuling-wang","given":"Yuling","family":"Wang","affiliation":"Beijing University of Posts and Telecommunications; Meituan"},{"name":"Hao Xu","slug":"hao-xu","given":"Hao","family":"Xu","affiliation":"Meituan"},{"name":"Yanhua Yu","slug":"yanhua-yu","given":"Yanhua","family":"Yu","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Mengdi Zhang","slug":"mengdi-zhang","given":"Mengdi","family":"Zhang","affiliation":"Meituan"},{"name":"Zhenhao Li","slug":"zhenhao-li","given":"Zhenhao","family":"Li","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Yuji Yang","slug":"yuji-yang","given":"Yuji","family":"Yang","affiliation":"Meituan"},{"name":"Wei Wu","slug":"wei-wu","given":"Wei","family":"Wu","affiliation":"Meituan"}],"abstract":"It is well established that graph neural networks (GNNs) can be interpreted and designed from the perspective of optimization objective. With this clear optimization objective, the deduced GNNs architecture has sound theoretical foundation, which is able to flexibly remedy the weakness of GNNs. However, this optimization objective is only proved for GNNs with single-relational graph. Can we infer a new type of GNNs for multi-relational graphs by extending this optimization objective, so as to simultaneously solve the issues in previous multi-relational GNNs, e.g., over-parameterization? In this paper, we propose a novel ensemble multi-relational GNNs by designing an ensemble multi-relational (EMR) optimization objective. This EMR optimization objective is able to derive an iterative updating rule, which can be formalized as an ensemble message passing (EnMP) layer with multi-relations. We further analyze the nice properties of EnMP layer, e.g., the relationship with multi-relational personalized PageRank. Finally, a new multi-relational GNNs which well alleviate the over-smoothing and over-parameterization issues are proposed. Extensive experiments conducted on four benchmark datasets well demonstrate the effectiveness of the proposed model.","doi":"10.24963/ijcai.2022/319","url":"https://ijcai.org/proceedings/2022/319","pdf":"https://ijcai.org/proceedings/2022/0319.pdf","markdown":"https://ijcai.org/proceedings/2022/319.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/319","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":319,"pages":{"first":2298,"last":2304},"keywords":["Data Mining: Mining Graphs","Data Mining: Mining Heterogenous Data","Machine Learning: Semi-Supervised Learning","Machine Learning: Representation learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987107","length_sec":78,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985250","length_sec":542,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Beck and Teboulle, 2003] Amir Beck and Marc Teboulle. Mirror descent and nonlinear projected subgradient meth- ods for convex optimization. Operations Research Letters, 31(3):167-175, 2003.","title":"Mirror descent and nonlinear projected subgradient methods for convex optimization","authors":"Amir Beck, Marc Teboulle","year":"2003","venue":"Operations Research Letters","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen and Ye, 2011] Yunmei Chen and Xiaojing Ye. Pro- jection onto a simplex. arXiv preprint arXiv:1101.6081, 2011.","title":"Projection onto a simplex","authors":"Chen, Ye ;, Yunmei Chen, Xiaojing Ye","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres- son, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filter- ing. Advances in neural information processing systems, 29:3844-3852, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fu et al., 2020] Xinyu Fu, Jiani Zhang, Ziqiao Meng, et al. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of The Web Conference 2020, pages 2331-2341, 2020.","title":"Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding","authors":"Xinyu Fu, Jiani Zhang, Ziqiao Meng","year":"2020","venue":"Proceedings of The Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Geng et al., 2012] Bo Geng, Dacheng Tao, Chao Xu, et al. Ensemble manifold regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6):1227- 1233, 2012.","title":"Ensemble manifold regularization","authors":"Bo Geng, Dacheng Tao, Chao Xu","year":"2012","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gilmer et al., 2017] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, et al. Neural message passing for quan- tum chemistry. In International conference on machine learning, pages 1263-1272. PMLR, 2017.","title":"Neural message passing for quantum chemistry","authors":"Justin Gilmer, S Samuel, Patrick F Schoenholz, Riley","year":"2017","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2020] Kexin Huang, Cao Xiao, Lucas Glass, et al. Skipgnn: predicting molecular interactions with skip-graph networks. Scientific reports, 10(1):1-16, 2020.","title":"Skipgnn: predicting molecular interactions with skip-graph networks","authors":"Kexin Huang, Cao Xiao, Lucas Glass","year":"2020","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ji et al., 2021] Houye Ji, Xiao Wang, Chuan Shi, et al. Het- erogeneous graph propagation network. IEEE Transac- tions on Knowledge and Data Engineering, 2021.","title":"Heterogeneous graph propagation network","authors":"Houye Ji, Xiao Wang, Chuan Shi","year":"2021","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Klicpera et al., 2018] Johannes Klicpera, Aleksandar Bo- jchevski, and Stephan Günnemann. Predict then propa- gate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997, 2018.","title":"Predict then propagate: Graph neural networks meet personalized pagerank","authors":"Klicpera","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lee et al., 2013] Sangkeun Lee, Sungchan Park, Minsuk Kahng, et al. Pathrank: Ranking nodes on a heterogeneous graph for flexible hybrid recommender systems. Expert Systems with Applications, 40(2):684-697, 2013.","title":"Pathrank: Ranking nodes on a heterogeneous graph for flexible hybrid recommender systems","authors":"Lee","year":"2013","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2021] Xiaorui Liu, Wei Jin, Yao Ma, et al. Elas- tic graph neural networks. In International Conference on Machine Learning, pages 6837-6849. PMLR, 2021.","title":"Elastic graph neural networks","authors":"Xiaorui Liu, Wei Jin, Yao Ma","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lv et al., 2021] Qingsong Lv, Ming Ding, Qiang Liu, et al. Are we really making much progress? revisiting, bench- marking and refining heterogeneous graph neural net- works. In Proceedings of the 27th ACM SIGKDD Con- ference on Knowledge Discovery & Data Mining, pages 1150-1160, 2021.","title":"Are we really making much progress? revisiting, benchmarking and refining heterogeneous graph neural networks","authors":"Qingsong Lv, Ming Ding, Qiang Liu","year":"2021","venue":"Proceedings of the 27th ACM SIGKDD Con-ference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ma et al., 2021] Yao Ma, Xiaorui Liu, Tong Zhao, et al. A unified view on graph neural networks as graph signal de- noising. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 1202-1211, 2021.","title":"A unified view on graph neural networks as graph signal denoising","authors":"Yao Ma, Xiaorui Liu, Tong Zhao","year":"2021","venue":"Proceedings of the 30th ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Oono and Suzuki, 2019] Kenta Oono and Taiji Suzuki. Graph neural networks exponentially lose expres- sive power for node classification. arXiv preprint arXiv:1905.10947, 2019.","title":"Graph neural networks exponentially lose expressive power for node classification","authors":"Suzuki ; ] Kenta Oono, Taiji Oono, Suzuki","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Schlichtkrull et al., 2018] Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, et al. Modeling relational data with graph convolutional networks. In European semantic web conference, pages 593-607. Springer, 2018.","title":"Modeling relational data with graph convolutional networks","authors":"Michael Schlichtkrull, Thomas N Kipf, Peter Bloem","year":"2018","venue":"European semantic web conference","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Thanapalasingam et al., 2021] Thiviyan Thanapalasingam, Lucas van Berkel, Peter Bloem, and Paul Groth. Rela- tional graph convolutional networks: A closer look. arXiv preprint arXiv:2107.10015, 2021.","title":"Thiviyan Thanapalasingam","authors":"Thanapalasingam","year":"2021","venue":"Relational graph convolutional networks: A closer look","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vashishth et al., 2019] Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition-based multi-relational graph convolutional networks. arXiv preprint arXiv:1911.03082, 2019.","title":"Composition-based multi-relational graph convolutional networks","authors":"Shikhar Vashishth, Soumya Sanyal, Nitin Vikram, Partha Talukdar","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, et al. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2019] Xiao Wang, Houye Ji, Chuan Shi, et al. Heterogeneous graph attention network. In The World Wide Web Conference, pages 2022-2032, 2019.","title":"Heterogeneous graph attention network","authors":"Xiao Wang, Houye Ji, Chuan Shi","year":"2019","venue":"The World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2018] Keyulu Xu, Chengtao Li, Yonglong Tian, et al. Representation learning on graphs with jumping knowledge networks. In International Conference on Ma- chine Learning, pages 5453-5462. PMLR, 2018.","title":"Representation learning on graphs with jumping knowledge networks","authors":"Xu","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2021] Yongyi Yang, Tang Liu, Yangkun Wang, et al. Graph neural networks inspired by classical itera- tive algorithms. In International Conference on Machine Learning, pages 11773-11783. PMLR, 2021.","title":"Graph neural networks inspired by classical iterative algorithms","authors":"Yongyi Yang, Tang Liu, Yangkun Wang","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yu et al., 2021] Donghan Yu, Yiming Yang, Ruohong Zhang, et al. Knowledge embedding based graph convo- lutional network. In Proceedings of the Web Conference 2021, pages 1619-1628, 2021.","title":"Knowledge embedding based graph convolutional network","authors":"Yu","year":"2021","venue":"Proceedings of the Web Conference 2021","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yun et al., 2019] Seongjun Yun, Minbyul Jeong, Raehyun Kim, et al. Graph transformer networks. Advances in Neural Information Processing Systems, 32:11983-11993, 2019.","title":"Graph transformer networks","authors":"Yun","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2021] Meiqi Zhu, Xiao Wang, Chuan Shi, et al. Interpreting and unifying graph neural networks with an optimization framework. In Proceedings of the Web Con- ference 2021, pages 1215-1226, 2021.","title":"Interpreting and unifying graph neural networks with an optimization framework","authors":"Meiqi Zhu, Xiao Wang, Chuan Shi","year":"2021","venue":"Proceedings of the Web Conference 2021","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}