{"type":"ScholarlyArticle","title":"When Do GNNs Work: Understanding and Improving Neighborhood Aggregation","authors":[{"name":"Yiqing Xie","slug":"yiqing-xie","given":"Yiqing","family":"Xie","affiliation":"University of Illinois at Urbana-Champaign , IL , USA; The Hong Kong University of Science and Technology , Hong Kong , China"},{"name":"Sha Li","slug":"sha-li","given":"Sha","family":"Li","affiliation":"University of Illinois at Urbana-Champaign , IL , USA"},{"name":"Carl Yang","slug":"carl-yang","given":"Carl","family":"Yang","affiliation":"Emory University , GA , USA"},{"name":"Raymond Chi-Wing Wong","slug":"raymond-chi-wing-wong","given":"Raymond Chi-Wing","family":"Wong","affiliation":"The Hong Kong University of Science and Technology , Hong Kong , China"},{"name":"Jiawei Han","slug":"jiawei-han","given":"Jiawei","family":"Han","affiliation":"University of Illinois at Urbana-Champaign , IL , USA"}],"abstract":"Graph Neural Networks (GNNs) have been shown to be  powerful in a wide range of graph-related tasks. While there exists various GNN models, a critical common ingredient is neighborhood aggregation, where the embedding of each node is updated by referring to the embedding of its neighbors. This paper aims to provide a better understanding of this mechanisms by asking the following question: Is neighborhood aggregation always necessary and beneficial? In short, the answer is no. We carve out two conditions under which neighborhood aggregation is not helpful: (1) when a node's neighbors are highly dissimilar and (2) when a node's embedding is already similar with that of its neighbors. We propose novel metrics that quantitatively measure these two circumstances and integrate them into an Adaptive-layer module. Our experiments show that allowing for node-specific aggregation degrees have significant advantage over current GNNs.","doi":"10.24963/ijcai.2020/181","url":"https://ijcai.org/proceedings/2020/181","pdf":"https://ijcai.org/proceedings/2020/0181.pdf","markdown":"https://ijcai.org/proceedings/2020/181.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/181","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Data Mining","track":"Main track","number":181,"pages":{"first":1303,"last":1309},"keywords":["Data Mining: Mining Graphs, Semi Structured Data, Complex Data","Machine Learning: Semi-Supervised Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25414/file_25414.m3u8","length_sec":274,"provider":"hls"}],"references":[{"position":1,"raw":"[Chen et al., 2020] Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. Measuring and relieving the over- smoothing problem for graph neural networks from the topological view. AAAI, 2020.","title":"Measuring and relieving the oversmoothing problem for graph neural networks from the topological view","authors":"Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, Xu Sun","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Hamilton et al., 2017] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, 2017.","title":"Inductive representation learning on large graphs","authors":"William L Hamilton, Zhitao Ying, Jure Leskovec","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Hou et al., 2020] Yifan Hou, Jian Zhang, James Cheng, Kaili Ma, Richard T. B. Ma, Hongzhi Chen, and Ming- Chang Yang. Measuring and improving the use of graph information in graph neural networks. In ICLR, 2020.","title":"Measuring and improving the use of graph information in graph neural networks","authors":"Yifan Hou, Jian Zhang, James Cheng, Kaili Ma, T B Richard, Hongzhi Ma, Ming-Chang Chen, Yang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Kipf and Welling, 2016] Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ICLR, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas Kipf, Max Welling","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Klicpera et al., 2019] Johannes Klicpera, Aleksandar Bo- jchevski, and Stephan Günnemann. Combining neural networks with personalized pagerank for classification on graphs. In ICLR, 2019.","title":"Combining neural networks with personalized pagerank for classification on graphs","authors":"Klicpera","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Li et al., 2018] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, 2018.","title":"Deeper insights into graph convolutional networks for semi-supervised learning","authors":"Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Namata et al., 2012] Galileo Mark Namata, Ben London, Lise Getoor, and Bert Huang. Query-driven active survey- ing for collective classification. In Workshop on Mining and Learning with Graphs, 2012.","title":"Query-driven active surveying for collective classification","authors":"Namata","year":"2012","venue":"Workshop on Mining and Learning with Graphs","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: online learning of social rep- resentations. KDD, 2014.","title":"Deepwalk: online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, and Tina Eliassi- Rad. Collective classification in network data. In AI mag- azine, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, Tina Eliassi-Rad","year":"2008","venue":"AI magazine","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Velickovic et al., 2017] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. ICLR, 2017.","title":"Graph attention networks","authors":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Wang et al., 2019] Minjie Wang, Lingfan Yu, Da Zheng, Quan Gan, Yu Gai, Zihao Ye, Mufei Li, Jinjing Zhou, Qi Huang, Chao Ma, Ziyue Huang, Qipeng Guo, Hao Zhang, Haibin Lin, Junbo Zhao, Jinyang Li, Alexander Smola, and Zheng Zhang. Deep graph library: Towards efficient and scalable deep learning on graphs, 2019.","title":"Deep graph library: Towards efficient and scalable deep learning on graphs","authors":"Minjie Wang, Lingfan Yu, Da Zheng, Quan Gan, Yu Gai, Zihao Ye, Mufei Li, Jinjing Zhou, Qi Huang, Chao Ma, Ziyue Huang, Qipeng Guo, Hao Zhang, Haibin Lin, Junbo Zhao, Jinyang Li, Alexander Smola, Zheng Zhang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Wu et al., 2012] Xiaoming Wu, Zhenguo Li, Anthony M. So, John Wright, and Shihfu Chang. Learning with par- tially absorbing random walks. In NIPS, pages 3077-3085. 2012.","title":"Learning with partially absorbing random walks","authors":"Xiaoming Wu, Zhenguo Li, Anthony M So, John Wright, Shihfu Chang","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Wu et al., 2019] Felix Wu, Tianyi Zhang, Amauri H. de Souza, Christopher Fifty, Tao Yu, and Kilian Q. Wein- berger. Simplifying graph convolutional networks. In ICML, 2019.","title":"Simplifying graph convolutional networks","authors":"Felix Wu, Tianyi Zhang, H Amauri, Christopher De Souza, Tao Fifty, Kilian Q Yu, Weinberger","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Yang et al., 2016] Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learn- ing with graph embeddings. In ICML, 2016.","title":"Revisiting semi-supervised learning with graph embeddings","authors":"Zhilin Yang, William W Cohen, Ruslan Salakhutdinov","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Zachary, 1977] Wayne W. Zachary. An information flow model for conflict and fission in small groups. 1977.","title":"An information flow model for conflict and fission in small groups","authors":"Wayne W Zachary","year":"1977","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Zhang and Meng, 2020] Jiawei Zhang and Lin Meng. Gres- net: Graph residual network for reviving deep gnns from suspended animation, 2020.","title":"Gresnet: Graph residual network for reviving deep gnns from suspended animation","authors":"Meng Zhang, Jiawei Zhang, Lin Meng","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Zhao and Akoglu, 2020] Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. In ICLR, 2020.","title":"Pairnorm: Tackling oversmoothing in gnns","authors":"Akoglu Zhao, Lingxiao Zhao, Leman Akoglu","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhou et al., 2018] Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Graph neural networks: A review of methods and applications. ArXiv, abs/1812.08434, 2018.","title":"Graph neural networks: A review of methods and applications","authors":"Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Maosong Sun","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}