{"type":"ScholarlyArticle","title":"Hierarchical Diffusion Scattering Graph Neural Network","authors":[{"name":"Ke Zhang","slug":"ke-zhang","given":"Ke","family":"Zhang","affiliation":"Jiangsu Provincal Joint International Research Laboratory of Medical Information Processing , School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Xinyan Pu","slug":"xinyan-pu","given":"Xinyan","family":"Pu","affiliation":"Jiangsu Provincal Joint International Research Laboratory of Medical Information Processing , School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Jiaxing Li","slug":"jiaxing-li","given":"Jiaxing","family":"Li","affiliation":"Jiangsu Provincal Joint International Research Laboratory of Medical Information Processing , School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Jiasong Wu","slug":"jiasong-wu","given":"Jiasong","family":"Wu","affiliation":"Jiangsu Provincal Joint International Research Laboratory of Medical Information Processing , School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Huazhong Shu","slug":"huazhong-shu","given":"Huazhong","family":"Shu","affiliation":"Jiangsu Provincal Joint International Research Laboratory of Medical Information Processing , School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Youyong Kong","slug":"youyong-kong","given":"Youyong","family":"Kong","affiliation":"Jiangsu Provincal Joint International Research Laboratory of Medical Information Processing , School of Computer Science and Engineering , Southeast University , Nanjing , China"}],"abstract":"Graph neural network (GNN) is popular now to solve the tasks in non-Euclidean space and most of them learn deep embeddings by aggregating the neighboring nodes. However, these methods are prone to some problems such as over-smoothing because of the single-scale perspective field and the nature of low-pass filter. To address these limitations, we introduce diffusion scattering network (DSN) to exploit high-order patterns. With observing the complementary relationship between multi-layer GNN and DSN, we propose Hierarchical Diffusion Scattering Graph Neural Network (HDS-GNN) to efficiently bridge DSN and GNN layer by layer to supplement GNN with multi-scale information and band-pass signals. Our model extracts node-level scattering representations by intercepting the low-pass filtering, and adaptively tunes the different scales to regularize multi-scale information. Then we apply hierarchical representation enhancement to improve GNN with the scattering features. We benchmark our model on nine real-world networks on the transductive semi-supervised node classification task. The experimental results demonstrate the effectiveness of our method.","doi":"10.24963/ijcai.2022/519","url":"https://ijcai.org/proceedings/2022/519","pdf":"https://ijcai.org/proceedings/2022/0519.pdf","markdown":"https://ijcai.org/proceedings/2022/519.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/519","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":519,"pages":{"first":3737,"last":3743},"keywords":["Machine Learning: Geometric Learning","Machine Learning: Semi-Supervised Learning","Machine Learning: Representation learning","Data Mining: Mining Graphs"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986600","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984743","length_sec":289,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Balcilar et al., 2021a] Muhammet Balcilar, Pierre Héroux, Benoit Gaüzère, Pascal Vasseur, Sébastien Adam, and Paul Honeine. Breaking the limits of message passing graph neural networks. In ICML. PMLR, 2021.","title":"Breaking the limits of message passing graph neural networks","authors":"Balcilar","year":"2021","venue":"ICML. PMLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Balcilar et al., 2021b] Muhammet Balcilar, Guillaume Ren- ton, Pierre Héroux, Benoit Gaüzère, Sébastien Adam, and Paul Honeine. Analyzing the expressive power of graph neural networks in a spectral perspective. In ICLR, 2021.","title":"Analyzing the expressive power of graph neural networks in a spectral perspective","authors":"Balcilar","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bergstra et al., 2013] James Bergstra, Daniel Yamins, and David Cox. Making a science of model search: Hyperpa- rameter optimization in hundreds of dimensions for vision architectures. In ICML, pages 115-123. PMLR, 2013.","title":"Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures","authors":"James Bergstra, Daniel Yamins, David Cox","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bojchevski and Günnemann, 2018] Aleksandar Bojchevski and Stephan Günnemann. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. In ICLR, 2018.","title":"Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking","authors":"Aleksandar Bojchevski, Stephan Günnemann","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bruna and Mallat, 2013] Joan Bruna and Stéphane Mallat. Invariant scattering convolution networks. IEEE TPAMI, 35(8):1872-1886, 2013.","title":"Invariant scattering convolution networks","authors":"Bruna, Mallat, Joan Bruna, Stéphane Mallat","year":"2013","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Coifman and Maggioni, 2006] Ronald R Coifman and Mauro Maggioni. Diffusion wavelets. Applied and computational harmonic analysis, 21(1):53-94, 2006.","title":"Diffusion wavelets","authors":"Mauro Ronald R Coifman, Maggioni","year":"2006","venue":"Applied and computational harmonic analysis","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fey and Lenssen, 2019] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geomet- ric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.","title":"Fast graph representation learning with PyTorch Geometric","authors":"Matthias Fey, Jan E Lenssen","year":"2019","venue":"ICLR Workshop on Representation Learning on Graphs and Manifolds","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gama et al., 2018] Fernando Gama, Alejandro Ribeiro, and Joan Bruna. Diffusion scattering transforms on graphs. In ICLR, 2018.","title":"Diffusion scattering transforms on graphs","authors":"Fernando Gama, Alejandro Ribeiro, Joan Bruna","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao et al., 2019] Feng Gao, Guy Wolf, and Matthew Hirn. Geometric scattering for graph data analysis. In ICML, pages 2122-2131. PMLR, 2019.","title":"Geometric scattering for graph data analysis","authors":"Gao","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hamilton et al., 2017] William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, pages 1025-1035, 2017.","title":"Inductive representation learning on large graphs","authors":"Rex William L Hamilton, Jure Ying, Leskovec","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jiang and Luo, 2021] Weiwei Jiang and Jiayun Luo. Graph neural network for traffic forecasting: A survey. arXiv preprint arXiv:2101.11174, 2021.","title":"Graph neural network for traffic forecasting: A survey","authors":"Luo Jiang, Weiwei Jiang, Jiayun Luo","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Klicpera et al., 2018] Johannes Klicpera, Aleksandar Bo- jchevski, and Stephan Günnemann. Predict then propa- gate: Graph neural networks meet personalized pagerank. In ICLR, 2018.","title":"Predict then propagate: Graph neural networks meet personalized pagerank","authors":"Klicpera","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kong et al., 2021] Youyong Kong, Shuwen Gao, Yingying Yue, Zhenhua Hou, Huazhong Shu, Chunming Xie, Zhijun Zhang, and Yonggui Yuan. Spatio-temporal graph convo- lutional network for diagnosis and treatment response pre- diction of major depressive disorder from functional con- nectivity. Human brain mapping, 2021.","title":"Spatio-temporal graph convolutional network for diagnosis and treatment response prediction of major depressive disorder from functional connectivity","authors":"Kong","year":"2021","venue":"Human brain mapping","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lee et al., 2019] Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling. In ICML, pages 3734- 3743. PMLR, 2019.","title":"Self-attention graph pooling","authors":"Lee","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2018] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, 2018.","title":"Deeper insights into graph convolutional networks for semi-supervised learning","authors":"Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liao et al., 2019] Renjie Liao, Zhizhen Zhao, Raquel Urta- sun, and Richard Zemel. Lanczosnet: Multi-scale deep graph convolutional networks. In ICLR, 2019.","title":"Lanczosnet: Multi-scale deep graph convolutional networks","authors":"Renjie Liao, Zhizhen Zhao, Raquel Urtasun, Richard Zemel","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mallat, 2012] Stéphane Mallat. Group invariant scatter- ing. Communications on Pure and Applied Mathematics, 65(10):1331-1398, 2012.","title":"Group invariant scattering","authors":"Stéphane Mallat","year":"2012","venue":"Communications on Pure and Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Min et al., 2020] Yimeng Min, Frederik Wenkel, and Guy Wolf. Scattering gcn: Overcoming oversmoothness in graph convolutional networks. In H. Larochelle, M. Ran- zato, R. Hadsell, M. F. Balcan, and H. Lin, editors, NIPS, volume 33, pages 14498-14508.","title":"Scattering gcn: Overcoming oversmoothness in graph convolutional networks","authors":"Min","year":"2020","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"Curran Associates, Inc., 2020.","title":null,"authors":"Curran Associates, Inc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Min et al., 2021] Yimeng Min, Frederik Wenkel, and Guy Wolf. Geometric scattering attention networks. In ICASSP, pages 8518-8522. IEEE, 2021.","title":"Geometric scattering attention networks","authors":"Min","year":"2021","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Nt and Maehara, 2019] Hoang Nt and Takanori Maehara. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550, 2019.","title":"Revisiting graph neural networks: All we have is low-pass filters","authors":"Maehara Nt, Hoang Nt, Takanori Maehara","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pei et al., 2019] Hongbin Pei, Bingzhe Wei, Kevin Chen- Chuan Chang, Yu Lei, and Bo Yang. Geom-gcn: Geo- metric graph convolutional networks. In ICLR, 2019.","title":"Geom-gcn: Geometric graph convolutional networks","authors":"Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan, Yu Chang, Bo Lei, Yang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93-93, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, Tina Eliassi-Rad","year":"2008","venue":"AI magazine","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shchur et al., 2018] Oleksandr Shchur, Maximil- ian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. In NeurIPS workshop, 2018.","title":"Pitfalls of graph neural network evaluation","authors":"Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, Stephan Günnemann","year":"2018","venue":"NeurIPS workshop","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2018] Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In ICML, pages 5453-5462. PMLR, 2018.","title":"Representation learning on graphs with jumping knowledge networks","authors":"Xu","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu et al., 2019] Bingbing Xu, Huawei Shen, Qi Cao, Ket- ing Cen, and Xueqi Cheng. Graph convolutional networks using heat kernel for semi-supervised learning. In IJCAI, 2019.","title":"Graph convolutional networks using heat kernel for semi-supervised learning","authors":"Bingbing Xu, Huawei Shen, Qi Cao, Keting Cen, Xueqi Cheng","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/267"},{"position":29,"raw":"[Zhang and Chen, 2018] Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. NIPS, 31:5165-5175, 2018.","title":"Link prediction based on graph neural networks","authors":"Chen ; ] Muhan Zhang, Yixin Zhang, Chen","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zou and Lerman, 2020] Dongmian Zou and Gilad Lerman. Graph convolutional neural networks via scattering. Ap- plied and Computational Harmonic Analysis, 49(3):1046- 1074, 2020.","title":"Graph convolutional neural networks via scattering","authors":"Lerman Zou","year":"2020","venue":"Applied and Computational Harmonic Analysis","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}