{"type":"ScholarlyArticle","title":"Graph Convolutional Networks using Heat Kernel for Semi-supervised Learning","authors":[{"name":"Bingbing Xu","slug":"bingbing-xu","given":"Bingbing","family":"Xu","affiliation":"CAS Key Laboratory of Network Data Science and Technology , Institute of Computing Technology , Chinese Academy of Sciences; School of Computer Science and Technology , University of Chinese Academy of Sciences , Beijing , China"},{"name":"Huawei Shen","slug":"huawei-shen","given":"Huawei","family":"Shen","affiliation":"CAS Key Laboratory of Network Data Science and Technology , Institute of Computing Technology , Chinese Academy of Sciences; School of Computer Science and Technology , University of Chinese Academy of Sciences , Beijing , China"},{"name":"Qi Cao","slug":"qi-cao","given":"Qi","family":"Cao","affiliation":"CAS Key Laboratory of Network Data Science and Technology , Institute of Computing Technology , Chinese Academy of Sciences; School of Computer Science and Technology , University of Chinese Academy of Sciences , Beijing , China"},{"name":"Keting Cen","slug":"keting-cen","given":"Keting","family":"Cen","affiliation":"CAS Key Laboratory of Network Data Science and Technology , Institute of Computing Technology , Chinese Academy of Sciences; School of Computer Science and Technology , University of Chinese Academy of Sciences , Beijing , China"},{"name":"Xueqi Cheng","slug":"xueqi-cheng","given":"Xueqi","family":"Cheng","affiliation":"CAS Key Laboratory of Network Data Science and Technology , Institute of Computing Technology , Chinese Academy of Sciences; School of Computer Science and Technology , University of Chinese Academy of Sciences , Beijing , China"}],"abstract":"Graph convolutional networks gain remarkable success in semi-supervised learning on graph-structured data. The key to graph-based semisupervised learning is capturing the smoothness of labels or features over nodes exerted by graph structure. Previous methods, spectral methods and spatial methods, devote to defining graph convolution as a weighted average over neighboring nodes, and then learn graph convolution kernels to leverage the smoothness to improve the performance of graph-based semi-supervised learning. One open challenge is how to determine appropriate neighborhood that reflects relevant information of smoothness manifested in graph structure. In this paper, we propose GraphHeat, leveraging heat kernel to enhance low-frequency filters and enforce smoothness in the signal variation on the graph. GraphHeat leverages the local structure of target node under heat diffusion to determine its neighboring nodes flexibly, without the constraint of order suffered by previous methods. GraphHeat achieves state-of-the-art results in the task of graph-based semi-supervised classification across three benchmark datasets: Cora, Citeseer and Pubmed.","doi":"10.24963/ijcai.2019/267","url":"https://ijcai.org/proceedings/2019/267","pdf":"https://ijcai.org/proceedings/2019/0267.pdf","markdown":"https://ijcai.org/proceedings/2019/267.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/267","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Knowledge Representation and Reasoning","track":"Main track","number":267,"pages":{"first":1928,"last":1934},"keywords":["Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning","Machine Learning: Semi-Supervised Learning","Machine Learning Applications: Networks"],"videos":[],"references":[{"position":1,"raw":"[Belkin et al., 2006] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled exam- ples. Journal of machine learning research, 7(Nov):2399- 2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"Mikhail Belkin, Partha Niyogi, Vikas Sindhwani","year":"2006","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bruna et al., 2014] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. Spectral networks and locally connected networks on graphs. In International Con- ference on Learning Representations (ICLR2014), CBLS, April 2014, 2014.","title":"Spectral networks and locally connected networks on graphs","authors":"Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann Lecun","year":"2014","venue":"International Conference on Learning Representations (ICLR2014)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chung and Graham, 1997] Fan RK Chung and Fan Chung Graham. Spectral graph theory. Number 92. American Mathematical Soc., 1997.","title":"Spectral graph theory","authors":"Fan Rk, Chung, Fan Chung, Graham","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres- son, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, pages 3844-3852, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Glorot and Bengio, 2010] Xavier Glorot and Yoshua Ben- gio. Understanding the difficulty of training deep feedfor- ward neural networks. In Proceedings of the thirteenth in- ternational conference on artificial intelligence and statis- tics, pages 249-256, 2010.","title":"Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks","authors":"Bengio Glorot","year":"2010","venue":"Proceedings of the thirteenth international conference on artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, pages 1024-1034, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hammond et al., 2011] David K Hammond, Pierre Van- dergheynst, and Rémi Gribonval. Wavelets on graphs via spectral graph theory. Applied and Computational Har- monic Analysis, 30(2):129-150, 2011.","title":"Wavelets on graphs via spectral graph theory","authors":"Pierre David K Hammond, Rémi Vandergheynst, Gribonval","year":"2011","venue":"Applied and Computational Harmonic Analysis","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hinton et al., 2012] Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. Deep neural networks for acous- tic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6):82-97, 2012.","title":"Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups","authors":"Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-Rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath","year":"2012","venue":"IEEE Signal processing magazine","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Rep- resentations (ICLR), 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lu and Getoor, 2003] Qing Lu and Lise Getoor. Link-based classification. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), pages 496- 503, 2003.","title":"Link-based classification","authors":"Getoor ; Lu, Qing Lu, Lise Getoor","year":"2003","venue":"Proceedings of the 20th International Conference on Machine Learning (ICML-03)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Monti et al., 2017] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proc. CVPR, volume 1, page 3, 2017.","title":"Geometric deep learning on graphs and manifolds using mixture model cnns","authors":"Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, Michael M Bronstein","year":"2017","venue":"Proc. CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social repre- sentations. In Proceedings of the 20th ACM SIGKDD in- ternational conference on Knowledge discovery and data mining, pages 701-710. ACM, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, Tina Eliassi-Rad","year":"2008","venue":"AI magazine","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shuman et al., 2013] David I Shuman, Sunil K Narang, Pas- cal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Ex- tending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Maga- zine, 30(3):83-98, 2013.","title":"The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains","authors":"David I Shuman, K Sunil, Pascal Narang, Antonio Frossard, Pierre Ortega, Vandergheynst","year":"2013","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Re- search, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Velickovic et al., 2017] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Weston et al., 2012] Jason Weston, Frédéric Ratle, Hossein Mobahi, and Ronan Collobert. Deep learning via semi- supervised embedding. In Neural Networks: Tricks of the Trade, pages 639-655. Springer, 2012.","title":"Deep learning via semisupervised embedding","authors":"Jason Weston, Frédéric Ratle, Hossein Mobahi, Ronan Collobert","year":"2012","venue":"Neural Networks: Tricks of the Trade","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2018] Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. arXiv preprint arXiv:1806.03536, 2018.","title":"Representation learning on graphs with jumping knowledge networks","authors":"Xu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2019] Bingbing Xu, Huawei Shen, Qi Cao, Yunqi Qiu, and Xueqi Cheng. Graph wavelet neural network. In International Conference on Learning Representations, 2019.","title":"Graph wavelet neural network","authors":"Bingbing Xu, Huawei Shen, Qi Cao, Yunqi Qiu, Xueqi Cheng","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2016] Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International Conference on Ma- chine Learning, pages 40-48, 2016.","title":"Revisiting semi-supervised learning with graph embeddings","authors":"Zhilin Yang, William Cohen, Ruslan Salakhudinov","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhu et al., 2003] Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03), pages 912-919, 2003.","title":"Semi-supervised learning using gaussian fields and harmonic functions","authors":"Xiaojin Zhu, Zoubin Ghahramani, John D Lafferty","year":"2003","venue":"Proceedings of the 20th International conference on Machine learning (ICML-03)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}