{"type":"ScholarlyArticle","title":"Graph Neural Architecture Search","authors":[{"name":"Yang Gao","slug":"yang-gao","given":"Yang","family":"Gao","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences , Beijing , China; School of Cyber Security , University of Chinese Academy of Sciences , Beijing , China"},{"name":"Hong Yang","slug":"hong-yang","given":"Hong","family":"Yang","affiliation":"Centre for Artificial Intelligence , University of Technology Sydney , Australia"},{"name":"Peng Zhang","slug":"peng-zhang","given":"Peng","family":"Zhang","affiliation":"Ant Financial Services Group , Hangzhou , China"},{"name":"Chuan Zhou","slug":"chuan-zhou","given":"Chuan","family":"Zhou","affiliation":"Academy of Mathematics and Systems Science , Chinese Academy of Sciences , Beijing , China; School of Cyber Security , University of Chinese Academy of Sciences , Beijing , China"},{"name":"Yue Hu","slug":"yue-hu","given":"Yue","family":"Hu","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences , Beijing , China; School of Cyber Security , University of Chinese Academy of Sciences , Beijing , China"}],"abstract":"Graph neural networks (GNNs) emerged recently as a powerful tool for analyzing non-Euclidean data such as social network data. Despite their success, the design of graph neural networks requires heavy manual work and domain knowledge. In this paper, we present a graph neural architecture search method (GraphNAS) that enables automatic design of the best graph neural architecture based on reinforcement learning. Specifically, GraphNAS uses a recurrent network to generate variable-length strings that describe the architectures of graph neural networks, and trains the recurrent network with policy gradient to maximize the expected accuracy of the generated architectures on a validation data set. Furthermore, to improve the search efficiency of GraphNAS on big networks, GraphNAS restricts the search space from an entire architecture space to a sequential concatenation of the best search results built on each single architecture layer. Experiments on real-world datasets demonstrate that GraphNAS can design a novel network architecture that rivals the best human-invented architecture in terms of validation set accuracy. Moreover, in a transfer learning task we observe that graph neural architectures designed by GraphNAS, when transferred to new datasets, still gain improvement in terms of prediction accuracy.","doi":"10.24963/ijcai.2020/195","url":"https://ijcai.org/proceedings/2020/195","pdf":"https://ijcai.org/proceedings/2020/0195.pdf","markdown":"https://ijcai.org/proceedings/2020/195.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/195","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Data Mining","track":"Main track","number":195,"pages":{"first":1403,"last":1409},"keywords":["Machine Learning Applications: Applications of Reinforcement Learning","Machine Learning Applications: Networks","Data Mining: Mining Graphs, Semi Structured Data, Complex Data"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24406/file_24406.m3u8","length_sec":296,"provider":"hls"}],"references":[{"position":1,"raw":"[Bello et al., 2017] Irwan Bello, Barret Zoph, Vijay Vasude- van, and Quoc V. Le. Neural optimizer search with rein- forcement learning. In ICML, 2017.","title":"Neural optimizer search with reinforcement learning","authors":"Irwan Bello, Barret Zoph, Vijay Vasudevan, V Quoc, Le","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bianchi et al., 2019] Filippo Maria Bianchi, Daniele Grat- tarola, Lorenzo Livi, and Cesare Alippi. Graph neu- ral networks with convolutional arma filters. ArXiv, abs/1901.01343, 2019.","title":"Graph neural networks with convolutional arma filters","authors":"Maria Filippo, Daniele Bianchi, Lorenzo Grattarola, Cesare Livi, Alippi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2019] Han Cai, Ligeng Zhu, and Song Han. Prox- ylessNAS: Direct neural architecture search on target task and hardware. In International Conference on Learning Representations, 2019.","title":"Prox-ylessNAS: Direct neural architecture search on target task and hardware","authors":"Han Cai, Ligeng Zhu, Song Han","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres- son, and Pierre Vandergheynst. Convolutional neural net- works on graphs with fast localized spectral filtering. In NIPS, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong and Yang, 2019] Xuanyi Dong and Yi Yang. Search- ing for a robust neural architecture in four gpu hours. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1761-1770, 2019.","title":"Searching for a robust neural architecture in four gpu hours","authors":"Dong, Yang ;, Xuanyi Dong, Yi Yang","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fey and Lenssen, 2019] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geomet- ric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.","title":"Fast graph representation learning with PyTorch Geometric","authors":"Matthias Fey, Jan E Lenssen","year":"2019","venue":"ICLR Workshop on Representation Learning on Graphs and Manifolds","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gori et al., 2005] M. Gori, G. Monfardini, and F. Scarselli. A new model for learning in graph domains. In IEEE In- ternational Joint Conference on Neural Networks, 2005.","title":"A new model for learning in graph domains","authors":"M Gori, G Monfardini, F Scarselli","year":"2005","venue":"IEEE International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hamilton et al., 2017] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, 2017.","title":"Inductive representation learning on large graphs","authors":"William L Hamilton, Zhitao Ying, Jure Leskovec","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu et al., 2019] Fenyu Hu, Yanqiao Zhu, Shu Wu, Liang Wang, and Tieniu Tan. Hierarchical graph convolutional networks for semi-supervised node classification. In Pro- ceedings of the Twenty-Eighth International Joint Confer- ence on Artificial Intelligence, (IJCAI), 2019.","title":"Hierarchical graph convolutional networks for semi-supervised node classification","authors":"Fenyu Hu, Yanqiao Zhu, Shu Wu, Liang Wang, Tieniu Tan","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Rep- resentations (ICLR), 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Klicpera et al., 2019] Johannes Klicpera, Aleksandar Bo- jchevski, and Stephan Günnemann. Combining neural networks with personalized pagerank for classification on graphs. In International Conference on Learning Repre- sentations, 2019.","title":"Combining neural networks with personalized pagerank for classification on graphs","authors":"Klicpera","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2018a] Hanxiao Liu, Karen Simonyan, and Yim- ing Yang. Darts: Differentiable architecture search. CoRR, abs/1806.09055, 2018.","title":"Darts: Differentiable architecture search","authors":"Liu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2018b] Ziqi Liu, Chaochao Chen, Longfei Li, Jun Zhou, Xiaolong Li, and Le Song. Geniepath: Graph neural networks with adaptive receptive paths. CoRR, abs/1802.00910, 2018.","title":"Geniepath: Graph neural networks with adaptive receptive paths","authors":"Liu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Niepert et al., 2016] Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. In ICML, 2016.","title":"Learning convolutional neural networks for graphs","authors":"Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pham et al., 2018] Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, and Jeff Dean. Efficient neural archi- tecture search via parameter sharing. In ICML, 2018.","title":"Efficient neural architecture search via parameter sharing","authors":"Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, Jeff Dean","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Real et al., 2018] Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le. Regularized evolution for im- age classifier architecture search. CoRR, abs/1802.01548, 2018.","title":"Regularized evolution for image classifier architecture search","authors":"Esteban Real, Alok Aggarwal, Yanping Huang, V Quoc, Le","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shchur et al., 2018] Oleksandr Shchur, Maximil- ian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. 2018.","title":"Pitfalls of graph neural network evaluation","authors":"Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, Stephan Günnemann","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Lawrence Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Lawrence Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Velickovic et al., 2017] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. CoRR, abs/1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Williams, 1992] Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine Learning, 8(3-4):229-256, 1992.","title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","authors":"Ronald J Williams","year":"1992","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu et al., 2019a] Felix Wu, Amauri H. Souza, Tianyi Zhang, Christopher Fifty, Rui Zhang, and Kilian Q. Wein- berger. Simplifying graph convolutional networks. In ICML, 2019.","title":"Simplifying graph convolutional networks","authors":"Wu","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu et al., 2019b] Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. Session-based recommendation with graph neural networks. In AAAI, 2019.","title":"Session-based recommendation with graph neural networks","authors":"Wu","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xie et al., 2019] Sirui Xie, H P Zheng, Chunxiao Liu, and Liang Lin. Snas: Stochastic neural architecture search. In International Conference on Learning Representations, 2019.","title":"Snas: Stochastic neural architecture search","authors":"Sirui Xie, H P Zheng, Chunxiao Liu, Liang Lin","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xu et al., 2018] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural net- works? CoRR, abs/1810.00826, 2018.","title":"How powerful are graph neural networks?","authors":"Xu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yan et al., 2018] Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph convolutional networks for skeleton-based action recognition. In AAAI, 2018.","title":"Spatial temporal graph convolutional networks for skeleton-based action recognition","authors":"Sijie Yan, Yuanjun Xiong, Dahua Lin","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[You et al., 2019] Jiaxuan You, Rex Ying, and Jure Leskovec. Position-aware graph neural networks. In ICML, 2019.","title":"Position-aware graph neural networks","authors":"You","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2018] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In IJCAI, 2018.","title":"Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting","authors":"Yu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/505"},{"position":28,"raw":"[Zaheer et al., 2017] Manzil Zaheer, Satwik Kottur, Siamak Ravanbhakhsh, Barnabas Poczos, and Alexander Smola. Deep sets. 2017.","title":"Deep sets","authors":"Manzil Zaheer, Satwik Kottur, Siamak Ravanbhakhsh, Barnabas Poczos, Alexander Smola","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zoph and Le, 2016] Barret Zoph and Quoc V. Le. Neural architecture search with reinforcement learning. CoRR, abs/1611.01578, 2016.","title":"Neural architecture search with reinforcement learning","authors":"Le ; Zoph, Barret Zoph, Quoc V Le","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}