{"type":"ScholarlyArticle","title":"Neighbor Combinatorial Attention for Critical Structure Mining","authors":[{"name":"Tanli Zuo","slug":"tanli-zuo","given":"Tanli","family":"Zuo","affiliation":"School of Data and Computer Science , Sun Yat-sen University , China"},{"name":"Yukun Qiu","slug":"yukun-qiu","given":"Yukun","family":"Qiu","affiliation":"School of Data and Computer Science , Sun Yat-sen University , China"},{"name":"Wei-Shi Zheng","slug":"wei-shi-zheng","given":"Wei-Shi","family":"Zheng","affiliation":"School of Data and Computer Science , Sun Yat-sen University , China; Peng Cheng Laboratory , Shenzhen 518005 , China; Key Laboratory of Machine Intelligence and Advanced Computing , Ministry of Education , China"}],"abstract":"Graph convolutional networks (GCNs) have been widely used to process graph-structured data. However, existing GNN methods do not explicitly extract critical structures, which reflect the intrinsic property of a graph. In this work, we propose a novel GCN module named Neighbor Combinatorial ATtention (NCAT) to find critical structure in graph-structured data. NCAT attempts to match combinatorial neighbors with learnable patterns and assigns different weights to each combination based on the matching degree between the patterns and combinations. By stacking several NCAT modules, we can extract hierarchical structures that is helpful for down-stream tasks. Our experimental results show that NCAT achieves state-of-the-art performance on several benchmark graph classification datasets. In addition, we interpret what kind of features our model learned by visualizing the extracted critical structures.","doi":"10.24963/ijcai.2020/456","url":"https://ijcai.org/proceedings/2020/456","pdf":"https://ijcai.org/proceedings/2020/0456.pdf","markdown":"https://ijcai.org/proceedings/2020/456.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/456","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":456,"pages":{"first":3299,"last":3305},"keywords":["Machine Learning: Deep Learning","Data Mining: Mining Graphs, Semi Structured Data, Complex Data"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25315/file_25315.m3u8","length_sec":295,"provider":"hls"}],"references":[{"position":1,"raw":"[Atwood and Towsley, 2016] James Atwood and Don Towsley. Diffusion-convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1993-2001, 2016.","title":"Diffusion-convolutional neural networks","authors":"Towsley Atwood, James Atwood, Don Towsley","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Borgwardt and Kriegel, 2005] Karsten M Borgwardt and Hans-Peter Kriegel. Shortest-path kernels on graphs. In Fifth IEEE international conference on data mining (ICDM'05), pages 8-pp, 2005.","title":"Shortest-path kernels on graphs","authors":"M Karsten, Hans-Peter Borgwardt, Kriegel","year":"2005","venue":"Fifth IEEE international conference on data mining (ICDM'05)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bruna et al., 2013] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and lo- cally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013.","title":"Spectral networks and locally connected networks on graphs","authors":"Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann Lecun","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres- son, and Pierre Vandergheynst. Convolutional neural net- works on graphs with fast localized spectral filtering. In Advances in neural information processing systems, pages 3844-3852, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proceedings of International Conference on Learning Representations, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"Proceedings of International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kipf and Welling, 2017] T.N. Kipf and M. Welling. Semi- supervised classification with graph convolutional net- works. In Proceedings of International Conference on Learning Representations, 2017.","title":"Semisupervised classification with graph convolutional networks","authors":"T N Kipf, M Welling","year":"2017","venue":"Proceedings of International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kriege et al., 2016] Nils M Kriege, Pierre-Louis Giscard, and Richard Wilson. On valid optimal assignment ker- nels and applications to graph classification. In Advances in Neural Information Processing Systems, pages 1623- 1631, 2016.","title":"On valid optimal assignment kernels and applications to graph classification","authors":"Pierre-Louis Nils M Kriege, Richard Giscard, Wilson","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- offrey Hinton. Deep learning. nature, 521(7553):436, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Cor- rado, and Jeffrey Dean. Efficient estimation of word repre- sentations in vector space. In Proceedings of International Conference on Learning Representations, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":"Proceedings of International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Niepert et al., 2016] Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. In International conference on ma- chine learning, pages 2014-2023, 2016.","title":"Learning convolutional neural networks for graphs","authors":"Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov","year":"2016","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Shervashidze et al., 2009] Nino Shervashidze, SVN Vish- wanathan, Tobias Petri, Kurt Mehlhorn, and Karsten Borg- wardt. Efficient graphlet kernels for large graph compar- ison. In Artificial Intelligence and Statistics, pages 488- 495, 2009.","title":"Efficient graphlet kernels for large graph comparison","authors":"Shervashidze","year":"2009","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Shervashidze et al., 2011] Nino Shervashidze, Pascal Schweitzer, Erik Jan van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(Sep):2539- 2561, 2011.","title":"Weisfeiler-lehman graph kernels","authors":"Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, Karsten M Borgwardt","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Shuman et al., 2013] David Shuman, Sunil Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Extend- ing high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 3(30):83-98, 2013.","title":"The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains","authors":"David Shuman, Sunil Narang, Pascal Frossard, Antonio Ortega, Pierre Vandergheynst","year":"2013","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tzeng and Wu, 2019] Ruo-Chun Tzeng and Shan-Hung Wu. Distributed, egocentric representations of graphs for detecting critical structures. In International Conference on Machine Learning, pages 6354-6362, 2019.","title":"Distributed, egocentric representations of graphs for detecting critical structures","authors":"Ruo-Chun Tzeng, Shan-Hung Wu","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ulyanov et al., 2016] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The miss- ing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016.","title":"Instance normalization: The missing ingredient for fast stylization","authors":"Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Verma and Zhang, 2018] Saurabh Verma and Zhi-Li Zhang. Graph capsule convolutional neural networks. arXiv preprint arXiv:1805.08090, 2018.","title":"Graph capsule convolutional neural networks","authors":"Saurabh Verma, Zhi-Li Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vishwanathan et al., 2010] S Vichy N Vishwanathan, Nicol N Schraudolph, Risi Kondor, and Karsten M Borgwardt. Graph kernels. Journal of Machine Learning Research, 11(Apr):1201-1242, 2010.","title":"Graph kernels","authors":"N Vichy, Nicol N Vishwanathan, Risi Schraudolph, Karsten M Kondor, Borgwardt","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wu et al., 2019] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S Yu. A comprehensive survey on graph neural networks. arXiv preprint arXiv:1901.00596, 2019.","title":"A comprehensive survey on graph neural networks","authors":"Wu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2018] Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In Proceedings of the 35th Inter- national Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, pages 5449-5458, 2018.","title":"Representation learning on graphs with jumping knowledge networks","authors":"Xu","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yanardag and Vishwanathan, 2015] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1365-1374, 2015.","title":"Pinar Yanardag and SVN Vishwanathan. Deep graph kernels","authors":"Vishwanathan Yanardag","year":"2015","venue":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ying et al., 2018] Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hierarchical graph representation learning with differen- tiable pooling. In Advances in Neural Information Pro- cessing Systems, pages 4800-4810, 2018.","title":"Hierarchical graph representation learning with differentiable pooling","authors":"Ying","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2018] Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learn- ing architecture for graph classification. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.","title":"An end-to-end deep learning architecture for graph classification","authors":"Zhang","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhou et al., 2018] Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Graph neu- ral networks: A review of methods and applications. arXiv preprint arXiv:1812.08434, 2018.","title":"Graph neural networks: A review of methods and applications","authors":"Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Maosong Sun","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}