{"type":"ScholarlyArticle","title":"Spatio-Temporal Attentive RNN for Node Classification in Temporal Attributed Graphs","authors":[{"name":"Dongkuan Xu","slug":"dongkuan-xu","given":"Dongkuan","family":"Xu","affiliation":"The Pennsylvania State University"},{"name":"Wei Cheng","slug":"wei-cheng","given":"Wei","family":"Cheng","affiliation":"NEC Laboratories America, Inc."},{"name":"Dongsheng Luo","slug":"dongsheng-luo","given":"Dongsheng","family":"Luo","affiliation":"The Pennsylvania State University"},{"name":"Xiao Liu","slug":"xiao-liu","given":"Xiao","family":"Liu","affiliation":"The Pennsylvania State University"},{"name":"Xiang Zhang","slug":"xiang-zhang","given":"Xiang","family":"Zhang","affiliation":"The Pennsylvania State University"}],"abstract":"Node classification in graph-structured data aims to classify the nodes where labels are only available for a subset of nodes. This problem has attracted considerable research efforts in recent years. In real-world applications, both graph topology and node attributes evolve over time. Existing techniques, however, mainly focus on static graphs and lack the capability to simultaneously learn both temporal and spatial/structural features. Node classification in temporal attributed graphs is challenging for two major aspects. First, effectively modeling the spatio-temporal contextual information is hard. Second, as temporal and spatial dimensions are entangled, to learn the feature representation of one target node, it’s desirable and challenging to differentiate the relative importance of different factors, such as different neighbors and time periods. In this paper, we propose STAR, a spatio-temporal attentive recurrent network model, to deal with the above challenges. STAR extracts the vector representation of neighborhood by sampling and aggregating local neighbor nodes. It further feeds both the neighborhood representation and node attributes into a gated recurrent unit network to jointly learn the spatio-temporal contextual information. On top of that, we take advantage of the dual attention mechanism to perform a thorough analysis on the model interpretability. Extensive experiments on real datasets demonstrate the effectiveness of the STAR model.","doi":"10.24963/ijcai.2019/548","url":"https://ijcai.org/proceedings/2019/548","pdf":"https://ijcai.org/proceedings/2019/0548.pdf","markdown":"https://ijcai.org/proceedings/2019/548.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/548","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":548,"pages":{"first":3947,"last":3953},"keywords":["Machine Learning: Semi-Supervised Learning","Machine Learning: Data Mining","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: a system for large-scale machine learn- ing. In OSDI, volume 16, pages 265-283, 2016.","title":"Tensorflow: a system for large-scale machine learning","authors":"Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard","year":"2016","venue":"OSDI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Avena-Koenigsberger et al., 2018] Andrea Avena- Koenigsberger, Bratislav Misic, and Olaf Sporns. Communication dynamics in complex brain networks. Nature Reviews Neuroscience, 19(1):17, 2018.","title":"Communication dynamics in complex brain networks","authors":"Andrea Avena-Koenigsberger, Bratislav Misic, Olaf Sporns","year":"2018","venue":"Nature Reviews Neuroscience","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Du et al., 2018] Lun Du, Yun Wang, Guojie Song, Zhicong Lu, and Junshan Wang. Dynamic network embedding: An extended approach for skip-gram based network embed- ding. In IJCAI, pages 2086-2092, 2018.","title":"Dynamic network embedding: An extended approach for skip-gram based network embedding","authors":"Lun Du, Yun Wang, Guojie Song, Zhicong Lu, Junshan Wang","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/288"},{"position":5,"raw":"[Gonzalez-Castillo et al., 2015] Javier Gonzalez-Castillo, Colin W Hoy, Daniel A Handwerker, Meghan E Robinson, Laura C Buchanan, Ziad S Saad, and Peter A Bandettini. Tracking ongoing cognition in individuals using brief, whole-brain functional connectivity patterns. PNAS, 112(28):8762-8767, 2015.","title":"Tracking ongoing cognition in individuals using brief, whole-brain functional connectivity patterns","authors":"Gonzalez-Castillo","year":"2015","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goyal et al., 2018a] Palash Goyal, Sujit Rokka Chhetri, and Arquimedes Canedo. dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. arXiv preprint arXiv:1809.02657, 2018.","title":"dyngraph2vec: Capturing network dynamics using dynamic graph representation learning","authors":"Goyal","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goyal et al., 2018b] Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. Dyngem: Deep embedding method for dy- namic graphs. arXiv preprint arXiv:1805.11273, 2018.","title":"Dyngem: Deep embedding method for dynamic graphs","authors":"Goyal","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In SIGKDD, pages 855-864. ACM, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS, pages 1024-1034, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2018] Jundong Li, Kewei Cheng, Liang Wu, and Huan Liu. Streaming link prediction on dynamic attributed networks. In WSDM, pages 369-377. ACM, 2018.","title":"Streaming link prediction on dynamic attributed networks","authors":"Li","year":"2018","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ma et al., 2018] Yao Ma, Ziyi Guo, Zhaochun Ren, Eric Zhao, Jiliang Tang, and Dawei Yin. Streaming graph neu- ral networks. arXiv preprint arXiv:1810.10627, 2018.","title":"Streaming graph neural networks","authors":"Yao Ma, Ziyi Guo, Zhaochun Ren, Eric Zhao, Jiliang Tang, Dawei Yin","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Maas et al., 2013] Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. Rectifier nonlinearities improve neural net- work acoustic models. In ICML, volume 30, page 3, 2013.","title":"Rectifier nonlinearities improve neural network acoustic models","authors":"Awni Y Andrew L Maas, Andrew Y Hannun, Ng","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ni et al., 2018] Jingchao Ni, Shiyu Chang, Xiao Liu, Wei Cheng, Haifeng Chen, Dongkuan Xu, and Xiang Zhang. Co-regularized deep multi-network embedding. In WWW, pages 469-478. International World Wide Web Confer- ences Steering Committee, 2018.","title":"Co-regularized deep multi-network embedding","authors":"Ni","year":"2018","venue":"International World Wide Web Conferences Steering Committee","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In SIGKDD, pages 701-710. ACM, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Preti et al., 2017] Maria Giulia Preti, Thomas AW Bolton, and Dimitri Van De Ville. The dynamic functional con- nectome: State-of-the-art and perspectives. Neuroimage, 160:41-54, 2017.","title":"The dynamic functional connectome: State-of-the-art and perspectives","authors":"Maria Giulia Preti, Thomas Aw Bolton, Dimitri Van De, Ville","year":"2017","venue":"Neuroimage","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shaw et al., 2018] Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representa- tions. arXiv preprint arXiv:1803.02155, 2018.","title":"Self-attention with relative position representations","authors":"Peter Shaw, Jakob Uszkoreit, Ashish Vaswani","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2016] Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network embedding. In KDD, pages 1225-1234. ACM, 2016.","title":"Structural deep network embedding","authors":"Wang","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017a] Shen Wang, Lifang He, Bokai Cao, Chun-Ta Lu, Philip S Yu, and Ann B Ragin. Structural deep brain network mining. In SIGKDD, pages 475-484. ACM, 2017.","title":"Structural deep brain network mining","authors":"Wang","year":"2017","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2017b] Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang. Community pre- serving network embedding. In AAAI, 2017.","title":"Community preserving network embedding","authors":"Wang","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2018] Wenchao Yu, Wei Cheng, Charu C Aggar- wal, Kai Zhang, Haifeng Chen, and Wei Wang. Netwalk: A flexible deep embedding approach for anomaly detec- tion in dynamic networks. In KDD, pages 2672-2681. ACM, 2018.","title":"Netwalk: A flexible deep embedding approach for anomaly detection in dynamic networks","authors":"Wenchao Yu, Wei Cheng, C Charu, Kai Aggarwal, Haifeng Zhang, Wei Chen, Wang","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2018] Lekui Zhou, Yang Yang, Xiang Ren, Fei Wu, and Yueting Zhuang. Dynamic network embedding by modeling triadic closure process. In AAAI, 2018.","title":"Dynamic network embedding by modeling triadic closure process","authors":"Lekui Zhou, Yang Yang, Xiang Ren, Fei Wu, Yueting Zhuang","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu et al., 2018] Dingyuan Zhu, Peng Cui, Ziwei Zhang, Jian Pei, and Wenwu Zhu. High-order proximity preserved embedding for dynamic networks. TKDE, 30(11):2134- 2144, 2018.","title":"High-order proximity preserved embedding for dynamic networks","authors":"Dingyuan Zhu, Peng Cui, Ziwei Zhang, Jian Pei, Wenwu Zhu","year":"2018","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zuo et al., 2018] Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, and Junjie Wu. Embedding tempo- ral network via neighborhood formation. In KDD, pages 2857-2866. ACM, 2018.","title":"Embedding temporal network via neighborhood formation","authors":"Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, Junjie Wu","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}