{"type":"ScholarlyArticle","title":"Temporal Attribute Prediction via Joint Modeling of Multi-Relational Structure Evolution","authors":[{"name":"Sankalp Garg","slug":"sankalp-garg","given":"Sankalp","family":"Garg","affiliation":"Indian Institute of Technology Delhi"},{"name":"Navodita Sharma","slug":"navodita-sharma","given":"Navodita","family":"Sharma","affiliation":"Indian Institute of Technology Madras"},{"name":"Woojeong Jin","slug":"woojeong-jin","given":"Woojeong","family":"Jin","affiliation":"University of Southern California"},{"name":"Xiang Ren","slug":"xiang-ren","given":"Xiang","family":"Ren","affiliation":"University of Southern California"}],"abstract":"Time series prediction is an important problem in machine learning. Previous methods for time series prediction did not involve additional information. With a lot of dynamic knowledge graphs available, we can use this additional information to predict the time series better. Recently, there has been a focus on the application of deep representation learning on dynamic graphs. These methods predict the structure of the graph by reasoning over the interactions in the graph at previous time steps. In this paper, we propose a new framework to incorporate the information from dynamic knowledge graphs for time series prediction. We show that if the information contained in the graph and the time series data are closely related, then this inter-dependence can be used to predict the time series with improved accuracy. Our framework, DArtNet, learns a static embedding for every node in the graph as well as a dynamic embedding which is dependent on the dynamic attribute value (time-series). Then it captures the information from the neighborhood by taking a relation specific mean and encodes the history information using RNN. We jointly train the model link prediction and attribute prediction. We evaluate our method on five specially curated datasets for this problem and show a consistent improvement in time series prediction results. We release the data and code of model DArtNet for future research.","doi":"10.24963/ijcai.2020/386","url":"https://ijcai.org/proceedings/2020/386","pdf":"https://ijcai.org/proceedings/2020/0386.pdf","markdown":"https://ijcai.org/proceedings/2020/386.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/386","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":386,"pages":{"first":2785,"last":2791},"keywords":["Data Mining: Mining Graphs, Semi Structured Data, Complex Data","Machine Learning: Time-series;Data Streams","Data Mining: Mining Spatial, Temporal Data"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24832/file_24832.m3u8","length_sec":253,"provider":"hls"}],"references":[{"position":1,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi- relational data. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 2787-2795.","title":"Translating embeddings for modeling multirelational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Curran Associates, Inc., 2013.","title":null,"authors":"Curran Associates, Inc","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Caruana, 1997] Rich Caruana. Multitask learning. Machine Learning, 28(1):41-75, 1997.","title":"Multitask learning","authors":"Caruana","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the proper- ties of neural machine translation: Encoder-decoder ap- proaches. CoRR, abs/1409.1259, 2014.","title":"On the properties of neural machine translation: Encoder-decoder approaches","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dasgupta et al., 2018] Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha Talukdar. HyTE: Hyperplane-based temporally aware knowledge graph embedding. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2001-2011, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. [","title":"HyTE: Hyperplane-based temporally aware knowledge graph embedding","authors":"Swayambhu Shib Sankar Dasgupta, Partha Nath Ray, Talukdar","year":"2018","venue":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dettmers et al., 2017] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. CoRR, abs/1707.01476, 2017.","title":"Convolutional 2d knowledge graph embeddings","authors":"Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[García-Durán et al., 2018] Alberto García-Durán, Sebasti- jan Dumancic, and Mathias Niepert. Learning se- quence encoders for temporal knowledge graph comple- tion. CoRR, abs/1809.03202, 2018.","title":"Learning sequence encoders for temporal knowledge graph completion","authors":"Alberto García-Durán, Sebastijan Dumancic, Mathias Niepert","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goyal et al., 2018] Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. Dyngem: Deep embedding method for dy- namic graphs. CoRR, abs/1805.11273, 2018.","title":"Dyngem: Deep embedding method for dynamic graphs","authors":"Palash Goyal, Nitin Kamra, Xinran He, Yan Liu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hamilton et al., 2017] William L. Hamilton, Rex Ying, and Jure Leskovec. Representation learning on graphs: Meth- ods and applications. CoRR, abs/1709.05584, 2017.","title":"Representation learning on graphs: Methods and applications","authors":"William L Hamilton, Rex Ying, Jure Leskovec","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hamilton, 1994] James Douglas Hamilton. Time series analysis. Princeton Univ. Press, Princeton, NJ, 1994.","title":"Time series analysis","authors":"James Douglas, Hamilton","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jin et al., 2019] Woojeong Jin, He Jiang, Changlin Zhang, Pedro Szekely, and Xiang Ren. Recurrent event network for reasoning over temporal knowledge graphs. arXiv preprint arXiv:1904.05530, 2019.","title":"Recurrent event network for reasoning over temporal knowledge graphs","authors":"Jin","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kipf and Welling, 2016] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. CoRR, abs/1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kipf et al., 2018] Thomas N. Kipf, Ethan Fetaya, Kuan- Chieh Wang, Max Welling, and Richard S. Zemel. Neu- ral relational inference for interacting systems. In ICML, 2018.","title":"Neural relational inference for interacting systems","authors":"Thomas N Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, Richard S Zemel","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kumar et al., 2018] Srijan Kumar, Xikun Zhang, and Jure Leskovec. Learning dynamic embeddings from temporal interactions. CoRR, abs/1812.02289, 2018.","title":"Learning dynamic embeddings from temporal interactions","authors":"Srijan Kumar, Xikun Zhang, Jure Leskovec","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2017] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Graph convolutional recurrent neural network: Data-driven traffic forecasting. CoRR, abs/1707.01926, 2017.","title":"Graph convolutional recurrent neural network: Data-driven traffic forecasting","authors":"Yaguang Li, Rose Yu, Cyrus Shahabi, Yan Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lippi et al., 2013] Marco Lippi, Matteo Bertini, and Paolo Frasconi. Short-term traffic flow forecasting: An experi- mental comparison of time-series analysis and supervised learning. IEEE Transactions on Intelligent Transportation Systems, 14:871-882, 2013.","title":"Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning","authors":"Lippi","year":"2013","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2011] Wei Liu, Yu Zheng, Sanjay Chawla, Jing Yuan, and Xie Xing. Discovering spatio-temporal causal interactions in traffic data streams. In Proceedings of the 17th ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, KDD '11, pages 1010- 1018, New York, NY, USA, 2011. ACM.","title":"Discovering spatio-temporal causal interactions in traffic data streams","authors":"Wei Liu, Yu Zheng, Sanjay Chawla, Jing Yuan, Xie Xing","year":"2011","venue":"Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '11","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Neil et al., 2018] Daniel Neil, Joss Briody, Alix Lacoste, Aaron Sim, Paidi Creed, and Amir Saffari. Interpretable graph convolutional neural networks for inference on noisy knowledge graphs. CoRR, abs/1812.00279, 2018.","title":"Interpretable graph convolutional neural networks for inference on noisy knowledge graphs","authors":"Daniel Neil, Joss Briody, Alix Lacoste, Aaron Sim, Paidi Creed, Amir Saffari","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pareja et al., 2019] Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, and Charles E. Leisersen. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. CoRR, abs/1902.10191, 2019.","title":"Evolvegcn: Evolving graph convolutional networks for dynamic graphs","authors":"Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Charles E Leisersen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rossi, 2018] Ryan A. Rossi. Relational time series forecast- ing. The Knowledge Engineering Review, 33:e1, 2018.","title":"Relational time series forecasting","authors":"Ryan A Rossi","year":"2018","venue":"The Knowledge Engineering Review","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ruder, 2017] Sebastian Ruder. An overview of multi-task learning in deep neural networks. CoRR, abs/1706.05098, 2017.","title":"An overview of multi-task learning in deep neural networks","authors":"Sebastian Ruder","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schlichtkrull et al., 2018] Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. Lecture Notes in Computer Science, page 593-607, 2018.","title":"Modeling relational data with graph convolutional networks","authors":"Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den, Ivan Berg, Max Titov, Welling","year":"2018","venue":"Lecture Notes in Computer Science","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. CoRR, abs/1409.3215, 2014.","title":"Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Trivedi et al., 2017] Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. Know-evolve: Deep reasoning in temporal knowledge graphs. CoRR, abs/1705.05742, 2017.","title":"Know-evolve: Deep reasoning in temporal knowledge graphs","authors":"Rakshit Trivedi, Hanjun Dai, Yichen Wang, Le Song","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Trivedi et al., 2018] Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Representation learning over dynamic graphs. CoRR, abs/1803.04051, 2018.","title":"Representation learning over dynamic graphs","authors":"Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, Hongyuan Zha","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2019] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. A comprehensive survey on graph neural networks. CoRR, abs/1901.00596, 2019.","title":"A comprehensive survey on graph neural networks","authors":"Wu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2014] Bishan Yang, Wen tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and re- lations for learning and inference in knowledge bases. CoRR, abs/1412.6575, 2014.","title":"Embedding entities and relations for learning and inference in knowledge bases","authors":"Bishan Yang, Wen Tau Yih, Xiaodong He, Jianfeng Gao, Li Deng","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[You et al., 2018] Jiaxuan You, Rex Ying, Xiang Ren, William L. Hamilton, and Jure Leskovec. Graphrnn: A deep generative model for graphs. CoRR, abs/1802.08773, 2018.","title":"Graphrnn: A deep generative model for graphs","authors":"Jiaxuan You, Rex Ying, Xiang Ren, William L Hamilton, Jure Leskovec","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}