{"type":"ScholarlyArticle","title":"Node Embedding over Temporal Graphs","authors":[{"name":"Uriel Singer","slug":"uriel-singer","given":"Uriel","family":"Singer","affiliation":"Technion , Israel"},{"name":"Ido Guy","slug":"ido-guy","given":"Ido","family":"Guy","affiliation":"eBay Research"},{"name":"Kira Radinsky","slug":"kira-radinsky","given":"Kira","family":"Radinsky","affiliation":"Technion , Israel; Institute of Technology"}],"abstract":"In this work, we present a method for node embedding in temporal graphs. We propose an algorithm that learns the evolution of a temporal graph's nodes and edges over time and incorporates this dynamics in a temporal node embedding framework for different graph prediction tasks. We present a joint loss function that creates a temporal embedding of a node by learning to combine its historical temporal embeddings, such that it optimizes per given task (e.g., link prediction). The algorithm is initialized using static node embeddings, which are then aligned over the representations of a node at different time points, and eventually adapted for the given task in a joint optimization. We evaluate the effectiveness of our approach over a variety of temporal graphs for the two fundamental tasks of temporal link prediction and multi-label node classification, comparing to competitive baselines and algorithmic alternatives. Our algorithm shows performance improvements across many of the datasets and baselines and is found particularly effective for graphs that are less cohesive, with a lower clustering coefficient.","doi":"10.24963/ijcai.2019/640","url":"https://ijcai.org/proceedings/2019/640","pdf":"https://ijcai.org/proceedings/2019/0640.pdf","markdown":"https://ijcai.org/proceedings/2019/640.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/640","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning Applications","track":"Main track","number":640,"pages":{"first":4605,"last":4612},"keywords":["Machine Learning Applications: Networks","Natural Language Processing: Embeddings","Machine Learning: Time-series;Data Streams"],"videos":[],"references":[{"position":1,"raw":"[Barabasi and Albert, 1999] Albert-Laszlo Barabasi and Reka Albert. Emergence of scaling in random networks. Science, 286(5439):509-512, 1999.","title":"Emergence of scaling in random networks","authors":"Albert ; Albert-Laszlo Barabasi, Reka Barabasi, Albert","year":"1999","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cheng et al., 2014] Justin Cheng, Lada Adamic, P. Alex Dow, Jon Michael Kleinberg, and Jure Leskovec. Can cas- cades be predicted? In Proc. of WWW, pages 925-936, 2014.","title":"Can cascades be predicted?","authors":"Justin Cheng, Lada Adamic, P Alex Dow, Jon Michael Kleinberg, Jure Leskovec","year":"2014","venue":"Proc. of WWW","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cui et al., 2018] Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. A survey on network embedding. IEEE TKDE, 2018.","title":"A survey on network embedding","authors":"Peng Cui, Xiao Wang, Jian Pei, Wenwu Zhu","year":"2018","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Du et al., 2018] Lun Du, Yun Wang, Guojie Song, Zhicong Lu, and Junshan Wang. Dynamic network embedding: An extended approach for skip-gram based network embed- ding. In IJCAI, pages 2086-2092, 2018.","title":"Dynamic network embedding: An extended approach for skip-gram based network embedding","authors":"Lun Du, Yun Wang, Guojie Song, Zhicong Lu, Junshan Wang","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/288"},{"position":5,"raw":"[Dunlavy et al., 2011] Daniel M Dunlavy, Tamara G Kolda, and Evrim Acar. Temporal link prediction using matrix and tensor factorizations. TKDD, 5(2):10, 2011.","title":"Temporal link prediction using matrix and tensor factorizations","authors":"Tamara G Daniel M Dunlavy, Evrim Kolda, Acar","year":"2011","venue":"TKDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goyal and Ferrara, 2018] Palash Goyal and Emilio Ferrara. Graph embedding techniques, applications, and perfor- mance: A survey. KBS, 151:78-94, 2018.","title":"Graph embedding techniques, applications, and performance: A survey","authors":"Palash Goyal, Emilio Ferrara","year":"2018","venue":"KBS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goyal et al., 2017] Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. Dyngem: Deep embedding method for dy- namic graphs. arXiv preprint, abs/1805.11273, 2017.","title":"Dyngem: Deep embedding method for dynamic graphs","authors":"Palash Goyal, Nitin Kamra, Xinran He, Yan Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. Node2vec: Scalable feature learning for networks. In Proc. of KDD, pages 855-864, 2016.","title":"Node2vec: Scalable feature learning for networks","authors":"Leskovec Grover, Aditya Grover, Jure Leskovec","year":"2016","venue":"Proc. of KDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hamilton et al., 2017] William L Hamilton, Rex Ying, and Jure Leskovec. Representation learning on graphs: Meth- ods and applications. IEEE DEB 2017, 2017.","title":"Representation learning on graphs: Methods and applications","authors":"Rex William L Hamilton, Jure Ying, Leskovec","year":"2017","venue":"IEEE DEB","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hawkes, 1971] Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.","title":"Spectra of some self-exciting and mutually exciting point processes","authors":"Alan G Hawkes","year":"1971","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hurley and Cattell, 1962] John R Hurley and Raymond B Cattell. The procrustes program: Producing direct rota- tion to test a hypothesized factor structure. SRBS, 1962.","title":"The procrustes program: Producing direct rotation to test a hypothesized factor structure","authors":"R John, Raymond B Hurley, Cattell","year":"1962","venue":"SRBS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kupavskii et al., 2012] Andrey Kupavskii, Liudmila Os- troumova, Alexey Umnov, Svyatoslav Usachev, Pavel Serdyukov, Gleb Gusev, and Andrey Kustarev. Prediction of retweet cascade size over time. In Proc. of CIKM, pages 2335-2338, 2012.","title":"Prediction of retweet cascade size over time","authors":"Andrey Kupavskii, Liudmila Ostroumova, Alexey Umnov, Svyatoslav Usachev, Pavel Serdyukov, Gleb Gusev, Andrey Kustarev","year":"2012","venue":"Proc. of CIKM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Leskovec et al., 2005] Jure Leskovec, Jon Kleinberg, and Christos Faloutsos. Graphs over time: Densification laws, shrinking diameters and possible explanations. In KDD, 2005.","title":"Graphs over time: Densification laws, shrinking diameters and possible explanations","authors":"Jure Leskovec, Jon Kleinberg, Christos Faloutsos","year":"2005","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Leskovec et al., 2008] Jure Leskovec, Lars Backstrom, Ravi Kumar, and Andrew Tomkins. Microscopic evolu- tion of social networks. In Proc. of KDD, pages 462-470, 2008.","title":"Microscopic evolution of social networks","authors":"Jure Leskovec, Lars Backstrom, Ravi Kumar, Andrew Tomkins","year":"2008","venue":"Proc. of KDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2014] Xiaoyi Li, Nan Du, Hui Li, Kang Li, Jing Gao, and Aidong Zhang. A deep learning approach to link prediction in dynamic networks. In Proc. of SDM, 2014.","title":"A deep learning approach to link prediction in dynamic networks","authors":"Xiaoyi Li, Nan Du, Hui Li, Kang Li, Jing Gao, Aidong Zhang","year":"2014","venue":"Proc. of SDM","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2016] Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. Gated graph se- quence neural networks. ICLR, 2016.","title":"Gated graph sequence neural networks","authors":"Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard Zemel","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Cor- rado, and Jeffrey Dean. Efficient estimation of word rep- resentations in vector space. ICLR, abs/1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nguyen et al., 2018] Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. Continuous-time dynamic network em- beddings. In Proc. of WWW Companion, pages 969-976, 2018.","title":"Continuous-time dynamic network embeddings","authors":"Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, Sungchul Kim","year":"2018","venue":"Proc. of WWW Companion","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Palangi et al., 2016] Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and Rabab K. Ward. Deep sentence embedding using the long short term memory network. IEEE TASLP, 2016.","title":"Deep sentence embedding using the long short term memory network","authors":"Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, Rabab K Ward","year":"2016","venue":"IEEE TASLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Patil and Nakamura, 2005] Ashwini Patil and Haruki Naka- mura. Hint: a database of annotated protein-protein inter- actions and their homologs. Biophysics, 1:21-24, 2005.","title":"Hint: a database of annotated protein-protein interactions and their homologs","authors":"Ashwini Patil, Haruki Nakamura","year":"2005","venue":"Biophysics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pei et al., 2016] Yulong Pei, Jianpeng Zhang, George HL Fletcher, and Mykola Pechenizkiy. Node classification in dynamic social networks. In AALTD Workshop, 2016.","title":"Node classification in dynamic social networks","authors":"Pei","year":"2016","venue":"AALTD Workshop","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In Proc. of KDD, pages 701-710, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"Proc. of KDD","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Trivedi et al., 2017] Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. Know-evolve: Deep temporal rea- soning for dynamic knowledge graphs. In ICML, 2017.","title":"Know-evolve: Deep temporal reasoning for dynamic knowledge graphs","authors":"Rakshit Trivedi, Hanjun Dai, Yichen Wang, Le Song","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Trivedi et al., 2019] Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Representation learning over dynamic graphs. ICLR, abs/1803.04051, 2019.","title":"Representation learning over dynamic graphs","authors":"Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, Hongyuan Zha","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2016] Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network embedding. In Proc. of KDD, pages 1225-1234, 2016.","title":"Structural deep network embedding","authors":"Wang","year":"2016","venue":"Proc. of KDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wasserman and Faust, 1994] Stanley Wasserman and Katherine Faust. Social network analysis: Methods and applications, volume 8. Cambridge university press, 1994.","title":"Social network analysis: Methods and applications","authors":"Faust Wasserman, Stanley Wasserman, Katherine Faust","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yan et al., 2007] Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, and Stephen Lin. Graph embedding and extensions: A general framework for di- mensionality reduction. IEEE TPAMI, 29(1), 2007.","title":"Graph embedding and extensions: A general framework for dimensionality reduction","authors":"Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, Stephen Lin","year":"2007","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yu et al., 2017] Wenchao Yu, Charu C. Aggarwal, and Wei Wang. Temporally factorized network modeling for evo- lutionary network analysis. 2017.","title":"Temporally factorized network modeling for evolutionary network analysis","authors":"Wenchao Yu, Charu C Aggarwal, Wei Wang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhou et al., 2018] Le-kui Zhou, Yang Yang, Xiang Ren, Fei Wu, and Yueting Zhuang. Dynamic network embedding by modeling triadic closure process. 2018.","title":"Dynamic network embedding by modeling triadic closure process","authors":"Le-Kui Zhou, Yang Yang, Xiang Ren, Fei Wu, Yueting Zhuang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zuo et al., 2019] Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, and Junjie Wu. Embedding temporal network via neighborhood formation. ACM, 2019.","title":"Embedding temporal network via neighborhood formation","authors":"Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, Junjie Wu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}