{"type":"ScholarlyArticle","title":"MERIT: Learning Multi-level Representations on Temporal Graphs","authors":[{"name":"Binbin Hu","slug":"binbin-hu","given":"Binbin","family":"Hu","affiliation":"Zhejiang University"},{"name":"Zhengwei Wu","slug":"zhengwei-wu","given":"Zhengwei","family":"Wu","affiliation":"Zhejiang University"},{"name":"Jun Zhou","slug":"jun-zhou","given":"Jun","family":"Zhou","affiliation":"Zhejiang University"},{"name":"Ziqi Liu","slug":"ziqi-liu","given":"Ziqi","family":"Liu","affiliation":"Zhejiang University"},{"name":"Zhigang Huangfu","slug":"zhigang-huangfu","given":"Zhigang","family":"Huangfu","affiliation":"Zhejiang University"},{"name":"Zhiqiang Zhang","slug":"zhiqiang-zhang","given":"Zhiqiang","family":"Zhang","affiliation":"Zhejiang University"},{"name":"Chaochao Chen","slug":"chaochao-chen","given":"Chaochao","family":"Chen","affiliation":"Zhejiang University"}],"abstract":"Recently,    representation   learning   on   temporal graphs   has   drawn   increasing   attention,   which aims  at  learning  temporal  patterns  to  characterize the evolving nature of dynamic graphs in real-world  applications.    Despite  effectiveness,  these methods   commonly   ignore   the   individual-   and combinatorial-level patterns derived from different types of interactions (e.g.,user-item), which are at the heart of the representation learning on temporal  graphs.   To  fill  this  gap,  we  propose  MERIT, a novel multi-level graph attention network for inductive representation learning on temporal graphs.We  adaptively  embed  the  original  timestamps  to a higher,  continuous dimensional space for learn-ing  individual-level  periodicity  through  Personalized Time Encoding (PTE) module.  Furthermore, we equip MERIT with Continuous time and Con-text aware Attention (Coco-Attention) mechanism which chronologically locates most relevant neighbors  by  jointly  capturing  multi-level  context  on temporal  graphs.   Finally,  MERIT  performs  multiple aggregations and propagations to explore and exploit high-order structural information for down-stream tasks. Extensive experiments on four public datasets demonstrate the effectiveness of MERITon both (inductive / transductive) link prediction and node classification task.","doi":"10.24963/ijcai.2022/288","url":"https://ijcai.org/proceedings/2022/288","pdf":"https://ijcai.org/proceedings/2022/0288.pdf","markdown":"https://ijcai.org/proceedings/2022/288.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/288","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":288,"pages":{"first":2073,"last":2079},"keywords":["Data Mining: Mining Spatial and/or Temporal Data","Data Mining: Mining Graphs"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986686","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984829","length_sec":184,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bai et al., 2019] Ting Bai, Lixin Zou, Wayne Xin Zhao, Pan Du, Weidong Liu, Jian-Yun Nie, and Ji-Rong Wen. Ctrec: A long-short demands evolution model for continuous- time recommendation. In SIGIR, pages 675-684, 2019.","title":"Ctrec: A long-short demands evolution model for continuoustime recommendation","authors":"Ting Bai, Lixin Zou, Wayne Xin Zhao, Pan Du, Weidong Liu, Jian-Yun Nie, Ji-Rong Wen","year":"2019","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bo et al., 2022] Deyu Bo, BinBin Hu, Xiao Wang, Zhiqiang Zhang, Chuan Shi, and Jun Zhou. Regularizing graph neural networks via consistency-diversity graph augmen- tations. page AAAI, 2022.","title":"Regularizing graph neural networks via consistency-diversity graph augmentations","authors":"Deyu Bo, Binbin Hu, Xiao Wang, Zhiqiang Zhang, Chuan Shi, Jun Zhou","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2018] Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering, 30(9):1616-1637, 2018.","title":"A comprehensive survey of graph embedding: Problems, techniques, and applications","authors":"Hongyun Cai, Vincent W Zheng, Kevin Chen, -Chuan Chang","year":"2018","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Du et al., 2018] Lun Du, Yun Wang, Guojie Song, Zhicong Lu, and Junshan Wang. Dynamic network embedding: An extended approach for skip-gram based network embed- ding. In IJCAI, pages 2086-2092, 2018.","title":"Dynamic network embedding: An extended approach for skip-gram based network embedding","authors":"Lun Du, Yun Wang, Guojie Song, Zhicong Lu, Junshan Wang","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/288"},{"position":5,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In SIGKDD, pages 855-864, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guo et al., 2019] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In AAAI, pages 922-929, 2019.","title":"Attention based spatialtemporal graph convolutional networks for traffic flow forecasting","authors":"Guo","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hamilton et al., 2017] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, pages 1024-1034, 2017.","title":"Inductive representation learning on large graphs","authors":"William L Hamilton, Zhitao Ying, Jure Leskovec","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2020] Hong Huang, Zixuan Fang, Xiao Wang, Youshan Miao, and Hai Jin. Motif-preserving temporal network embedding. In IJCAI, pages 1237-1243, 2020.","title":"Motif-preserving temporal network embedding","authors":"Huang","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/172"},{"position":9,"raw":"[Kazemi et al., 2020] Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain, Ivan Kobyzev, Akshay Sethi, Peter Forsyth, and Pascal Poupart. Representation learning for dynamic graphs: A survey. Journal of Machine Learning Research, 21(70):1-73, 2020.","title":"Representation learning for dynamic graphs: A survey","authors":"Rishab Seyed Mehran Kazemi, Kshitij Goel, Ivan Jain, Akshay Kobyzev, Peter Sethi, Pascal Forsyth, Poupart","year":"2020","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kumar et al., 2019] Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In SIGKDD, pages 1269- 1278, 2019.","title":"Predicting dynamic embedding trajectory in temporal interaction networks","authors":"Srijan Kumar, Xikun Zhang, Jure Leskovec","year":"2019","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2022] Hongrui Liu, Binbin Hu, Xiao Wang, Chuan Shi, Zhiqiang Zhang, and Jun Zhou. Confidence may cheat: Self-training on graph neural networks under distribution shift. page WWW, 2022.","title":"Confidence may cheat: Self-training on graph neural networks under distribution shift","authors":"Hongrui Liu, Binbin Hu, Xiao Wang, Chuan Shi, Zhiqiang Zhang, Jun Zhou","year":"2022","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lu et al., 2019] Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S Yu, and Yanfang Ye. Temporal network embedding with micro-and macro-dynamics. In CIKM, pages 469-478, 2019.","title":"Temporal network embedding with micro-and macro-dynamics","authors":"Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S Yu, Yanfang Ye","year":"2019","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Minh et al., 2006] Ha Quang Minh, Partha Niyogi, and Yuan Yao. Mercer's theorem, feature maps, and smooth- ing. In COLT, pages 154-168, 2006.","title":"Mercer's theorem, feature maps, and smoothing","authors":"Ha Quang, Minh, Partha Niyogi, Yuan Yao","year":"2006","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nguyen et al., 2018] Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. Continuous-time dynamic network em- beddings. In WWW, pages 969-976, 2018.","title":"Continuous-time dynamic network embeddings","authors":"Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, Sungchul Kim","year":"2018","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Seo et al., 2018] Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. Structured sequence modeling with graph convolutional recurrent networks. In NIPS, pages 362-373, 2018.","title":"Structured sequence modeling with graph convolutional recurrent networks","authors":"Seo","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Singer et al., 2019] Uriel Singer, Ido Guy, and Kira Radin- sky. Node embedding over temporal graphs. In IJCAI, pages 4605-4612, 2019.","title":"Node embedding over temporal graphs","authors":"Uriel Singer, Ido Guy, Kira Radinsky","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/640"},{"position":18,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Velickovic et al., 2018] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2016] Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network embedding. In SIGKDD, pages 1225-1234, 2016.","title":"Structural deep network embedding","authors":"Wang","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2021] Xuhong Wang, Ding Lyu, Mengjian Li, Yang Xia, Qi Yang, Xinwen Wang, Xinguang Wang, Ping Cui, Yupu Yang, Bowen Sun, et al. Apan: Asyn- chronous propagation attention network for real-time tem- poral graph embedding. In ICMD, pages 2628-2638, 2021.","title":"Apan: Asynchronous propagation attention network for real-time temporal graph embedding","authors":"Wang","year":"2021","venue":"ICMD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2019] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Self-attention with functional time representation learning. In NIPS, pages 15915-15925, 2019.","title":"Self-attention with functional time representation learning","authors":"Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, Kannan Achan","year":"2019","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2020] Da Xu, Chuanwei Ruan, Evren Körpeoglu, Sushant Kumar, and Kannan Achan. Inductive representa- tion learning on temporal graphs. In ICLR, 2020.","title":"Inductive representation learning on temporal graphs","authors":"Da Xu, Chuanwei Ruan, Evren Körpeoglu, Sushant Kumar, Kannan Achan","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2020] Shuo Yang, Zhiqiang Zhang, Jun Zhou, Yang Wang, Wang Sun, Xingyu Zhong, Yanming Fang, Quan Yu, and Yuan Qi. Financial risk analysis for smes with graph-based supply chain mining. In IJCAI, pages 4661-4667, 2020.","title":"Financial risk analysis for smes with graph-based supply chain mining","authors":"Shuo Yang, Zhiqiang Zhang, Jun Zhou, Yang Wang, Wang Sun, Xingyu Zhong, Yanming Fang, Quan Yu, Yuan Qi","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/643"},{"position":25,"raw":"[Yu et al., 2018] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In IJCAI, pages 3634-3640, 2018.","title":"Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting","authors":"Yu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/505"},{"position":26,"raw":"[Zhang et al., 2018] Ziwei Zhang, Peng Cui, Xiao Wang, Jian Pei, Xuanrong Yao, and Wenwu Zhu. Arbitrary-order proximity preserved network embedding. In SIGKDD, pages 2778-2786, 2018.","title":"Arbitrary-order proximity preserved network embedding","authors":"Ziwei Zhang, Peng Cui, Xiao Wang, Jian Pei, Xuanrong Yao, Wenwu Zhu","year":"2018","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhu et al., 2017] Yu Zhu, Hao Li, Yikang Liao, Beidou Wang, Ziyu Guan, Haifeng Liu, and Deng Cai. What to do next: Modeling user behaviors by time-lstm. In IJCAI, pages 3602-3608, 2017.","title":"What to do next: Modeling user behaviors by time-lstm","authors":"Yu Zhu, Hao Li, Yikang Liao, Beidou Wang, Ziyu Guan, Haifeng Liu, Deng Cai","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/504"},{"position":28,"raw":"[Zuo et al., 2018] Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, and Junjie Wu. Embedding temporal network via neighborhood formation. In SIGKDD, pages 2857-2866, 2018.","title":"Embedding temporal network via neighborhood formation","authors":"Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, Junjie Wu","year":"2018","venue":"SIGKDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}