{"type":"ScholarlyArticle","title":"Exploiting Mutual Information for Substructure-aware Graph Representation Learning","authors":[{"name":"Pengyang Wang","slug":"pengyang-wang","given":"Pengyang","family":"Wang","affiliation":"University of Central Florida"},{"name":"Yanjie Fu","slug":"yanjie-fu","given":"Yanjie","family":"Fu","affiliation":"University of Central Florida"},{"name":"Yuanchun Zhou","slug":"yuanchun-zhou","given":"Yuanchun","family":"Zhou","affiliation":"Computer Network Information Center , Chinese Academy of Sciences"},{"name":"Kunpeng Liu","slug":"kunpeng-liu","given":"Kunpeng","family":"Liu","affiliation":"University of Central Florida"},{"name":"Xiaolin Li","slug":"xiaolin-li","given":"Xiaolin","family":"Li","affiliation":"Nanjing University"},{"name":"Kien Hua","slug":"kien-hua","given":"Kien","family":"Hua","affiliation":"University of Central Florida"}],"abstract":"In this paper, we design and evaluate a new substructure-aware Graph Representation Learning (GRL) approach. GRL aims to map graph structure information into low-dimensional representations. While extensive efforts have been made for modeling global and/or local structure information, GRL can be improved by substructure information. Some recent studies exploit adversarial learning to incorporate substructure awareness, but hindered by unstable convergence. This study will address the major research question: is there a better way to integrate substructure awareness into GRL? As subsets of the graph structure, interested substructures (i.e., subgraph) are unique and representative for differentiating graphs, leading to the high correlation between the representation of the graph-level structure and substructures. Since mutual information (MI) is to evaluate the mutual dependence between two variables, we develop a MI inducted substructure-aware GRL method. We decompose the GRL pipeline into two stages: (1) node-level, where we introduce to maximize MI between the original and learned representation by the intuition that the original and learned representation should be highly correlated; (2) graph-level, where we preserve substructures by maximizing MI between the graph-level structure and substructure representation. Finally, we present extensive experimental results to demonstrate the improved performances of our method with real-world data.","doi":"10.24963/ijcai.2020/472","url":"https://ijcai.org/proceedings/2020/472","pdf":"https://ijcai.org/proceedings/2020/0472.pdf","markdown":"https://ijcai.org/proceedings/2020/472.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/472","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":472,"pages":{"first":3415,"last":3421},"keywords":["Data Mining: Applications","Data Mining: Mining Spatial, Temporal Data","Multidisciplinary Topics and Applications: Recommender Systems"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24968/file_24968.m3u8","length_sec":233,"provider":"hls"}],"references":[{"position":1,"raw":"[Belghazi et al., 2018] Mohamed Ishmael Belghazi, Aris- tide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Ben- gio, Aaron Courville, and R Devon Hjelm. Mine: mutual information neural estimation. arXiv preprint arXiv:1801.04062, 2018.","title":"Mohamed Ishmael Belghazi, Aristide Baratin","authors":"Belghazi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bernardo et al., 2007] JM Bernardo, MJ Bayarri, JO Berger, AP Dawid, D Heckerman, AFM Smith, and M West. Generative or discriminative? getting the best of both worlds. Bayesian statistics, 8(3):3-24, 2007.","title":"Generative or discriminative? getting the best of both worlds","authors":"Bernardo, J O Bayarri, Berger, D Dawid, Heckerman, M Smith, West","year":"2007","venue":"Bayesian statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2007] Deng Cai, Xiaofei He, and Jiawei Han. Spectral regression: A unified subspace learning frame- work for content-based image retrieval. In Proceedings of the 15th ACM international conference on Multimedia, pages 403-412. ACM, 2007.","title":"Spectral regression: A unified subspace learning framework for content-based image retrieval","authors":"Deng Cai, Xiaofei He, Jiawei Han","year":"2007","venue":"Proceedings of the 15th ACM international conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2018] Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering, 30(9):1616-1637, 2018.","title":"A comprehensive survey of graph embedding: Problems, techniques, and applications","authors":"Hongyun Cai, Vincent W Zheng, Kevin Chen, -Chuan Chang","year":"2018","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2015] Shaosheng Cao, Wei Lu, and Qiongkai Xu. Grarep: Learning graph representations with global structural information. In Proceedings of the 24th ACM in- ternational on conference on information and knowledge management, pages 891-900. ACM, 2015.","title":"Grarep: Learning graph representations with global structural information","authors":"Cao","year":"2015","venue":"Proceedings of the 24th ACM international on conference on information and knowledge management","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fu et al., 2019] Yanjie Fu, Pengyang Wang, Jiadi Du, Le Wu, and Li Xiaolin. Efficient region embedding with multi-view spatial networks: A perspective of locality- constrained spatial autocorrelations. In Proceedings of the 33th AAAI Conference on Artificial Intelligence, page to appear. AAAI, 2019.","title":"Efficient region embedding with multi-view spatial networks: A perspective of localityconstrained spatial autocorrelations","authors":"Yanjie Fu, Pengyang Wang, Jiadi Du, Le Wu, Li Xiaolin","year":"2019","venue":"Proceedings of the 33th AAAI Conference on Artificial Intelligence, page to appear","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hjelm et al., 2018] R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representa- tions by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670, 2018.","title":"Learning deep representations by mutual information estimation and maximization","authors":"Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, Yoshua Bengio","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kipf and Welling, 2016a] Thomas N Kipf and Max Welling. Semi-supervised classification with graph con- volutional networks. arXiv preprint arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"Welling ; Kipf, N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kipf and Welling, 2016b] Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308, 2016.","title":"Variational graph auto-encoders","authors":"Welling ; Kipf, N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2018] Kunpeng Liu, Pengyang Wang, Jiawei Zhang, Yanjie Fu, and Sajal K Das. Modeling the inter- action coupling of multi-view spatiotemporal contexts for destination prediction. In Proceedings of the 2018 SIAM International Conference on Data Mining, pages 171-179.","title":"Modeling the interaction coupling of multi-view spatiotemporal contexts for destination prediction","authors":"Kunpeng Liu, Pengyang Wang, Jiawei Zhang, Yanjie Fu, Das","year":null,"venue":"Proceedings of the 2018 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119, 2013. Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social repre- sentations. In Proceedings of the 20th ACM SIGKDD in- ternational conference on Knowledge discovery and data mining, pages 701-710. ACM, 2014.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Shervashidze et al., 2011] Nino Shervashidze, Pascal Schweitzer, Erik Jan van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(Sep):2539- 2561, 2011.","title":"Weisfeiler-lehman graph kernels","authors":"Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, Karsten M Borgwardt","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Veličković et al., 2018] Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. arXiv preprint arXiv:1809.10341, 2018.","title":"Deep graph infomax","authors":"Petar Veličković, William Fedus, Pietro William L Hamilton, Yoshua Liò, Devon Bengio, Hjelm","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Wang et al., 2018a] Pengyang Wang, Yanjie Fu, Jiawei Zhang, Pengfei Wang, Yu Zheng, and Charu Aggarwal. You are how you drive: Peer and temporal-aware repre- sentation learning for driving behavior analysis. In Pro- ceedings of the 24th ACM SIGKDD International Con- ference on Knowledge Discovery & Data Mining, pages 2457-2466. ACM, 2018.","title":"You are how you drive: Peer and temporal-aware representation learning for driving behavior analysis","authors":"Wang","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wang et al., 2018b] Pengyang Wang, Jiawei Zhang, Guan- nan Liu, Yanjie Fu, and Charu Aggarwal. Ensemble- spotting: Ranking urban vibrancy via poi embedding with multi-view spatial graphs. In Proceedings of the 2018 SIAM International Conference on Data Mining, pages 351-359. SIAM, 2018.","title":"Ensemblespotting: Ranking urban vibrancy via poi embedding with multi-view spatial graphs","authors":"Wang","year":"2018","venue":"Proceedings of the 2018 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2019] Pengyang Wang, Yanjie Fu, Hui Xiong, and Xiaolin Li. Adversarial substructured representation learning for mobile user profiling. In Proceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, pages 130-138. ACM, 2019.","title":"Adversarial substructured representation learning for mobile user profiling","authors":"Wang","year":"2019","venue":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Yanardag and Vishwanathan, 2015] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1365-1374. ACM, 2015.","title":"Pinar Yanardag and SVN Vishwanathan. Deep graph kernels","authors":"Vishwanathan Yanardag","year":"2015","venue":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yang et al., 2014] Dingqi Yang, Daqing Zhang, Vincent W Zheng, and Zhiyong Yu. Modeling user activity prefer- ence by leveraging user spatial temporal characteristics in lbsns. IEEE Transactions on Systems, Man, and Cybernet- ics: Systems, 45(1):129-142, 2014.","title":"Modeling user activity preference by leveraging user spatial temporal characteristics in lbsns","authors":"Yang","year":"2014","venue":"IEEE Transactions on Systems, Man, and Cybernetics: Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhang and Wang, 2016] Qing Zhang and Houfeng Wang. Not all links are created equal: An adaptive embedding approach for social personalized ranking. In Proceedings of the 39th International ACM SIGIR conference on Re- search and Development in Information Retrieval, pages 917-920. ACM, 2016.","title":"Not all links are created equal: An adaptive embedding approach for social personalized ranking","authors":"Wang ; ] Qing Zhang, Houfeng Zhang, Wang","year":"2016","venue":"Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang et al., 2019] Yunchao Zhang, Pengyang Wang, Xi- aolin Li, Yu Zheng, and Yanjie Fu. Unifying inter-region autocorrelation and intra-region structures for spatial em- bedding via collective adversarial learning. In Proceed- ings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.","title":"Unifying inter-region autocorrelation and intra-region structures for spatial embedding via collective adversarial learning","authors":"Zhang","year":"2019","venue":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}