{"type":"ScholarlyArticle","title":"Network Embedding with Dual Generation Tasks","authors":[{"name":"Jie Liu","slug":"jie-liu","given":"Jie","family":"Liu","affiliation":"Nankai University , Tianjin , China"},{"name":"Na Li","slug":"na-li","given":"Na","family":"Li","affiliation":"Nankai University , Tianjin , China"},{"name":"Zhicheng He","slug":"zhicheng-he","given":"Zhicheng","family":"He","affiliation":"Nankai University , Tianjin , China"}],"abstract":"We study the problem of Network Embedding (NE) for content-rich networks. NE models aim to learn efficient low-dimensional dense vectors for network vertices which are crucial to many network analysis tasks. The core problem of content-rich network embedding is to learn and integrate the semantic information conveyed by network structure and node content. In this paper, we propose a general end-to-end model, Dual GEnerative Network Embedding (DGENE), to leverage the complementary information of network structure and content. In this model, each vertex is regarded as an object with two modalities: node identity and textual content. Then we formulate two dual generation tasks. One is Node Identification (NI) which recognizes nodes’ identities given their contents. Inversely, the other one is Content Generation (CG) which generates textual contents given the nodes’ identities. We develop specific Content2Node and Node2Content models for the two tasks. Under the DGENE framework, the two dual models are learned by sharing and integrating intermediate layers, with which they mutually enhance each other. Extensive experimental results show that our model yields a significant performance gain compared to the state-of-the-art NE methods. Moreover, our model has an interesting and useful byproduct, that is, a component of our model can generate texts, which is potentially useful for many tasks.","doi":"10.24963/ijcai.2019/709","url":"https://ijcai.org/proceedings/2019/709","pdf":"https://ijcai.org/proceedings/2019/0709.pdf","markdown":"https://ijcai.org/proceedings/2019/709.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/709","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":709,"pages":{"first":5102,"last":5108},"keywords":["Machine Learning: Learning Generative Models","Natural Language Processing: Natural Language Generation","Machine Learning: Unsupervised Learning","Machine Learning Applications: Networks"],"videos":[],"references":[{"position":1,"raw":"[Cao et al., 2015] Shaosheng Cao, Wei Lu, and Qiongkai Xu. Grarep: Learning graph representations with global structural information. In CIKM, pages 891-900, 2015.","title":"Grarep: Learning graph representations with global structural information","authors":"Cao","year":"2015","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2016] Jifan Chen, Qi Zhang, and Xuanjing Huang. Incorporate group information to enhance network embedding. In CIKM, pages 1901-1904, 2016.","title":"Incorporate group information to enhance network embedding","authors":"Chen","year":"2016","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Elman, 1990] Jeffrey L. Elman. Finding structure in time. Cognitive Science, 14(2):179-211, 1990.","title":"Finding structure in time","authors":"Jeffrey L Elman","year":"1990","venue":"Cognitive Science","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Graves, 2013] Alex Graves. Generating sequences with re- current neural networks. CoRR, abs/1308.0850, 2013.","title":"Generating sequences with recurrent neural networks","authors":"Alex Graves","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In KDD, pages 855-864, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma. Dual learn- ing for machine translation. In NIPS, pages 820-828, 2016.","title":"Dual learning for machine translation","authors":"Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, Wei-Ying Ma","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. In EMNLP, pages 1746-1751, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2015] Jiwei Li, Minh-Thang Luong, and Dan Ju- rafsky. A hierarchical neural autoencoder for paragraphs and documents. In ACL, pages 1106-1115, 2015.","title":"A hierarchical neural autoencoder for paragraphs and documents","authors":"Jiwei Li, Minh-Thang Luong, Dan Jurafsky","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2018] Jie Liu, Zhicheng He, Lai Wei, and Yalou Huang. Content to node: Self-translation network embed- ding. In KDD, pages 1794-1802, 2018.","title":"Content to node: Self-translation network embedding","authors":"Jie Liu, Zhicheng He, Lai Wei, Yalou Huang","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Luong et al., 2015] Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. Multi-task sequence to sequence learning. CoRR, abs/1511.06114, 2015.","title":"Multi-task sequence to sequence learning","authors":"Minh-Thang Luong, Quoc V Le, Ilya Sutskever, Oriol Vinyals, Lukasz Kaiser","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ma and Hovy, 2016] Xuezhe Ma and Eduard H. Hovy. End- to-end sequence labeling via bi-directional lstm-cnns-crf. In ACL, pages 1064-1074, 2016.","title":"Xuezhe Ma and Eduard H. Hovy. Endto-end sequence labeling via bi-directional lstm-cnns-crf","authors":"Hovy Ma","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: online learning of social rep- resentations. In KDD, pages 701-710, 2014.","title":"Deepwalk: online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Shi et al., 2018] Yu Shi, Qi Zhu, Fang Guo, Chao Zhang, and Jiawei Han. Easing embedding learning by compre- hensive transcription of heterogeneous information net- works. In KDD, pages 2190-2199, 2018.","title":"Easing embedding learning by comprehensive transcription of heterogeneous information networks","authors":"Yu Shi, Qi Zhu, Fang Guo, Chao Zhang, Jiawei Han","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sun et al., 2016] Xiaofei Sun, Jiang Guo, Xiao Ding, and Ting Liu. A general framework for content-enhanced network representation learning. CoRR, abs/1610.02906, 2016.","title":"A general framework for content-enhanced network representation learning","authors":"Xiaofei Sun, Jiang Guo, Xiao Ding, Ting Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tan et al., 2017] Jiwei Tan, Xiaojun Wan, and Jianguo Xiao. Abstractive document summarization with a graph- based attentional neural model. In ACL, pages 1171-1181, 2017.","title":"Abstractive document summarization with a graphbased attentional neural model","authors":"Tan","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tang et al., 2015a] Jian Tang, Meng Qu, and Qiaozhu Mei. PTE: predictive text embedding through large-scale het- erogeneous text networks. In KDD, pages 1165-1174, 2015.","title":"PTE: predictive text embedding through large-scale heterogeneous text networks","authors":"Tang","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tang et al., 2015b] Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. LINE: large- scale information network embedding. In WWW, pages 1067-1077, 2015.","title":"LINE: largescale information network embedding","authors":"Tang","year":"2015","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tang et al., 2017] Duyu Tang, Nan Duan, Tao Qin, and Ming Zhou. Question answering and question generation as dual tasks. CoRR, abs/1706.02027, 2017.","title":"Question answering and question generation as dual tasks","authors":"Tang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tu et al., 2016] Cunchao Tu, Weicheng Zhang, Zhiyuan Liu, and Maosong Sun. Max-margin deepwalk: Discrimi- native learning of network representation. In IJCAI, pages 3889-3895, 2016.","title":"Max-margin deepwalk: Discriminative learning of network representation","authors":"Tu","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tu et al., 2017] Cunchao Tu, Han Liu, Zhiyuan Liu, and Maosong Sun. CANE: context-aware network embedding for relation modeling. In ACL, pages 1722-1731, 2017.","title":"CANE: context-aware network embedding for relation modeling","authors":"Tu","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vinyals et al., 2015] Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey E. Hinton. Grammar as a foreign language. In NIPS, pages 2773- 2781, 2015.","title":"Grammar as a foreign language","authors":"Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, Geoffrey E Hinton","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017a] Suhang Wang, Charu C. Aggarwal, Jil- iang Tang, and Huan Liu. Attributed signed network em- bedding. In CIKM, pages 137-146, 2017.","title":"Attributed signed network embedding","authors":"Wang","year":"2017","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2017b] Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang. Community preserv- ing network embedding. In AAAI, pages 203-209, 2017.","title":"Community preserving network embedding","authors":"Wang","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Williams and Zipser, 1989] Ronald J. Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2):270- 280, 1989.","title":"A learning algorithm for continually running fully recurrent neural networks","authors":"Ronald J Williams, David Zipser","year":"1989","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xia et al., 2017] Yingce Xia, Tao Qin, Wei Chen, Jiang Bian, Nenghai Yu, and Tie-Yan Liu. Dual supervised learning. In ICML, pages 3789-3798, 2017.","title":"Dual supervised learning","authors":"Yingce Xia, Tao Qin, Wei Chen, Jiang Bian, Nenghai Yu, Tie-Yan Liu","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2015] Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y. Chang. Network represen- tation learning with rich text information. In IJCAI, pages 2111-2117, 2015.","title":"Network representation learning with rich text information","authors":"Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, Edward Y Chang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang et al., 2016] Zichao Yang, Diyi Yang, Chris Dyer, Xi- aodong He, Alexander J. Smola, and Eduard H. Hovy. Hi- erarchical attention networks for document classification. In NAACL, pages 1480-1489, 2016.","title":"Hierarchical attention networks for document classification","authors":"Yang","year":"2016","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yu et al., 2018] Wenchao Yu, Wei Cheng, Charu C. Aggar- wal, Kai Zhang, Haifeng Chen, and Wei Wang. Netwalk: A flexible deep embedding approach for anomaly detec- tion in dynamic networks. In KDD, pages 2672-2681, 2018.","title":"Netwalk: A flexible deep embedding approach for anomaly detection in dynamic networks","authors":"Wenchao Yu, Wei Cheng, Charu C Aggarwal, Kai Zhang, Haifeng Chen, Wei Wang","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}