{"type":"ScholarlyArticle","title":"Triplet Enhanced AutoEncoder: Model-free Discriminative Network Embedding","authors":[{"name":"Yao Yang","slug":"yao-yang","given":"Yao","family":"Yang","affiliation":"Data Mining Lab , University of Electronic Science and Technology of China"},{"name":"Haoran Chen","slug":"haoran-chen","given":"Haoran","family":"Chen","affiliation":"Data Mining Lab , University of Electronic Science and Technology of China"},{"name":"Junming Shao","slug":"junming-shao","given":"Junming","family":"Shao","affiliation":"Data Mining Lab , University of Electronic Science and Technology of China"}],"abstract":"Deep autoencoder is widely used in dimensionality reduction because of the expressive power of the neural network. Therefore, it is naturally suitable for embedding tasks, which essentially compresses high-dimensional information into a low-dimensional latent space. In terms of network representation, methods based on autoencoder such as SDNE and DNGR have achieved comparable results with the state-of-arts. However, all of them do not leverage label information, which leads to the embeddings lack the characteristic of discrimination. In this paper, we present Triplet Enhanced AutoEncoder (TEA), a new deep network embedding approach from the perspective of metric learning. Equipped with the triplet-loss constraint, the proposed approach not only allows capturing the topological structure but also preserving the discriminative information. Moreover, unlike existing discriminative embedding techniques, TEA is independent of any specific classifier, we call it the model-free property. Extensive empirical results on three public datasets (i.e, Cora, Citeseer and BlogCatalog) show that TEA is stable and achieves state-of-the-art performance compared with both supervised and unsupervised network embedding approaches on various percentages of labeled data. The source code can be obtained from https://github.com/yybeta/TEA.","doi":"10.24963/ijcai.2019/745","url":"https://ijcai.org/proceedings/2019/745","pdf":"https://ijcai.org/proceedings/2019/0745.pdf","markdown":"https://ijcai.org/proceedings/2019/745.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/745","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":745,"pages":{"first":5363,"last":5369},"keywords":["Machine Learning Applications: Networks","Natural Language Processing: Embeddings"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2018] Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. A comprehensive survey of graph embedding: Problems, techniques, and applications. TKDE, 30(9):1616-1637, 2018.","title":"A comprehensive survey of graph embedding: Problems, techniques, and applications","authors":"Hongyun Cai, Vincent W Zheng, Kevin Chen, -Chuan Chang","year":"2018","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2015] Shaosheng Cao, Wei Lu, and Qiongkai Xu. Grarep: Learning graph representations with global structural information. In CIKM, pages 891-900, 2015.","title":"Grarep: Learning graph representations with global structural information","authors":"Cao","year":"2015","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2016] Shaosheng Cao, Wei Lu, and Qiongkai Xu. Deep neural networks for learning graph represen- tations. In AAAI, pages 1145-1152, 2016.","title":"Deep neural networks for learning graph representations","authors":"Cao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cheng et al., 2016] De Cheng, Yihong Gong, Sanping Zhou, Jinjun Wang, and Nanning Zheng. Person re- identification by multi-channel parts-based cnn with im- proved triplet loss function. In CVPR, pages 1335-1344, 2016.","title":"Person reidentification by multi-channel parts-based cnn with improved triplet loss function","authors":"De Cheng, Yihong Gong, Sanping Zhou, Jinjun Wang, Nanning Zheng","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ding et al., 2015] Shengyong Ding, Liang Lin, Guangrun Wang, and Hongyang Chao. Deep feature learning with relative distance comparison for person re-identification. Pattern Recognition, 48(10):2993-3003, 2015.","title":"Deep feature learning with relative distance comparison for person re-identification","authors":"Ding","year":"2015","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In KDD, pages 855-864. ACM, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS, pages 1024-1034, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Levy and Goldberg, 2014] Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In NeurIPS, pages 2177-2185, 2014.","title":"Neural word embedding as implicit matrix factorization","authors":"Goldberg Levy, Omer Levy, Yoav Goldberg","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2016] Juzheng Li, Jun Zhu, and Bo Zhang. Dis- criminative deep random walk for network classification. In ACL, pages 1004-1013, 2016.","title":"Discriminative deep random walk for network classification","authors":"Li","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liao et al., 2015] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li. Person re-identification by local maximal occurrence representation and metric learning. In CVPR, pages 2197-2206, 2015.","title":"Person re-identification by local maximal occurrence representation and metric learning","authors":"Shengcai Liao, Yang Hu, Xiangyu Zhu, Stan Z Li","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2017] Hao Liu, Jiashi Feng, Meibin Qi, Jian- guo Jiang, and Shuicheng Yan. End-to-end compara- tive attention networks for person re-identification. TIP, 26(7):3492-3506, 2017.","title":"End-to-end comparative attention networks for person re-identification","authors":"Hao Liu, Jiashi Feng, Meibin Qi, Jianguo Jiang, Shuicheng Yan","year":"2017","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed rep- resentations of words and phrases and their composition- ality. In NeurIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ou et al., 2016] Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. Asymmetric transitivity preserv- ing graph embedding. In KDD, pages 1105-1114. ACM, 2016.","title":"Asymmetric transitivity preserving graph embedding","authors":"Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, Wenwu Zhu","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pan et al., 2016] Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang. Tri-party deep network representation. pages 1895-1901, 2016.","title":"Tri-party deep network representation","authors":"Pan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2016/271"},{"position":16,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In KDD, pages 701-710. ACM, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In CVPR, pages 815-823, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shao et al., 2015] Junming Shao, Zhichao Han, Qinli Yang, and Tao Zhou. Community detection based on distance dynamics. In KDD, pages 1075-1084. ACM, 2015.","title":"Community detection based on distance dynamics","authors":"Junming Shao, Zhichao Han, Qinli Yang, Tao Zhou","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tang et al., 2015] Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large- scale information network embedding. In WWW, pages 1067-1077, 2015.","title":"Line: Largescale information network embedding","authors":"Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, Qiaozhu Mei","year":"2015","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tu et al., 2016] Cunchao Tu, Weicheng Zhang, Zhiyuan Liu, Maosong Sun, et al. Max-margin deepwalk: Discrim- inative learning of network representation. In IJCAI, pages 3889-3895, 2016.","title":"Max-margin deepwalk: Discriminative learning of network representation","authors":"Tu","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/547"},{"position":22,"raw":"[Velickovic et al., 2018] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2016] Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network embedding. In KDD, pages 1225-1234. ACM, 2016.","title":"Structural deep network embedding","authors":"Wang","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang. Community preserv- ing network embedding. In AAAI, pages 203-209, 2017.","title":"Community preserving network embedding","authors":"Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, Shiqiang Yang","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wei et al., 2017] Xiaokai Wei, Linchuan Xu, Bokai Cao, and Philip S Yu. Cross view link prediction by learning noise-resilient representation consensus. In WWW, pages 1611-1619, 2017.","title":"Cross view link prediction by learning noise-resilient representation consensus","authors":"Wei","year":"2017","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Weigend et al., 1991] Andreas S Weigend, David E Rumel- hart, and Bernardo A Huberman. Generalization by weight-elimination with application to forecasting. In NeurIPS, pages 875-882, 1991.","title":"Generalization by weight-elimination with application to forecasting","authors":"Andreas S Weigend, David E Rumelhart, Bernardo A Huberman","year":"1991","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Weinberger and Saul, 2009] Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. JMLR, 10(Feb):207-244, 2009.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"Q Kilian, Lawrence K Weinberger, Saul","year":"2009","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2015] Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. Network represen- tation learning with rich text information. In IJCAI, pages 2111-2117, 2015.","title":"Network representation learning with rich text information","authors":"Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, Edward Y Chang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/299"},{"position":29,"raw":"[Zhang et al., 2016] Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. Homophily, structure, and content augmented network representation learning. In ICDM, pages 609-618. IEEE, 2016.","title":"Homophily, structure, and content augmented network representation learning","authors":"Zhang","year":"2016","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhou et al., 2017] Chang Zhou, Yuqiong Liu, Xiaofei Liu, Zhongyi Liu, and Jun Gao. Scalable graph embedding for asymmetric proximity. In AAAI, pages 2942-2948, 2017.","title":"Scalable graph embedding for asymmetric proximity","authors":"Chang Zhou, Yuqiong Liu, Xiaofei Liu, Zhongyi Liu, Jun Gao","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}