{"type":"ScholarlyArticle","title":"An Adaptive Hierarchical Compositional Model for Phrase Embedding","authors":[{"name":"Bing Li","slug":"bing-li","given":"Bing","family":"Li","affiliation":"† School of Computer Science and Engineering , Northeastern University , China"},{"name":"Xiaochun Yang","slug":"xiaochun-yang","given":"Xiaochun","family":"Yang","affiliation":"† School of Computer Science and Engineering , Northeastern University , China"},{"name":"Bin Wang","slug":"bin-wang","given":"Bin","family":"Wang","affiliation":""},{"name":"Wei Wang","slug":"wei-wang","given":"Wei","family":"Wang","affiliation":"* University of New South Wales , Australia; § Dongguan University of Technology , China"},{"name":"Wei Cui","slug":"wei-cui","given":"Wei","family":"Cui","affiliation":"‡ College of Electrical Engineering and Automation , Shandong University of Science and Technology"},{"name":"Xianchao Zhang","slug":"xianchao-zhang","given":"Xianchao","family":"Zhang","affiliation":"+ School of Software , Dalian University of Technology , China"}],"abstract":"Phrase embedding aims at representing phrases in a vector space and it is important for the performance of many NLP tasks. Existing models only regard a phrase as either full-compositional or non-compositional, while ignoring the hybrid-compositionality that widely exists, especially in long phrases. This drawback prevents them from having a deeper insight into the semantic structure for long phrases and as a consequence, weakens the accuracy of the embeddings. In this paper, we present a novel method for jointly learning compositionality and phrase embedding by adaptively weighting different compositions using an implicit hierarchical structure. Our model has the ability of adaptively adjusting among different compositions without entailing too much model complexity and time cost. To the best of our knowledge, our work is the first effort that considers hybrid-compositionality in phrase embedding. The experimental evaluation demonstrates that our model outperforms state-of-the-art methods in both similarity tasks and analogy tasks.","doi":"10.24963/ijcai.2018/576","url":"https://ijcai.org/proceedings/2018/576","pdf":"https://ijcai.org/proceedings/2018/0576.pdf","markdown":"https://ijcai.org/proceedings/2018/576.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/576","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":576,"pages":{"first":4144,"last":4151},"keywords":["Natural Language Processing: Embeddings","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Baroni and Zamparelli, 2010] Marco Baroni and Roberto Zamparelli. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic s- pace. In EMNLP, pages 1183-1193. ACL, 2010.","title":"Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space","authors":"Zamparelli Baroni, Marco Baroni, Roberto Zamparelli","year":"2010","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Clark and Pulman, 2007] Stephen Clark and Stephen Pul- man. Combining symbolic and distributional models of meaning. In AAAI Spring Symposium, 2007.","title":"Combining symbolic and distributional models of meaning","authors":"Stephen Clark, Stephen Pulman","year":"2007","venue":"AAAI Spring Symposium","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Coecke et al., 2010] Bob Coecke, Mehrnoosh Sadrzadeh, and Stephen Clark. Mathematical foundations for a com- positional distributional model of meaning. arXiv preprint arXiv:1003.4394, 2010.","title":"Mathematical foundations for a compositional distributional model of meaning","authors":"Coecke","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Farahmand et al., 2015] Meghdad Farahmand, Aaron Smith, and Joakim Nivre. A multiword expression data set: Annotating non-compositionality and conven- tionalization for english noun compounds. In MWE@ NAACL-HLT, pages 29-33, 2015.","title":"A multiword expression data set: Annotating non-compositionality and conventionalization for english noun compounds","authors":"Farahmand","year":"2015","venue":"MWE@ NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hashimoto and Tsuruoka, 2016] Kazuma Hashimoto and Yoshimasa Tsuruoka. Adaptive joint learning of composi- tional and non-compositional phrase embeddings. In ACL, pages 205-215, 2016.","title":"Adaptive joint learning of compositional and non-compositional phrase embeddings","authors":"Tsuruoka Hashimoto, Kazuma Hashimoto, Yoshimasa Tsuruoka","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kartsaklis et al., 2014] Dimitri Kartsaklis, Nal Kalchbren- ner, and Mehrnoosh Sadrzadeh. Resolving lexical ambi- guity in tensor regression models of meaning. In ACL, pages 212-217, 2014.","title":"Resolving lexical ambiguity in tensor regression models of meaning","authors":"Dimitri Kartsaklis, Nal Kalchbrenner, Mehrnoosh Sadrzadeh","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Katz and Giesbrecht, 2006] Graham Katz and Eugenie Giesbrecht. Automatic identification of non-compositional multi-word expressions using latent semantic analysis. In COLING, pages 12-19, 2006.","title":"Automatic identification of non-compositional multi-word expressions using latent semantic analysis","authors":"Giesbrecht Katz, Graham Katz, Eugenie Giesbrecht","year":"2006","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kiela and Clark, 2013] Douwe Kiela and Stephen Clark. Detecting compositionality of multi-word expressions us- ing nearest neighbours in vector space models. In EMNLP, pages 1427-1432, 2013.","title":"Detecting compositionality of multi-word expressions using nearest neighbours in vector space models","authors":"Clark ; Kiela, Douwe Kiela, Stephen Clark","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Levy and Goldberg, 2014] Omer Levy and Yoav Goldberg. Dependency-based word embeddings. In ACL, pages 302- 308, 2014.","title":"Dependency-based word embeddings","authors":"Goldberg Levy, Omer Levy, Yoav Goldberg","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Levy et al., 2015] Ran Levy, Liat Ein-Dor, Shay Hummel, Ruty Rinott, and Noam Slonim. Tr9856: A multi-word ter- m relatedness benchmark. In ACL, pages 419-424, 2015.","title":"Tr9856: A multi-word term relatedness benchmark","authors":"Ran Levy, Liat Ein-Dor, Shay Hummel, Ruty Rinott, Noam Slonim","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2013] Peipei Li, Haixun Wang, Kenny Q. Zhu, Zhongyuan Wang, and Xindong Wu. Computing term similarity by large probabilistic isa knowledge. In CIK- M, pages 1401-1410, New York, NY, USA, 2013. ACM.","title":"Computing term similarity by large probabilistic isa knowledge","authors":"Peipei Li, Haixun Wang, Kenny Q Zhu, Zhongyuan Wang, Xindong Wu","year":"2013","venue":"CIK-M","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[MacKay, 2003] David JC MacKay. Information theory, in- ference and learning algorithms. Cambridge university press, 2003.","title":"Information theory, inference and learning algorithms","authors":"J C David, Mackay","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[McCarthy et al., 2007] Diana McCarthy, Sriram Venkatap- athy, and Aravind K Joshi. Detecting compositionality of verb-object combinations using selectional preferences. In EMNLP-CoNLL, pages 369-379, 2007.","title":"Detecting compositionality of verb-object combinations using selectional preferences","authors":"Diana Mccarthy, Sriram Venkatapathy, Aravind K Joshi","year":"2007","venue":"EMNLP-CoNLL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Quoc V Le, and Ilya Sutskever. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168, 2013.","title":"Exploiting similarities among languages for machine translation","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mitchell and Lapata, 2010] Jeff Mitchell and Mirella Lap- ata. Composition in distributional models of semantics. Cognitive science, 34(8):1388-1429, 2010.","title":"Composition in distributional models of semantics","authors":"Lapata Mitchell","year":"2010","venue":"Cognitive science","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Socher et al., 2012] Richard Socher, Brody Huval, Christo- pher D Manning, and Andrew Y Ng. Semantic com- positionality through recursive matrix-vector spaces. In EMNLP-CoNLL, pages 1201-1211. ACL, 2012.","title":"Semantic compositionality through recursive matrix-vector spaces","authors":"Richard Socher, Brody Huval, Christopher D Manning, Andrew Y Ng","year":"2012","venue":"EMNLP-CoNLL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Socher et al., 2013a] Richard Socher, John Bauer, Christo- pher D Manning, et al. Parsing with compositional vector grammars. In ACL, pages 455-465, 2013.","title":"Parsing with compositional vector grammars","authors":"Socher","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Socher et al., 2013b] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for se- mantic compositionality over a sentiment treebank. In EMNLP, pages 1631-1642, 2013.","title":"Recursive deep models for semantic compositionality over a sentiment treebank","authors":"Socher","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tian et al., 2017] Ran Tian, Naoaki Okazaki, and Kentaro Inui. The mechanism of additive composition. Machine Learning, 106(7):1083-1130, 2017.","title":"The mechanism of additive composition","authors":"Ran Tian, Naoaki Okazaki, Kentaro Inui","year":"2017","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Venkatapathy and Joshi, 2005] Sriram Venkatapathy and Aravind K Joshi. Measuring the relative compositional- ity of verb-noun (vn) collocations by integrating features. In HLT-EMNLP, pages 899-906. ACL, 2005.","title":"Measuring the relative compositionality of verb-noun (vn) collocations by integrating features","authors":"Sriram Venkatapathy, Aravind K Joshi","year":"2005","venue":"HLT-EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2017] Shaonan Wang, Jiajun Zhang, and Chengqing Zong. Exploiting word internal structures for generic chinese sentence representation. In EMNLP, pages 298-303, 2017.","title":"Exploiting word internal structures for generic chinese sentence representation","authors":"Wang","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yazdani et al., 2015] Majid Yazdani, Meghdad Farahmand, and James Henderson. Learning semantic composition to detect non-compositionality of multiword expressions. In EMNLP, pages 1733-1742, 2015.","title":"Learning semantic composition to detect non-compositionality of multiword expressions","authors":"Majid Yazdani, Meghdad Farahmand, James Henderson","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yin et al., 2016] Yichun Yin, Furu Wei, Li Dong, Kaimeng Xu, Ming Zhang, and Ming Zhou. Unsupervised word and dependency path embeddings for aspect term extraction. IJCAI, pages 2979-2985, 2016.","title":"Unsupervised word and dependency path embeddings for aspect term extraction","authors":"Yichun Yin, Furu Wei, Li Dong, Kaimeng Xu, Ming Zhang, Ming Zhou","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/423"},{"position":26,"raw":"[Yu and Dredze, 2015] Mo Yu and Mark Dredze. Learning composition models for phrase embeddings. TACL, 3:227- 242, 2015.","title":"Learning composition models for phrase embeddings","authors":"Mo Yu, Mark Dredze","year":"2015","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2014] Jiajun Zhang, Shujie Liu, Mu Li, Ming Zhou, Chengqing Zong, et al. Bilingually-constrained phrase embeddings for machine translation. In ACL, pages 111-121, 2014.","title":"Bilingually-constrained phrase embeddings for machine translation","authors":"Jiajun Zhang, Shujie Liu, Mu Li, Ming Zhou, Chengqing Zong","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}