{"type":"ScholarlyArticle","title":"Enhancing Semantic Representations of Bilingual Word Embeddings with Syntactic Dependencies","authors":[{"name":"Linli Xu","slug":"linli-xu","given":"Linli","family":"Xu","affiliation":"Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology , University of Science and Technology of China"},{"name":"Wenjun Ouyang","slug":"wenjun-ouyang","given":"Wenjun","family":"Ouyang","affiliation":"Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology , University of Science and Technology of China"},{"name":"Xiaoying Ren","slug":"xiaoying-ren","given":"Xiaoying","family":"Ren","affiliation":"Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology , University of Science and Technology of China"},{"name":"Yang Wang","slug":"yang-wang","given":"Yang","family":"Wang","affiliation":"Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology , University of Science and Technology of China"},{"name":"Liang Jiang","slug":"liang-jiang","given":"Liang","family":"Jiang","affiliation":"Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology , University of Science and Technology of China"}],"abstract":"Cross-lingual representation is a technique that can both represent different languages in the same latent vector space and enable the knowledge transfer across languages. To learn such representations, most of existing works require parallel sentences with word-level alignments and assume that aligned words have similar Bag-of-Words (BoW) contexts.  However, due to differences in grammar structures among different languages, the contexts of aligned words in different languages may appear at different positions of the sentence. To address this issue of different syntactics across different languages, we propose a model of bilingual word embeddings integrating syntactic dependencies (DepBiWE) by producing dependency parse-trees which encode the accurate relative positions for the contexts of aligned words. In addition, a new method is proposed to learn bilingual word embeddings from dependency-based contexts and BoW contexts jointly. Extensive experimental results on a real world dataset clearly validate the superiority of the proposed model DepBiWE on various natural language processing (NLP) tasks.","doi":"10.24963/ijcai.2018/628","url":"https://ijcai.org/proceedings/2018/628","pdf":"https://ijcai.org/proceedings/2018/0628.pdf","markdown":"https://ijcai.org/proceedings/2018/628.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/628","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":628,"pages":{"first":4517,"last":4524},"keywords":["Natural Language Processing: Text Classification","Natural Language Processing: Embeddings","Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[AP et al., 2014] Sarath Chandar AP, Stanislas Lauly, Hugo Larochelle, Mitesh Khapra, Balaraman Ravindran, Vikas C Raykar, and Amrita Saha. An autoencoder approach to learning bilingual word representations. In Advances in Neural Information Processing Systems, pages 1853-1861, 2014.","title":"An autoencoder approach to learning bilingual word representations","authors":"Sarath Chandar, A P, Stanislas Lauly, Hugo Larochelle, Mitesh Khapra, Balaraman Ravindran, Amrita Vikas C Raykar, Saha","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2003] Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural proba- bilistic language model. Journal of Machine Learning Re- search, 3(Feb):1137-1155, 2003.","title":"A neural probabilistic language model","authors":"Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Jauvin","year":"2003","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bond and Foster, 2013] Francis Bond and Ryan Foster. Linking and extending an open multilingual wordnet. In ACL (1), pages 1352-1362, 2013.","title":"Francis Bond and Ryan Foster. Linking and extending an open multilingual wordnet","authors":"Foster Bond","year":"2013","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Camacho-Collados et al., 2015] José Camacho-Collados, Mohammad Taher Pilehvar, and Roberto Navigli. A framework for the construction of monolingual and cross-lingual word similarity datasets. In ACL (2), pages 1-7, 2015.","title":"José Camacho-Collados, Mohammad Taher Pilehvar, and Roberto Navigli. A framework for the construction of monolingual and cross-lingual word similarity datasets","authors":"Camacho-Collados","year":"2015","venue":"ACL (2)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collobert and Weston, 2008] Ronan Collobert and Jason Weston. A unified architecture for natural language pro- cessing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Ma- chine Learning, pages 160-167, 2008.","title":"A unified architecture for natural language processing: Deep neural networks with multitask learning","authors":"Weston ; Collobert, Ronan Collobert, Jason Weston","year":"2008","venue":"Proceedings of the 25th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[De Marneffe and Manning, 2008] Marie-Catherine De Marneffe and Christopher D Manning. The stan- ford typed dependencies representation. In Coling 2008: proceedings of the workshop on cross-framework and cross-domain parser evaluation, pages 1-8. Association for Computational Linguistics, 2008.","title":"The stanford typed dependencies representation","authors":"Marie-Catherine De, Marneffe, Christopher D Manning","year":"2008","venue":"Coling 2008: proceedings of the workshop on cross-framework and cross-domain parser evaluation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duong et al., 2016] Long Duong, Hiroshi Kanayama, Tengfei Ma, Steven Bird, and Trevor Cohn. Learning crosslingual word embeddings without bilingual corpora. arXiv preprint arXiv:1606.09403, 2016.","title":"Learning crosslingual word embeddings without bilingual corpora","authors":"Duong","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dyer et al., 2013] Chris Dyer, Victor Chahuneau, and Noah A Smith. A simple, fast, and effective reparame- terization of ibm model 2. Association for Computational Linguistics, 2013.","title":"A simple, fast, and effective reparameterization of ibm model 2","authors":"Chris Dyer, Victor Chahuneau, Noah A Smith","year":"2013","venue":"Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Faruqui and Dyer, 2014] Manaal Faruqui and Chris Dyer. Improving vector space word representations using mul- tilingual correlation. Association for Computational Lin- guistics, 2014.","title":"Improving vector space word representations using multilingual correlation","authors":"Dyer Faruqui, Manaal Faruqui, Chris Dyer","year":"2014","venue":"Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gouws and Søgaard, 2015] Stephan Gouws and Anders Søgaard. Simple task-specific bilingual word embeddings. In HLT-NAACL, pages 1386-1390, 2015.","title":"Simple task-specific bilingual word embeddings","authors":"Søgaard Gouws, Stephan Gouws, Anders Søgaard","year":"2015","venue":"HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gouws et al., 2015] Stephan Gouws, Yoshua Bengio, and Greg Corrado. Bilbowa: Fast bilingual distributed rep- resentations without word alignments. In Proceedings of the 32nd International Conference on Machine Learning, pages 748-756, 2015.","title":"Bilbowa: Fast bilingual distributed representations without word alignments","authors":"Stephan Gouws, Yoshua Bengio, Greg Corrado","year":"2015","venue":"Proceedings of the 32nd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hermann and Blunsom, 2014] Karl Moritz Hermann and Phil Blunsom. Multilingual models for compositional dis- tributed semantics. arXiv preprint arXiv:1404.4641, 2014.","title":"Multilingual models for compositional distributed semantics","authors":"Blunsom Hermann, Karl Moritz Hermann, Phil Blunsom","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Iacobacci et al., 2015] Ignacio Iacobacci, Moham- mad Taher Pilehvar, and Roberto Navigli. Sensem- bed: Learning sense embeddings for word and relational similarity. In The Meeting of the Association for Computa- tional Linguistics and the International Joint Conference on Natural Language Processing, pages 86-91, 2015.","title":"Sensembed: Learning sense embeddings for word and relational similarity","authors":"Ignacio Iacobacci, Mohammad Taher Pilehvar, Roberto Navigli","year":"2015","venue":"The Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Klementiev et al., 2012] Alexandre Klementiev, Ivan Titov, and Binod Bhattarai. Inducing crosslingual distributed representations of words. 2012.","title":"Inducing crosslingual distributed representations of words","authors":"Alexandre Klementiev, Ivan Titov, Binod Bhattarai","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Koehn, 2005] Philipp Koehn. Europarl: A parallel corpus for statistical machine translation. In MT summit, vol- ume 5, pages 79-86, 2005.","title":"Europarl: A parallel corpus for statistical machine translation","authors":"Philipp Koehn, Koehn","year":"2005","venue":"MT summit","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Levy and Goldberg, 2014] Omer Levy and Yoav Goldberg. Dependency-based word embeddings. In ACL (2), pages 302-308, 2014.","title":"Dependency-based word embeddings","authors":"Goldberg Levy, Omer Levy, Yoav Goldberg","year":"2014","venue":"ACL (2)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Levy et al., 2015] Omer Levy, Yoav Goldberg, and Ido Da- gan. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Asso- ciation for Computational Linguistics, 3:211-225, 2015.","title":"Improving distributional similarity with lessons learned from word embeddings","authors":"Omer Levy, Yoav Goldberg, Ido Dagan","year":"2015","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lewis et al., 2004] David D Lewis, Yiming Yang, Tony G Rose, and Fan Li. Rcv1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5(Apr):361-397, 2004.","title":"Rcv1: A new benchmark collection for text categorization research","authors":"Yiming David D Lewis, Tony G Yang, Fan Rose, Li","year":"2004","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Luong et al., 2015] Thang Luong, Hieu Pham, and Christo- pher D Manning. Bilingual word representations with monolingual quality in mind. In HLT-NAACL, pages 151- 159, 2015.","title":"Bilingual word representations with monolingual quality in mind","authors":"Luong","year":"2015","venue":"HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Quoc V Le, and Ilya Sutskever. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168, 2013.","title":"Exploiting similarities among languages for machine translation","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mikolov et al., 2013c] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Raganato et al., 2016] Alessandro Raganato, Claudio Delli Bovi, and Roberto Navigli. Automatic construction and evaluation of a large semantically enriched wikipedia. In International Joint Conference on Artificial Intelligence, pages 2894-2900, 2016.","title":"Automatic construction and evaluation of a large semantically enriched wikipedia","authors":"Alessandro Raganato, Claudio Delli Bovi, Roberto Navigli","year":"2016","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/411"},{"position":25,"raw":"[Shi et al., 2015] Tianze Shi, Zhiyuan Liu, Yang Liu, and Maosong Sun. Learning cross-lingual word embeddings via matrix co-factorization. In ACL (2), pages 567-572, 2015.","title":"Learning cross-lingual word embeddings via matrix co-factorization","authors":"Tianze Shi, Zhiyuan Liu, Yang Liu, Maosong Sun","year":"2015","venue":"ACL (2)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Socher et al., 2013] Richard Socher, John Bauer, Christo- pher D Manning, and Andrew Y Ng. Parsing with com- positional vector grammars. In ACL (1), pages 455-465, 2013.","title":"Parsing with compositional vector grammars","authors":"Richard Socher, John Bauer, Christopher D Manning, Andrew Y Ng","year":"2013","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Upadhyay et al., 2016] Shyam Upadhyay, Manaal Faruqui, Chris Dyer, and Roth Dan. Cross-lingual models of word embeddings: An empirical comparison. In Meeting of the Association for Computational Linguistics, pages 1661- 1670, 2016.","title":"Cross-lingual models of word embeddings: An empirical comparison","authors":"Shyam Upadhyay, Manaal Faruqui, Chris Dyer, Roth Dan","year":"2016","venue":"Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yin et al., 2016] Yichun Yin, Furu Wei, Li Dong, Kaimeng Xu, Ming Zhang, and Ming Zhou. Unsupervised word and dependency path embeddings for aspect term extrac- tion. In International Joint Conference on Artificial Intel- ligence, pages 2979-2985, 2016.","title":"Unsupervised word and dependency path embeddings for aspect term extraction","authors":"Yichun Yin, Furu Wei, Li Dong, Kaimeng Xu, Ming Zhang, Ming Zhou","year":"2016","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/423"},{"position":29,"raw":"[Zhou et al., 2015] Guangyou Zhou, Tingting He, Jun Zhao, and Wensheng Wu. A subspace learning framework for cross-lingual sentiment classification with partial parallel data. In International Joint Conference on Artificial Intel- ligence, pages 1426-1433, 2015.","title":"A subspace learning framework for cross-lingual sentiment classification with partial parallel data","authors":"Zhou","year":"2015","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/205"},{"position":30,"raw":"[Zou et al., 2013] Will Y Zou, Richard Socher, Daniel Cer, and Christopher D Manning. Bilingual word embeddings for phrase-based machine translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1393-1398, 2013.","title":"Bilingual word embeddings for phrase-based machine translation","authors":"Will Y Zou, Richard Socher, Daniel Cer, Christopher D Manning","year":"2013","venue":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}