{"type":"ScholarlyArticle","title":"Point Set Registration for Unsupervised Bilingual Lexicon Induction","authors":[{"name":"Hailong Cao","slug":"hailong-cao","given":"Hailong","family":"Cao","affiliation":"Harbin Institute of Technology"},{"name":"Tiejun Zhao","slug":"tiejun-zhao","given":"Tiejun","family":"Zhao","affiliation":"Harbin Institute of Technology"}],"abstract":"Inspired by the observation that word embeddings exhibit isomorphic structure across languages, we propose a novel method to induce a bilingual lexicon from only two sets of word embeddings, which are trained on monolingual source and target data respectively. This is achieved by formulating the task as point set registration which is a more general problem. We show that a transformation from the source to the target embedding space can be learned automatically without any form of cross-lingual supervision. By properly adapting a traditional point set registration model to make it be suitable for processing word embeddings, we achieved state-of-the-art performance on the unsupervised bilingual lexicon induction task. The point set registration problem has been well-studied and can be solved by many elegant models, we thus opened up a new opportunity to capture the universal lexical semantic structure across languages.","doi":"10.24963/ijcai.2018/555","url":"https://ijcai.org/proceedings/2018/555","pdf":"https://ijcai.org/proceedings/2018/0555.pdf","markdown":"https://ijcai.org/proceedings/2018/555.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/555","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":555,"pages":{"first":3991,"last":3997},"keywords":["Natural Language Processing: Machine Translation","Machine Learning Applications: Applications of Unsupervised Learning","Natural Language Processing: Embeddings"],"videos":[],"references":[{"position":1,"raw":"[Artetxe et al., 2016] Mikel Artetxe, Gorka Labaka, and Eneko Agirre. Learning principled bilingual mappings of word embeddings while preserving monolingual invari- ance. In EMNLP, pages 2289-2294, 2016.","title":"Learning principled bilingual mappings of word embeddings while preserving monolingual invariance","authors":"Mikel Artetxe, Gorka Labaka, Eneko Agirre","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Artetxe et al., 2017] Mikel Artetxe, Gorka Labaka, and Eneko Agirre. Learning bilingual word embeddings with (almost) no bilingual data. In ACL, pages 451-462, 2017.","title":"Learning bilingual word embeddings with (almost) no bilingual data","authors":"Mikel Artetxe, Gorka Labaka, Eneko Agirre","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bond and Foster, 2013] Francis Bond and Ryan Foster. Linking and extending an open multilingual wordnet. In ACL, pages 1352-1362, 2013.","title":"Francis Bond and Ryan Foster. Linking and extending an open multilingual wordnet","authors":"Foster Bond","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chandar et al., 2014] A. P. Sarath Chandar, Stanislas Lauly, Hugo Larochelle, Mitesh M. Khapra, Balaraman Ravin- dran, Vikas C. Raykar, and Amrita Saha. An autoencoder approach to learning bilingual word representations. In NIPS, pages 1853-1861, 2014.","title":"An autoencoder approach to learning bilingual word representations","authors":"A P Sarath Chandar, Stanislas Lauly, Hugo Larochelle, M Mitesh, Balaraman Khapra, Ravindran, C Vikas, Amrita Raykar, Saha","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collobert and Weston, 2008] Ronan Collobert and Jason Weston. A unified architecture for natural language pro- cessing: Deep neural networks with multitask learning. In ICML, pages 160-167, 2008.","title":"A unified architecture for natural language processing: Deep neural networks with multitask learning","authors":"Weston ; Collobert, Ronan Collobert, Jason Weston","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Conneau et al., 2017] Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data. CoRR, ab- s/1710.04087, 2017.","title":"Word translation without parallel data","authors":"Alexis Conneau, Guillaume Lample, Marc'aurelio Ranzato, Ludovic Denoyer, Hervé Jégou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dou et al., 2015] Qing Dou, Ashish Vaswani, Kevin Knight, and Chris Dyer. Unifying bayesian inference and vector space models for improved decipherment. In ACL-IJCNLP, pages 836-845, 2015.","title":"Unifying bayesian inference and vector space models for improved decipherment","authors":"Qing Dou, Ashish Vaswani, Kevin Knight, Chris Dyer","year":"2015","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Faruqui and Dyer, 2014] Manaal Faruqui and Chris Dyer. Improving vector space word representations using mul- tilingual correlation. In EACL, pages 462-471, 2014.","title":"Improving vector space word representations using multilingual correlation","authors":"Dyer Faruqui, Manaal Faruqui, Chris Dyer","year":"2014","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In NIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gouws et al., 2015] Stephan Gouws, Yoshua Bengio, and Greg Corado. Fast bilingual distributed representations without word alignments. In ICML, pages 748-756, 2015.","title":"Fast bilingual distributed representations without word alignments","authors":"Stephan Gouws, Yoshua Bengio, Greg Corado","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2015] Jiang Guo, Wanxiang Che, David Yarowsky, Haifeng Wang, and Ting Liu. Cross-lingual dependency parsing based on distributed representations. In ACL-IJCNLP, pages 1234-1244, 2015.","title":"Cross-lingual dependency parsing based on distributed representations","authors":"Jiang Guo, Wanxiang Che, David Yarowsky, Haifeng Wang, Ting Liu","year":"2015","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hermann and Blunsom, 2014] Karl Moritz Hermann and Phil Blunsom. Multilingual models for compositional dis- tributed semantics. In ACL, pages 58-68, 2014.","title":"Multilingual models for compositional distributed semantics","authors":"Blunsom Hermann, Karl Moritz Hermann, Phil Blunsom","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Klementiev et al., 2012] Alexandre Klementiev, Ivan Titov, and Binod Bhattarai. Inducing crosslingual distributed representations of words. In COLING, pages 1459-1474, 2012.","title":"Inducing crosslingual distributed representations of words","authors":"Alexandre Klementiev, Ivan Titov, Binod Bhattarai","year":"2012","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luong et al., 2015] Thang Luong, Hieu Pham, and Christo- pher D. Manning. Bilingual word representations with monolingual quality in mind. In NAACL VSM, pages 151- 159, 2015.","title":"Bilingual word representations with monolingual quality in mind","authors":"Luong","year":"2015","venue":"NAACL VSM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Quoc V. Le, and Ilya Sutskever. Exploiting similarities among languages for machine translation. arXiv:1309.4168, abs/1309.4168, 2013.","title":"Exploiting similarities among languages for machine translation","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In NIPS, pages 3111-3119. 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Myronenko and Song, 2010] Andriy Myronenko and Xubo Song. Point set registration: Coherent point drifts. IEEE Transactions on PAMI, 32(12):2262-2275, 2010.","title":"Point set registration: Coherent point drifts","authors":"Andriy Myronenko, Xubo Song","year":"2010","venue":"IEEE Transactions on PAMI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nuhn et al., 2012] Malte Nuhn, Arne Mauser, and Herman- n Ney. Deciphering foreign language by combining lan- guage models and context vectors. In ACL, pages 156- 164, 2012.","title":"Deciphering foreign language by combining language models and context vectors","authors":"Malte Nuhn, Arne Mauser, Hermann Ney","year":"2012","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shi et al., 2015] Tianze Shi, Zhiyuan Liu, Yang Liu, and Maosong Sun. Learning cross-lingual word embeddings via matrix co-factorization. In ACL-IJCNLP, pages 567- 572, 2015.","title":"Learning cross-lingual word embeddings via matrix co-factorization","authors":"Shi","year":"2015","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Smith et al., 2017] Samuel L. Smith, David H. P. Turban, Steven Hamblin, and Nils Y. Hammerla. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. In ICLR, 2017.","title":"Offline bilingual word vectors, orthogonal transformations and the inverted softmax","authors":"L Samuel, Smith, H P David, Steven Turban, Nils Y Hamblin, Hammerla","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Upadhyay et al., 2016] Shyam Upadhyay, Manaal Faruqui, Chris Dyer, and Dan Roth. Cross-lingual models of word embeddings: An empirical comparison. In ACL, pages 1661-1670, 2016.","title":"Cross-lingual models of word embeddings: An empirical comparison","authors":"Shyam Upadhyay, Manaal Faruqui, Chris Dyer, Dan Roth","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vulić and Moens, 2015] Ivan Vulić and Marie-Francine Moens. Bilingual word embeddings from non-parallel document-aligned data applied to bilingual lexicon induc- tion. In ACL-IJCNLP, pages 719-725, 2015.","title":"Bilingual word embeddings from non-parallel document-aligned data applied to bilingual lexicon induction","authors":"Moens Vulić, Ivan Vulić, Marie-Francine Moens","year":"2015","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xing et al., 2015] Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. Normalized word embedding and orthogonal transform for bilingual word translation. In NAACL-HLT, pages 1006-1011, 2015.","title":"Normalized word embedding and orthogonal transform for bilingual word translation","authors":"Chao Xing, Dong Wang, Chao Liu, Yiye Lin","year":"2015","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2014] Jiajun Zhang, Shujie Liu, Mu Li, Ming Zhou, and Chengqing Zong. Bilingually-constrained phrase embeddings for machine translation. In ACL, pages 111-121, 2014.","title":"Bilingually-constrained phrase embeddings for machine translation","authors":"Jiajun Zhang, Shujie Liu, Mu Li, Ming Zhou, Chengqing Zong","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2017] Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. Adversarial training for unsupervised bilingual lexicon induction. In ACL, pages 1959-1970, 2017.","title":"Adversarial training for unsupervised bilingual lexicon induction","authors":"Meng Zhang, Yang Liu, Huanbo Luan, Maosong Sun","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zou et al., 2013] Will Y. Zou, Richard Socher, Daniel Cer, and Christopher D. Manning. Bilingual word embeddings for phrase-based machine translation. In EMNLP, pages 1393-1398, 2013.","title":"Bilingual word embeddings for phrase-based machine translation","authors":"Will Y Zou, Richard Socher, Daniel Cer, Christopher D Manning","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}