{"type":"ScholarlyArticle","title":"Inverted Bilingual Topic Models for Lexicon Extraction from Non-parallel Data","authors":[{"name":"Tengfei Ma","slug":"tengfei-ma","given":"Tengfei","family":"Ma","affiliation":"Research Center"},{"name":"Tetsuya Nasukawa","slug":"tetsuya-nasukawa","given":"Tetsuya","family":"Nasukawa","affiliation":"IBM Research-Tokyo"}],"abstract":"Topic models have been successfully applied in lexicon extraction. However, most previous methods are limited to document-aligned data. In this paper, we try to address two challenges of applying topic models to lexicon extraction in non-parallel data: 1) hard to model the word relationship and 2) noisy seed dictionary. To solve these two challenges, we propose two new bilingual topic models to better capture the semantic information of each word while discriminating the multiple translations in a noisy seed dictionary. We extend the scope of topic models by inverting the roles of \"word\" and \"document\". In addition, to solve the problem of noise in seed dictionary, we incorporate the probability of translation selection in our models. Moreover, we also propose an effective measure to evaluate the similarity of words in different languages and select the optimal translation pairs. Experimental results using real world data demonstrate the utility and efficacy of the proposed models.","doi":"10.24963/ijcai.2017/569","url":"https://ijcai.org/proceedings/2017/569","pdf":"https://ijcai.org/proceedings/2017/0569.pdf","markdown":"https://ijcai.org/proceedings/2017/569.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/569","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":569,"pages":{"first":4075,"last":4081},"keywords":["Uncertainty in AI: Graphical Models","Natural Language Processing: Information Extraction"],"videos":[],"references":[{"position":1,"raw":"[Andrade et al., 2010] Daniel Andrade, Tetsuya Nasukawa, and Jun'ichi Tsujii. Robust measurement and comparison of context similarity for finding translation pairs. In Pro- ceedings of the 23rd International Conference on Compu- tational Linguistics, pages 19-27. Association for Compu- tational Linguistics, 2010.","title":"Robust measurement and comparison of context similarity for finding translation pairs","authors":"Daniel Andrade, Tetsuya Nasukawa, Jun'ichi Tsujii","year":"2010","venue":"Proceedings of the 23rd International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blei et al., 2003] David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993-1022, 2003.","title":"Latent dirichlet allocation","authors":"Andrew Y David M Blei, Michael I Ng, Jordan","year":"2003","venue":"Journal of machine Learning research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bollegala et al., 2015] Danushka Bollegala, Georgios Kon- tonatsios, and Sophia Ananiadou. A cross-lingual simi- larity measure for detecting biomedical term translations. PloS one, 10(6):e0126196, 2015.","title":"A cross-lingual similarity measure for detecting biomedical term translations","authors":"Danushka Bollegala, Georgios Kontonatsios, Sophia Ananiadou","year":"2015","venue":"PloS one","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boyd-Graber and Blei, 2009] Jordan Boyd-Graber and David M Blei. Multilingual topic models for unaligned text. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pages 75-82. AUAI Press, 2009.","title":"Multilingual topic models for unaligned text","authors":"Boyd-Graber Blei, ; ] Jordan Boyd-Graber, David M Blei","year":"2009","venue":"Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chang and Blei, 2009] Jonathan Chang and Daokvid M Blei. Relational topic models for document networks. In International conference on artificial intelligence and statistics, pages 81-88, 2009.","title":"Relational topic models for document networks","authors":"Chang, Blei, Jonathan Chang, Daokvid M Blei","year":"2009","venue":"International conference on artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dietz et al., 2007] Laura Dietz, Steffen Bickel, and Tobias Scheffer. Unsupervised prediction of citation influences. In Proceedings of the 24th international conference on Machine learning, pages 233-240. ACM, 2007.","title":"Unsupervised prediction of citation influences","authors":"Laura Dietz, Steffen Bickel, Tobias Scheffer","year":"2007","venue":"Proceedings of the 24th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duong et al., 2016] Long Duong, Hiroshi Kanayama, Tengfei Ma, Steven Bird, and Trevor Cohn. Learn- ing crosslingual word embeddings without bilingual corpora. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, page 12851295, 2016.","title":"Learning crosslingual word embeddings without bilingual corpora","authors":"Duong","year":"2016","venue":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fung and Yee, 1998] Pascale Fung and Lo Yuen Yee. An ir approach for translating new words from nonparallel, comparable texts. In Proceedings of the 17th interna- tional conference on Computational linguistics-Volume 1, pages 414-420. Association for Computational Linguis- tics, 1998.","title":"An ir approach for translating new words from nonparallel, comparable texts","authors":"Pascale Fung, Lo Yuen, Yee","year":"1998","venue":"Proceedings of the 17th international conference on Computational linguistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gouws and Søgaard, 2015] Stephan Gouws and Anders Søgaard. Simple task-specific bilingual word embeddings. In Proceedings of NAACL-HLT, pages 1386-1390, 2015.","title":"Simple task-specific bilingual word embeddings","authors":"Søgaard Gouws, Stephan Gouws, Anders Søgaard","year":"2015","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Haghighi et al., 2008] Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, and Dan Klein. Learning bilingual lexi- cons from monolingual corpora. ACL-08: HLT, page 771, 2008.","title":"Learning bilingual lexicons from monolingual corpora","authors":"Haghighi","year":"2008","venue":"ACL-08: HLT","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Laws et al., 2010] Florian Laws, Lukas Michelbacher, Beate Dorow, Christian Scheible, Ulrich Heid, and Hin- rich Schütze. A linguistically grounded graph model for bilingual lexicon extraction. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 614-622. Association for Computational Linguistics, 2010.","title":"A linguistically grounded graph model for bilingual lexicon extraction","authors":"Laws","year":"2010","venue":"Proceedings of the 23rd International Conference on Computational Linguistics: Posters","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lopez, 2008] Adam Lopez. Statistical machine translation. ACM Computing Surveys (CSUR), 40(3):8, 2008.","title":"Statistical machine translation","authors":"Adam Lopez","year":"2008","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Quoc V Le, and Ilya Sutskever. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168, 2013.","title":"and Ilya Sutskever. Exploiting similarities among languages for machine translation","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mimno et al., 2009] David Mimno, Hanna M Wallach, Ja- son Naradowsky, David A Smith, and Andrew McCallum. Polylingual topic models. In Proceedings of the 2009 Con- ference on Empirical Methods in Natural Language Pro- cessing: Volume 2-Volume 2, pages 880-889. Association for Computational Linguistics, 2009.","title":"Polylingual topic models","authors":"David Mimno, Jason Hanna M Wallach, David A Naradowsky, Andrew Smith, Mccallum","year":"2009","venue":"Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ni et al., 2009] Xiaochuan Ni, Jian-Tao Sun, Jian Hu, and Zheng Chen. Mining multilingual topics from wikipedia. In Proceedings of the 18th international conference on World wide web, pages 1155-1156. ACM, 2009.","title":"Mining multilingual topics from wikipedia","authors":"Xiaochuan Ni, Jian-Tao Sun, Jian Hu, Zheng Chen","year":"2009","venue":"Proceedings of the 18th international conference on World wide web","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pekar et al., 2006] Viktor Pekar, Ruslan Mitkov, Dimitar Blagoev, and Andrea Mulloni. Finding translations for low-frequency words in comparable corpora. Machine Translation, 20(4):247-266, 2006.","title":"Finding translations for low-frequency words in comparable corpora","authors":"Ruslan Viktor Pekar, Dimitar Mitkov, Andrea Blagoev, Mulloni","year":"2006","venue":"Machine Translation","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rapp, 1995] Reinhard Rapp. Identifying word translations in non-parallel texts. In Proceedings of the 33rd an- nual meeting on Association for Computational Linguis- tics, pages 320-322. Association for Computational Lin- guistics, 1995.","title":"Identifying word translations in non-parallel texts","authors":"Reinhard Rapp, Rapp","year":"1995","venue":"Proceedings of the 33rd annual meeting on Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shezaf and Rappoport, 2010] Daphna Shezaf and Ari Rap- poport. Bilingual lexicon generation using non-aligned signatures. In Proceedings of the 48th annual meeting of the association for computational linguistics, pages 98- 107. Association for Computational Linguistics, 2010.","title":"Daphna Shezaf and Ari Rappoport. Bilingual lexicon generation using non-aligned signatures","authors":"Rappoport Shezaf","year":"2010","venue":"Proceedings of the 48th annual meeting of the association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Søgaard et al., 2015] Anders Søgaard, Željko Agić, Héctor Martínez Alonso, Barbara Plank, Bernd Bohnet, and Anders Johannsen. Inverted indexing for cross-lingual nlp. In The 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference of the Asian Federation of Natural Language Processing (ACL-IJCNLP 2015), 2015.","title":"Inverted indexing for cross-lingual nlp","authors":"Søgaard","year":"2015","venue":"The 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference of the Asian Federation of Natural Language Processing (ACL-IJCNLP 2015)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tamura et al., 2012] Akihiro Tamura, Taro Watanabe, and Eiichiro Sumita. Bilingual lexicon extraction from compa- rable corpora using label propagation. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natu- ral Language Processing and Computational Natural Lan- guage Learning, pages 24-36. Association for Computa- tional Linguistics, 2012.","title":"Bilingual lexicon extraction from comparable corpora using label propagation","authors":"Akihiro Tamura, Taro Watanabe, Eiichiro Sumita","year":"2012","venue":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vulic and Moens, 2013] Ivan Vulic and Marie-Francine Moens. Cross-lingual semantic similarity of words as the similarity of their semantic word responses. In Proceed- ings of the 2013 Conference of the North American Chap- ter of the Association for Computational Linguistics: Hu- man Language Technologies (NAACL-HLT 2013), pages 106-116. ACL, 2013.","title":"Ivan Vulic and Marie-Francine Moens. Cross-lingual semantic similarity of words as the similarity of their semantic word responses","authors":"Moens Vulic","year":"2013","venue":"Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2013)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vulić et al., 2011] Ivan Vulić, Wim De Smet, and Marie- Francine Moens. Identifying word translations from com- parable corpora using latent topic models. In Proceedings of the 49th Annual Meeting of the Association for Com- putational Linguistics: Human Language Technologies: short papers-Volume 2, pages 479-484. Association for Computational Linguistics, 2011.","title":"Identifying word translations from comparable corpora using latent topic models","authors":"Ivan Vulić, Wim De Smet, Marie-Francine Moens","year":"2011","venue":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2017] Meng Zhang, Haoruo Peng3 Yang Liu, and Huanbo Luan1 Maosong Sun. Bilingual lexicon in- duction from non-parallel data with minimal supervision. 2017.","title":"Bilingual lexicon induction from non-parallel data with minimal supervision","authors":"Meng Zhang, Haoruo, Yang Liu, Huanbo Luan1, Maosong Sun","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}