Proceedings

Learning Bilingual Lexicons Using the Visual Similarity of Labeled Web Images

Download BibTeX Cite

Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. Pages 1764-1768.

https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-296

© 2011 International Joint Conferences on Artificial Intelligence

Abstract

Speakers of many different languages use the Internet. A common activity among these users is uploading images and associating these images with words (in their own language) as captions, filenames, or surrounding text. We use these explicit, monolingual, image-to-word connections to successfully learn implicit, bilingual, word-to-word translations. Bilingual pairs of words are proposed as translations if their corresponding images have similar visual features. We generate bilingual lexicons in 15 language pairs, focusing on words that have been automatically identified as physical objects. The use of visual similarity substantially improves performance over standard approaches based on string similarity: for generated lexicons with 1000 translations, including visual information leads to an absolute improvement in accuracy of 8-12% over string edit distance alone.

  1. [Barnard et al., 2003] K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. JMLR, 3:1107-1135, 2003.
  2. [Brants and Franz, 2009] T. Brants and A. Franz. Web 1T 5- gram, 10 European Languages, Version 1. LDC2009T25, 2009.
  3. [Deng et al., 2009] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, 2009.
  4. [Deselaers et al., 2008] T. Deselaers, D. Keysers, and H. Ney. Features for image retrieval: an experimental com- parison. Information Retrieval, 11:77-107, 2008.
  5. [Etzioni et al., 2007] O. Etzioni, K. Reiter, S. Soderland, and M. Sammer. Lexical translation with application to image search on the web. In MT Summit XI, 2007.
  6. [Feng and Lapata, 2010] Y. Feng and M. Lapata. Topic mod- els for image annotation and text illustration. In HLT- NAACL, 2010.
  7. [Fergus et al., 2005] R. Fergus, L. Fei-Fei, P. Perona, and A. Zisserman. Learning object categories from Google's Image Search. In ICCV, 2005.
  8. [Fung and Yee, 1998] P. Fung and L. Y. Yee. An IR approach for translating new words from nonparallel, comparable texts. In COLING-ACL, 1998.
  9. [Haghighi et al., 2008] A. Haghighi, P. Liang, T. Berg- Kirkpatrick, and D. Klein. Learning bilingual lexicons from monolingual corpora. In ACL-08: HLT, 2008.
  10. [Hodosh et al., 2010] M. Hodosh, P. Young, C. Rashtchian, and J. Hockenmaier. Cross-caption coreference resolution for automatic image understanding. In CoNLL, 2010.
  11. [Klementiev and Roth, 2006] A. Klementiev and D. Roth. Named entity transliteration and discovery from multilin- gual comparable corpora. In HLT-NAACL, 2006.
  12. [Koehn and Knight, 2002] P. Koehn and K. Knight. Learning a translation lexicon from monolingual corpora. In ACL Workshop on Unsupervised Lexical Acquistion, 2002.
  13. [Lavrenko et al., 2003] V. Lavrenko, R. Manmatha, and J. Jeon. A model for learning the semantics of pictures. In NIPS, 2003.
  14. [Lin et al., 2010] D. Lin, K. Church, H. Ji, S. Sekine, D. Yarowsky, S. Bergsma, K. Patil, E. Pitler, R. Lathbury, V. Rao, K. Dalwani, and S. Narsale. New tools for web- scale N-grams. In LREC, 2010.
  15. [Lowe, 2004] D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60:91-110, 2004.
  16. [Rapp, 1999] R. Rapp. Automatic identification of word translations from unrelated English and German corpora. In ACL, 1999.
  17. [Schafer and Yarowsky, 2002] C. Schafer and D. Yarowsky. Inducing translation lexicons via diverse similarity mea- sures and bridge languages. In CoNLL, 2002.
  18. [Snow et al., 2006] R. Snow, D. Jurafsky, and A. Y. Ng. Se- mantic taxonomy induction from heterogenous evidence. In COLING-ACL, 2006.
  19. [Van Durme and Lall, 2010] B. Van Durme and A. Lall. On- line generation of locality sensitive hash signatures. In ACL Short Papers, 2010.
  20. [von Ahn and Dabbish, 2004] L. von Ahn and L. Dabbish. Labeling images with a computer game. In CHI, 2004.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards