Learning Bilingual Lexicons Using the Visual Similarity of Labeled Web Images

Bergsma, Shane; Van Durme, Benjamin

doi:10.5591/978-1-57735-516-8/IJCAI11-296

Learning Bilingual Lexicons Using the Visual Similarity of Labeled Web Images

Shane Bergsma, Benjamin Van Durme

Download BibTeX Cite

Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. Pages 1764-1768.

https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-296

Abstract

Speakers of many different languages use the Internet. A common activity among these users is uploading images and associating these images with words (in their own language) as captions, filenames, or surrounding text. We use these explicit, monolingual, image-to-word connections to successfully learn implicit, bilingual, word-to-word translations. Bilingual pairs of words are proposed as translations if their corresponding images have similar visual features. We generate bilingual lexicons in 15 language pairs, focusing on words that have been automatically identified as physical objects. The use of visual similarity substantially improves performance over standard approaches based on string similarity: for generated lexicons with 1000 translations, including visual information leads to an absolute improvement in accuracy of 8-12% over string edit distance alone.

Shane Bergsma, and Benjamin Van Durme. 2011. Learning Bilingual Lexicons Using the Visual Similarity of Labeled Web Images. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI-11. IJCAI, 1764–1768. https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-296

@inproceedings{ijcai2011p296,
  title     = {Learning Bilingual Lexicons Using the Visual Similarity of Labeled Web Images},
  author    = {Bergsma, Shane and Van Durme, Benjamin},
  booktitle = {Proceedings of the Twenty-Second International Joint Conference on
               Artificial Intelligence, {IJCAI-11}},
  publisher = {AAAI Press/International Joint Conferences on Artificial Intelligence},
  editor    = {Toby Walsh},
  pages     = {1764--1768},
  year      = {2011},
  month     = {7},
  doi       = {10.5591/978-1-57735-516-8/IJCAI11-296},
  url       = {https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-296},
}

TY  - CPAPER
TI  - Learning Bilingual Lexicons Using the Visual Similarity of Labeled Web Images
AU  - Bergsma, Shane
AU  - Van Durme, Benjamin
T2  - Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI-11
PB  - AAAI Press/International Joint Conferences on Artificial Intelligence
PY  - 2011
SP  - 1764
EP  - 1768
DO  - 10.5591/978-1-57735-516-8/IJCAI11-296
UR  - https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-296
ER  -