---
title: "Learning Bilingual Lexicons Using the Visual Similarity of Labeled Web Images"
authors:
  - "Shane Bergsma"
  - "Benjamin Van Durme"
doi: "10.5591/978-1-57735-516-8/IJCAI11-296"
url: "https://ijcai.org/proceedings/2011/296"
pdf: "https://ijcai.org/proceedings/2011/0296.pdf"
venue: "Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence"
year: 2011
pages: "1764-1768"
track: ""
keywords:
---

# Learning Bilingual Lexicons Using the Visual Similarity of Labeled Web Images

Shane Bergsma, Benjamin Van Durme

*Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence*, pages 1764-1768. DOI: [10.5591/978-1-57735-516-8/IJCAI11-296](https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-296)

## Abstract

Speakers of many different languages use the Internet. A common activity among these users is uploading images and associating these images with words (in their own language) as captions, filenames, or surrounding text. We use these explicit, monolingual, image-to-word connections to successfully learn implicit, bilingual, word-to-word translations. Bilingual pairs of words are proposed as translations if their corresponding images have similar visual features. We generate bilingual lexicons in 15 language pairs, focusing on words that have been automatically identified as physical objects. The use of visual similarity substantially improves performance over standard approaches based on string similarity: for generated lexicons with 1000 translations, including visual information leads to an absolute improvement in accuracy of 8-12% over string edit distance alone.