Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis

Gabrilovich, Evgeniy; Markovitch, Shaul

Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis

Download BibTeX Cite

Proceedings of the Twentieth International Joint Conference on Artificial Intelligence. Pages 1606-1610.

Abstract

Computing semantic relatedness of natural language texts requires access to vast amounts of common-sense and domain-specific world knowledge. We propose Explicit Semantic Analysis (ESA), a novel method that represents the meaning of texts in a high-dimensional space of concepts derived from Wikipedia. We use machine learning techniques to explicitly represent the meaning of any text as a weighted vector of Wikipedia-based concepts. Assessing the relatedness of texts in this space amounts to comparing the corresponding vectors using conventional metrics (e.g., cosine). Compared with the previous state of the art, using ESA results in substantial improvements in correlation of computed relatedness scores with human judgments: from r=0.56 to 0.75 for individual words and from r=0.60 to 0.72 for texts. Importantly, due to the use of natural concepts, the ESA model is easy to explain to human users.

Evgeniy Gabrilovich, and Shaul Markovitch. 2007. Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, IJCAI-07. IJCAI, 1606–1610. https://ijcai.org/proceedings/2007/259

@inproceedings{ijcai2007p259,
  title     = {Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis},
  author    = {Gabrilovich, Evgeniy and Markovitch, Shaul},
  booktitle = {Proceedings of the Twentieth International Joint Conference on
               Artificial Intelligence, {IJCAI-07}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Ramon López de Mántaras},
  pages     = {1606--1610},
  year      = {2007},
  url       = {https://ijcai.org/proceedings/2007/259},
}

TY  - CPAPER
TI  - Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis
AU  - Gabrilovich, Evgeniy
AU  - Markovitch, Shaul
T2  - Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, IJCAI-07
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2007
SP  - 1606
EP  - 1610
UR  - https://ijcai.org/proceedings/2007/259
ER  -