Identifying Synonyms among Distributionally Similar Words

1 Introduction

The distributional hypothesis states that words with similar meanings tend to appear in similar contexts [Harris, 1968]. Consider the words adversary and foe. Both of them are often used as the objects of the verbs:

batter, crush, defeat, demonize, deter, outsmart, ...

and modified by the adjectives:

ardent, bitter, formidable, old, tough, worthy, ...

There have been many proposals for computing distribu­ tional similarity of words [Hindle, 1990; Pereira eta/., 1993; Lin, 1998]. The list (1) shows the top-20 distributionally sim­ ilar words of adversary, obtained with Lin's method [Lin, 1998] on a 3GB newspaper corpus.

1. adversary: enemy, foe, ally, antagonist, opponent, rival, detractor, neighbor, supporter, competitor, partner, trading partner, accuser, terrorist, critic, Republican, advocate, skeptic, challenger

Compared with manually compiled thesauri, distribution­ ally similar words often offer much better coverage. Com­ pare ( 1) with the entry for adversary in Webster Collegiate Thesaurus [Kay, 1988]:

1. adversary: Synonyms: opponent, antagonist, anti, con, match, opposer, oppugnant; Related Words: assau1ter, attacker Contrasted Words: backer, supporter, upholder; Antonyms: ally

The thesaurus entry missed many synonyms such as: enemy, foe, rival, competitor and challenger.

A problem with the distributionally similar words, how­ ever, is that many of them are antonyms, e.g., ally and sup­ porter in ( 1 ). The problem gets worse if a word belongs to a semantic category with many members, since all of them tend to have similar distributions. This is demonstrated in

Lijuan Qin and Ming Zhou

Microsoft Research Asia No.49, Zhichun Road Hai Dian District, Beijing, China, 100080

the following list of top-20 distributionally similar words for orange.

1. orange: yellow, lemon, peach, pink, lime, purple, tomato, onion, mango, lavender, avocado, red, pineapple, pear, blue, plum, cucumber, melon, turquoise, tangerine

In many applications, such as information retrieval and ma­ chine translation, the presence of antonyms or other types of semantically incompatible words (e.g. , orange-pink) can be devastating. This paper presents two methods for identifying synonyms among distributionally similar words.

2 Methods

2.1 Patterns of Incompatibility

Consider the following phrasal patterns:

1. a. fromXto Y

b. either X or Y

If two words X and Y appear in one of these patterns, they are very likely to be semantically incompatible. For example, the following table shows the queries and the hits (the number of return documents) from the search engine Alta Vista:

(5) Query

Given a query x NEARy, Alta Vista returns documents where the words x andy appear close to each other. When two words are unrelated, the hits for the NEAR query tend to be low.

Motivated by the above examples, we propose to iden­ tify semantically incompatible word pairs by searching on the Web for instantiations of the patterns in ( 4). We define a score:

where hits(query) is the number of hits returned by AltaVista for the query, P is the set of patterns in ( 4) and E is a small constant to prevent the denominator of the above formula to be 0 (we set f=0.0001). The lower the score, the less likely that the words x andy are synonyms. To determine whether or not distributionally similar words x and y are synonyms, we compute score(x, y) . If the value is higher than 8=2000, ( x, y) is classified as a pair of synonyms.

2.2 Using Bilingual Dictionaries

The second method is based on the observation that transla­ tions of a word from another language are often synonyms of one another. For example, (7) contains the English transla­ tions of the French word defenseur. Many of them are syn­ onyms.

(7) advocate, attorney, counsel, fullback, intercessor, lawyer

When two such words are not synonyms, the reason is typi­ cally that the French word have multiple senses and the En­ glish words are translations of the French word in different senses. Under such circumstances, the distribution of the English words are usually quite different (e.g., lawyer and fullback appear in very different contexts). We can there­ fore identify synonyms of a word w by intersecting the set of words that share with w the same French (or any other lan­ guage) translation and the set of distributionally similar words of w. For example, the top-20 distributionally similar words of lawyer are:

1. lawyer: attorney, counsel, prosecutor, doctor, official, judge, executive, manager, investigator, consultant, aide, agent, physician, expert, banker, officer, politician, lobbyist, teacher, accountant

The intersection of (7) and (8) gives us the synonyms of lawyer: attorney and counsel.

Since this method generally has high precision and low re­ call (see the next section), we can use this method with mul­ tiple bilingual dictionaries separately and take the union of the results. In our experiments, we used 7 dictionaries from http://www.freewaresite.com/onldictJ

English-Swedish, English-Spanish, English­ Japanese, English-German, English-French and English-Esperanto.

3 Evaluation

Using the algorithm in [Lin, 1998] on a 3GB newspaper cor­ pus, we computed the distributional similarity between about 45,000 words. We randomly selected 80 pairs of synonyms and 80 pairs of antonyms from Webster's Collegiate The­ saurus [Kay, 1988] that are also among the top-50 distribu­ tionally similar words of each other. We then used the meth­ ods presented in the previous section to determine which pairs are synonyms. Let S be the set of true synonym pairs and S' be the set of pairs classified as synonyms. The precision and recall measures are defined as follows :

The table in (9) shows the evaluation results:

(9) Method

4 Related Work

The problem we address here is related to semantic orienta­ tion. The semantic orientation of a word is positive (or neg­ ative) if it is generally associated with good (or bad) things. For example, The words simple and simplistic have similar meanings, but simplistic has a negative semantic orientation. The algorithm in [Hatzivassiloglou and McKeown, 1997] is based on the fact that conjoined adjectives generally have the same orientation. They use a small set of adjectives with known orientation to determine the orientations of other ad­ jectives. [Turney, 2002] computed the degree of positive or negative semantic orientation of a word w with the hit counts from Alta Vista for the queries w NEAR excellent and w NEAR poor. While semantic orientation is bipolar, the problem we are dealing with is multipolar. For example, Turney's method is not able to tell that the words red, orange, yellow, green, .. . have incompatible meanings.

5 Conclusion

Distributionally similar words include many antonyms and other semantically incompatible words, which minimizes their use in many applications. We have presented two meth­ ods for identifying synonyms among distributionally similar words. Our preliminary evaluation with known synonyms and antonyms extracted from Webster Collegiate Thesaurus has produced promising results.