Positive, Negative, or Neutral: Learning an Expanded Opinion Lexicon from Emoticon-Annotated Tweets

1 Introduction

The language used in Twitter 1 provides substantial challenges for sentiment analysis. The words used in this platform include many abbreviations, acronyms, and misspelled words that are not observed in traditional media or covered by popular lexicons. The diversity and sparseness of these informal words make the manual creation of a Twitter-oriented opinion lexicon a time-consuming task.

In this article we propose a supervised framework for opinion lexicon expansion for the language used in Twitter. Taking SentiWordnet as inspiration, each word in our expanded lexicon has a probability distribution, indicating how positive, negative, and neutral it is. The estimated probabilities can be used to represent intensities for a specific sentiment category e.g., the word awesome is more positive than the word adequate. Furthermore, the neutral dimension may be useful for discarding non-opinion words in text-level polarity classification tasks. In contrast, unsupervised lexicon expansion techniques such as semantic orientation [Turney, 2002] provide a single numerical score for each word, and it is unclear how to impose thresholds on this score for neutrality detection.

All the entries in our lexicon are associated with a corresponding part-of-speech tag. By relying on POS-tagged words, homographs 2 with different POS-tags will be disambiguated [Wilks and Stevenson, 1998]. For instance, the word apple will receive different sentiment scores when used to refer to a common noun (a fruit) or a proper noun (a company).

1 http://www.twitter.com

2 Words that share the same spelling but have different meanings.

The proposed methodology takes a stream of tweets noisily labelled according to their polarity using emoticon-based annotation. In this approach, only tweets with positive or negative emoticons are considered and labelled according to the polarity indicated by the emoticon. This idea has been widely used before to train message-level sentiment classifiers [Bifet and Frank, 2010], [Go et al., 2009].

We calculate two types of word-level attributes from the stream of annotated tweets to train a word classifier: Semantic Orientation (SO) [Turney, 2002], which is based on the mutual information between word and sentiment class, and Stochastic Gradient Descent (SGD) score, which learns a linear relationship between word and sentiment class. Additionally, we consider syntactic information of the word in its context by including the POS tag of the word as a nominal attribute.

To train a word-level sentiment classifier using supervised learning, we also need sentiment labels for the words. These labels are provided by a seed lexicon taken from the union of four different hand-made lexicons after discarding all polarity clashes from the intersection.

To the best of our knowledge, this is the first article in which the lexicon expansion of Twitter opinion words using POS disambiguation and supervised learning is studied and evaluated. Additionally, this is the first study in which scores for positive, negative, and neutral sentiment are provided for Twitter-specific expressions.

We test our approach on two collections of automatically labelled tweets. The results indicate that our supervised framework outperforms semantic orientation when the detection of neutral words is considered. We also evaluate the usefulness of the expanded lexicon for message-level polarity classification of tweets, showing significant improvements in performance.

This article is organised as follows. In Section 2 we provide a review of existing work in opinion lexicon expansion. In Section 3 we describe the seed lexicon used to label the words for the training set. In Section 4 we explain the mechanisms studied to automatically create collections of labelled tweets. The creation of our word-level time-series is described in Section 5, together with the features used for training the classifier. In Section 6 we present the experiments we conducted to evaluate the proposed approach and discuss results. The main findings and conclusions are discussed in Section 7.

2 RelatedWork

There are two types of resources that can be exploited for lexicon expansion: thesauri and document collections. The simplest approach using a thesaurus such as WordNet 3 is to expand a seed lexicon of labelled opinion words using synonyms and antonyms from the lexical relations provided by the thesaurus [Hu and Liu, 2004], [Kim and Hovy, 2004]. The hypothesis behind this approach is that synonyms have the same polarity and antonyms have the opposite. This process is normally iterated several times. In [Esuli and Sebastiani, 2005], a supervised classifier was trained using a seed of labelled words that was obtained through synonyms and antonyms expansion. For each word, a vector space model is created from the definition or gloss provided by the WordNet dictionary. This representation is used to train a word-level classifier that is used for lexicon expansion. An equivalent approach was applied later to create SentiWordnet 4 in [Esuli and Sebastiani, 2006], [Baccianella et al., 2010]. In SentiWordNet, each WordNet synset or group of synonyms is assigned into classes positive, negative and neutral in the range [0, 1].

A limitation of thesaurus-based approaches is their inability to capture domain-dependent words. Corpus-based approaches exploit syntactic or co-occurrence patterns to expand the lexicon to the words found within a collection of documents. In [Turney, 2002], the expansion is done through a measure referred to as semantic orientation, which is based on the the point-wise mutual information (PMI) between two random variables:

The semantic orientation of a word is the difference between the PMI of the word with a positive emotion and a negative emotion. Different ways have been proposed to represent the joint probabilities of words and emotions. In Turney's work [Turney, 2002], they are estimated using the number of hits returned by a search engine in response to a query composed of the target word together with the word "excellent" and another query using the word "poor".

The same idea was used for Twitter lexicon expansion in [Becker et al., 2013], [Mohammad et al., 2013], [Zhou et al., 2014], which all model the joint probabilities from collections of tweets labelled in automatic ways. In [Becker et al., 2013], the tweets are labelled with a trained classifier using thresholds for the different classes to ensure high precision. In [Zhou et al., 2014], they are labelled with emoticons to create domain-specific lexicons. In [Mohammad et al., 2013], they are labelled with emoticons and hashtags associated with emotions to create two different lexicons. These lexicons were tested for tweet-level polarity classification.

3 http://wordnet.princeton.edu/

4 http://sentiwordnet.isti.cnr.it/

3 Ground-TruthWord Polarities

In this section, we describe the seed lexicon used to label the training dataset for our word sentiment classifier. We create it by taking the union of the following manually created lexical resources:

MPQA Subjectivity Lexicon: This lexicon was created by Wilson et al. [Wilson et al., 2005] and is part of OpinionFinder 5 , a system that automatically detects subjective sentences in document corpora. The lexicon has positive, negative, and neutral words.

Bing Liu: This lexicon is maintained and distributed by Bing Liu 6 and was used in several of his papers [Liu, 2012]. It has positive and negative entries.

Afinn: This strength-oriented lexicon [Nielsen, 2011] has positive words scored from 1 to 5 and negative words scored from -1 to -5. It includes slang, obscene words, acronyms and Web jargon. We tagged words with negative and positive scores to negative and positive classes respectively.

NRC emotion Lexicon: This emotion-oriented lexicon [Mohammad and Turney, 2013] was created by conducting a tagging process on the crowdsourcing Amazon Mechanical Turk platform. In this lexicon, the words are annotated according to eight emotions: joy, trust, sadness, anger, surprise, fear, anticipation, and disgust, and two polarity classes: positive and negative. There are many words that are not associated with any emotional state and are tagged as neutral. In this work, we consider positive, negative, and neutral tags from this lexicon.

As we need to reduce the noise in our training data, we discard all words where a polarity clash is observed. A polarity clash is a word that receives two or more different tags in the union of lexicons. The number of words for the different polarity classes in the different lexicons is displayed in Table 1.

From the table, we can observe that the number of words per class is significantly reduced after removing the clashes from the union. The total number of clashes is 1074.

This high number of clashes found among different handmade lexicons indicates two things: 1) Different human annotators can disagree when tagging a word to polarity classes. 2) There are several words that can belong to more than one sentiment class. Due to this, we can say that word-level polarity classification is a hard and subjective problem.

5 http://mpqa.cs.pitt.edu/opinionfinder/opinionfinder 2/

6 http://www.cs.uic.edu/∼liub/FBS/sentiment-analysis.html

4 Obtaining Labelled Tweets

In order to calculate the attributes for our word-level classifier, we require a collection of time-stamped tweets with their corresponding polarity labels.

We rely on the emoticon-based annotation approach in which tweets exhibiting positive :) and negative :( emoticons are labelled according to the emoticon's polarity [Go et al., 2009]. Afterwards, the emoticon used to label the passage is removed from the content.

We consider two collections of tweets covering multiple topics: The Edinburgh corpus (ED) [Petrovi´c et al., 2010], and the Stanford Sentiment corpus (STS) [Go et al., 2009].

The ED corpus has 97 million tweets which were collected using the Twitter streaming API in a period spanning November 11th 2009 until February 1st 2010. This collection includes tweets in multiple languages. As was done in [Bifet and Frank, 2010], non-English tweets were filtered out, and tweets without emoticons were discarded.

The STS corpus was created by periodically sending queries :) and :( to the Twitter search API between April 6th 2009 to June 25th 2009. All the tweets in this collection are written in English.

The number of tweets for each polarity class in the two corpora is given in Table 2. We can observe that when using the streaming API (ED), positive emoticons occur much more frequently than negative ones.

5 Word-level Features

All the tweets from the annotated collection are lowercased, tokenised and POS-tagged. We use the TweetNLP library [Gimpel et al., 2011], which provides a tokeniser and a tagger specifically for the language used in Twitter. We prepend a POS-tag prefix to each word in order to differentiate homographs exhibiting different POS-tags.

To calculate the proposed features, we treat the time-sorted collection of tweets as a data stream and create two timeseries for each POS-tagged word observed in the vocabulary: the SGD series, and the SO series. These time-series intend to capture the evolution of the relationship between a word and the sentiment that it expresses.

The first time-series is calculated by incrementally training a linear support vector machine using stochastic gradient descent (SGD) [Zhang, 2004]. The weights of this linear model correspond to POS-tagged words and are updated in an incremental fashion. We optimise the hinge loss function with an L2 penalty and a learning rate equal to 0.1:

The variables w, b, and λ correspond to the weight vector, the bias, and the regularisation parameter, respectively. The class labels y are assumed to be in {+1,-1}, corresponding to positively and negatively labelled tweets, respectively. The regularisation parameter was set to 0.0001. The model's weights determine how strongly the presence of a word influences the prediction of negative and positive classes [Bifet and Frank, 2010]. The SGD time-series is created by applying this learning process to a collection of labelled tweets and storing the word's coefficients in different time windows. We use time windows of 1, 000 examples.

The second time-series corresponds to the accumulated semantic orientation (SO) introduced in Section 2. Let count be a function that counts the number of times that a word or a sentiment label has been observed during a certain period of time. We calculate the SO score for each POS-tagged word in an accumulated way according to the following expression:

We use time windows of 1, 000 examples and the Laplace correction to avoid the zero-frequency problem.

We rely on our time-series to extract features that are used to train our world-level polarity classifier. These features summarise location and dispersion properties of the timeseries, and are listed in Table 3. Location-oriented features mean, trimm.mean and median measure the central tendency of the time-series. The feature last.element corresponds to the last value observed in the time-series. This attribute would be equivalent to the traditional semantic orientation measure for the SO time-series. The features sd, iqr, sg, and sg.diff measure the level of dispersion of the time-series.

In addition to these time-series features, we include the POS-tag of the word as a nominal attribute. We include this attribute based on the hypothesis that non-neutral words are more likely to exhibit certain POS tags than neutral ones [Zhou et al., 2014].

6 Experiments

In this section, we present our experimental results for Twitter lexicon expansion. In the first part, we study the word-level polarity classification problem. In the second part, we expand the seed lexicon using the trained classifier and use it for message-level polarity classification of tweets.

6.1 Word-level polarity classification

We calculated the time-series described in Section 5 for the most frequent 10, 000 POS-tagged words found in each of our two datasets. The time-series were calculated using MOA 7 , a data stream mining framework.

To create training and test data for machine learning, all the POS-tagged words matching the seed lexicon are labelled according to the lexicon's polarities. It is interesting to consider how frequently positive, negative, and neutral words occur in a collection of tweets. The number of words labelled as positive, negative, and neutral for both the ED and STS datasets is given in Table 4. As shown in the table, neutral words are the most frequent words in both datasets. Moreover, positive words are more frequent than negative ones.

Next, we focus on the word-level classification problem. With the aim of gaining a better understanding of the problem, we study three word-level classification problems:

1. Neutrality: Classify words as neutral (objective) or nonneutral (subjective). We label positive and negative words as non-neutral for this task.
2. PosNeg: Classify words to positive or negative classes. We remove all neutral words for this task.
3. Polarity: Classify words to classes positive, negative or neutral. This is the primary classification problem we aim to solve.

We study the information provided by each feature with respect to the three classification tasks described above. This is done by calculating the information gain of each feature. This score is normally used for decision tree learning and measures the reduction of entropy within each class after performing the best split induced by the feature. The information gain obtained for the different attributes in relation to the three classification tasks is shown in Table 5. The attributes achieving the highest information gain per task are marked in bold.

We can observe that variables measuring the location of the SO and SGD time-series tend to be more informative than the ones measuring dispersion. Moreover, the information gain of these variables is much higher for PosNeg than for neutrality. SGD and SO are competitive measures for neutrality, but SO is better for PosNeg. An interesting insight is that features that measure the central tendency of the time-series tend to be more informative than the last values, especially for SGD. These measures smooth the fluctuations of the SGD time-series. We can see that the feature sgd.mean is the best attribute for neutrality classification in both datasets. We can also see that POS tags are useful for neutrality detection, but useless for PosNeg. Therefore, we can conclude that positive and negative words have a similar distribution of POS tags.

7 http://moa.cs.waikato.ac.nz/

We trained supervised classifiers for the three different classification problems in both datasets STS and ED. The classification experiments were performed using WEKA 8 , a machine learning environment. We studied the following learning algorithms in preliminary experiments: RBF SVM, Logistic regression, C4.5, and Random Forest. As the RBF SVM produced the best performance among the different methods, we used this method in our classification experiments with a nested grid search procedure for parameter tuning where internal cross-validation is used to find C and γ.

The evaluation was done using 10 times 10-folds-crossvalidation and different subsets of attributes were compared. All the methods are compared with the baseline of using the last value of the semantic orientation, based on a corrected resampled paired t-student test with an α level of 0.05 [Nadeau and Bengio, 2003]. We used the following subsets of attributes: 1) SO: Includes only the feature SO.last. This is the baseline and is equivalent to the standard semantic orientation measure with the decision boundaries provided by the SVM. 2) ALL: Includes all the features. 3)SGD.TS+POS: Includes all the features from the SGD time-series and the POS tag. 4)SO.TS+POS: Includes all the features from the SO time-series and the POS tag. 5)SO+POS: Includes the features so.last and the POS tag.

We evaluate the weighted area under the ROC curves (AUCs) (to deal with class imbalances) for the four different subsets of attributes in the two datasets. The classification results are presented in Table 6. The symbols ◦ and · correspond to statistically significant improvements and degradations with respect to the baseline, respectively.

We can observe a much lower performance in Neutrality detection than in PosNeg. This indicates that the detection of neutral Twitter words is much harder than distinguishing between positive and negative words. The performance on both datasets tends to be similar. However, the results for STS are better than for ED. This suggests that a collection of balanced positively and negatively labelled tweets may be more suitable for lexicon expansion. Another result is that the combination of all features leads to a significant improvement over the baseline for neutrality and polarity classification. In the PosNeg classification task, we can see that the baseline is very strong. This suggests that SO is very good for discriminating between positive and negative words, but not strong enough when neutral words are included. Regarding SO and SGD time-series, we can conclude that they are competitive for neutrality detection. However, SO-based features are better for PosNeg and Polarity tasks.

6.2 Lexicon expansion

The ultimate goal of the polarity-classification of words is to produce a Twitter-oriented opinion lexicon emulating the properties of SentiWordet, i.e., a lexicon of POS-tagged disambiguated entries with their corresponding distribution for positive, negative, and neutral classes. To do this, we fit a logistic regression model to the output of the support vector machine trained for the polarity problem using all the attributes. The resulting model is then used to classify the remaining unlabelled words. This process is performed for both STS and ED datasets.

8 http://www.cs.waikato.ac.nz/ml/weka/

A sample from the expanded word list is given in Table 7. We can see that each entry has the following attributes: the word, the POS-tag, the sentiment label that corresponds to the class with maximum probability, and the distribution. We inspected the expanded lexicon and observed that the estimated probabilities are intuitively plausible. However, there are some words for which the estimated distribution is questionable, such as the word same in Table 7.

The provided probabilities can also be used to explore the sentiment intensities of words. In Figure 1, we visualise the expanded lexicon intensities of words classified as positive and negative through word clouds. The sizes of the words are proportional to the log odds ratios log2( P(pos) P (neg) ) and log2( P(neg) P(pos) ) for positive and negative words, respectively.

Next, we study the usefulness of our expanded lexicons based on ED and STS for Twitter polarity classification. This involves categorising entire tweets into positive or negative sentiment classes.

The experiments were performed on three collections of tweets that were manually assigned to positive and negative classes. The first collection is 6HumanCoded 9 , which was used to evaluate SentiStrength [Thelwall et al., 2012]. In this dataset, tweets are scored according to positive and negative numerical scores. We use the difference of these scores to create polarity classes and discard messages where it is equal to zero. The other datasets are Sanders 10 , and SemEval 11 . The number of positive and negative tweets per corpus is given in Table 8.

We train a logistic regression on the labelled collections of tweets based on simple count-based features extracted using the lexicons. We compute features in the following manner. We count the number of positive and negative words from the seed lexicon matching the content of the tweet. From the expanded lexicons we create a positive and a negative score. The positive score is calculated by adding the positive probabilities of POS-tagged words labelled as positive within the tweet's content. The negative score is calculated in an analogous way from the negative probabilities. Words are discarded as non-opinion words whenever the expanded lexicons labelled them as neutral.

We study three different setups based on these attributes. 1) Baseline: It includes the attributes calculated from the seed lexicon. 2) ED: It includes the baseline and the attributes from the ED expanded lexicon. 3) STS: This one is analogus to ED, but using the STS lexicon.

In the same way as in the word-level classification task, we rely on the weighted AUC as evaluation measure, and we compare the different setups with the baseline using the corrected paired t-tests. The classification results obtained for the different setups are shown in Table 9.

The results indicate that the expanded lexicons produce meaningful improvements in performance over the baseline on the different datasets. The performance of STS is slightly better than that of ED. This pattern was also observed in the word-level classification performance shown in Table 6. This suggests that the two different ways of evaluating the lexicon expansion, one at the word level and the other at the message level, are consistent with each other.

9 http://sentistrength.wlv.ac.uk/documentation/ 6humanCodedDataSets.zip

10 http://www.sananalytics.com/lab/twitter-sentiment/

11 http://www.cs.york.ac.uk/semeval-2013/task2/

7 Conclusions

In this article, we presented a supervised method for opinion lexicon expansion in the context of tweets. The method creates a lexicon with disambiguated POS entries and a probability distribution for positive, negative, and neutral classes.

The experimental results show that the supervised fusion of POS tags, SGD weights, and semantic orientation, produces a significant improvement for three-dimensional word-level polarity classification compared to using semantic orientation alone. We can also conclude that, as attributes describing the central location of SGD and SO time-series smooth the temporal fluctuations in the sentiment pattern of a word, they tend to be more informative than the last values of the series for word-level polarity classification.

To the best of our knowledge, our method is the first approach for creating Twitter opinion lexicons with these characteristics. Considering that these characteristics are very similar to those of SentiWordNet, a well-known publicly available lexical resource, we believe that several sentiment analysis methods that are based on SentiWordnet can be easily adapted to Twitter by relying on our lexicon 12 .

Our supervised framework for lexicon expansion opens several directions for further research. The method could be used to create domain-specific lexicons by relying on tweets collected from the target domain. However, there are many domains such as politics, in which emoticons are not frequently used to express positive and negative opinions. New ways for automatically labelling collections of tweets should be explored. We could rely on other domain-specific expressions such as hashtags, or use message-level classifiers trained from domains in which emoticons are frequently used.

Other types of word-level features based on the context of the word can be explored. We could rely on well-known opinion properties such as negations, opinion shifters, and intensifiers, to create these features.

As our word-level features are based on time-series, they could be easily calculated in an on-line fashion from a stream of time-evolving tweets. Based on this, we could study the dynamics of opinion words. New opinion words could be discovered because the change of the distribution in certain words could be tracked. This approach could be used for online lexicon expansion in specific domains, and potentially be useful for high impact events on Twitter, such as elections and sports competitions.