Incorporating Domain and Sentiment Supervision in Representation Learning for Domain Adaptation

1 Introduction

Domain adaptation [Daum´e III and Marcu, 2006] aims at learning a robust classifier with labeled data on a source domain, and at the same time attemps to enforce the classifier to work well on a target domain. The demand for quick transfer between domains are commonly seen in text classification tasks, particularly urgent in sentiment analysis.

The key challenge in domain adaptation is that the data distribution or feature representation may be quite different across domains. For instance, we may use quite different vocabularies in writing reviews, say attractive, boring in books reviews while reliable, affordable in electronics reviews. The simplest solution is to train a classifier for each domain, unfortunately, this requires heavy annotation on every single domain. Another solution may be to train a robust classifier with labeled samples from a source domain, and to enforce the classifier to work well on a target domain.

Several models had been proposed for domain adaptation such as instance reweighting [Huang et al., 2006; Mansour et al., 2009], joint feature representation learning [Blitzer et al., 2006; Xue et al., 2008; Glorot et al., 2011b; Chen et al., 2012] , and feature projection [Bollegala et al.,

Jiashen Sun, Xuan Zhu

Samsung R&D Institute Beijing, China jiashens.sun@samsung.com xuan.zhu@samsung.com

2014]. Among these methods, Stacked Denoising Autoencoders (SDA) for domain adaptation [Glorot et al., 2011b] , one of the joint feature representation learning methods, has been reported to be quite effective. The SDA model learns a neural network using reviews from all domains, and then trains a SVM sentiment classifier on the source domain using the learned representations. The SVM classifier is then applied to other domains and gains remarkable improvements over other methods.

Intuitively, if the learned representations have a good separation for domain-specific and sentiment-specific constituents, a sentiment classifier will be less susceptible to those domain-specific features and more favorable for domain adaptation. Furthermore, if the learned representations are specialized for sentiment classification during training, more sentiment information can be encoded and thus improve the performance of sentiment classification.

However, despite the success of SDA, the model simply mixed all samples from different domains during the training process. Such an oversimplified strategy does not encourage the model to discriminate domain-specific and sentimentspecific features. There is also no supervision for training sentiment-favorable representations which are more related to the classification goal.

This is what we do in this paper. We guide the representation learning process with two types of supervision - one is domain supervision, endowing the proposed model with the capability of predicting the domain of the input. We believe this factor will encourage the model to better discriminate domain-specific and sentiment-specific features. The other is sentiment supervision which utilizes the source domain sentiment labels. In this way, the model will be able to learn sentiment-favorable representations.

More specifically, on top of SDA, our learning objective consists of two terms: the reconstruction loss between the input and decoded output; the prediction loss for predicting the domain or/and sentiment label in the output layer. Therefore, the domain supervision and sentiment supervision can be easily introduced by adding the "domain loss" and "sentiment loss" respectively. Once the representation is constructed, we use the learned representations to train a classifier on a source domain, and predict the labels of the data on a target domain. Experimental results show that the two factors significantly improve the performance of domain adaptation.

To summarize, our main contributions in this work are as follows:

• We reveal two key factors in representation learning for domain adaptation - domain supervision to better disentangle domain-specific and sentiment-specific features, and sentiment supervision which guides the model to learn sentiment-favorable representations.
• We proposed three modified SDAs for representation learning - SDA-DS, SDA-SS and SDA-DSS. Experimental results show that our models can learn better representations and significantly improve the performance of domain adaptation.

The rest of the paper is structured as follows: We survey related work in section 2, introduce our method in detail in section 3, and present the experiments in section 4. We discuss our work in section 5 and summarize the work in section 6.

2 RelatedWork

Domain adaptation, first introduced by Daum´e III and Marcu [2006], aims to generalize classifiers across domains. Our work is mainly related to the following approaches for domain adaptation. These works fall into three types.

The first type is instance reweighting in which instances are assigned different weights so that these weighted instances have similar feature distributions. Huang et al. [2006] proposed kernel mean matching (KMM), a kernel-based method, to assign weights to instances. Mansour et al. [2009] investigated domain adaptation with multiple sources. They reweighted the patterns in different data sources by estimating the data distribution.

The second type is to transform the data representations of the source and target domains to a joint learned feature space. The idea is to share feature distributions across all domains so that a classifier trained on one domain is applicable for other domains. Blitzer et al. [2006] proposed Structural Corresponding Learning (SCL) to learn a low-rank representation of the data. They heuristically selected those features that is frequent in both source and target domains. Ben et al. [2007] analysed the effect of representation transforming. Pan et al.[2010] proposed spectral feature alignment (SFA) which used domain independent words to align domain-specific words into unified clusters and discover robust representations across different domains. Glorot et al.

[2011b] proposed to use a deep learning approach (SDA), which can encode complex nonlinear feature mapping, to learn representations for domain adaptation and obtained remarkable improvements. A large amount of data across all domains are mixed together to train the SDA model. As an extension to SDA, mSDA is proposed to accelerate the training of SDA by hundreds of times and has comparable performance [Chen et al., 2012]. A vital property of mSDA is that it has a closed form solution. A particularly interesting conjecture is that the success of SDA may derive from the fact that domain-specific and sentiment-specific features are separated in the learned representations. Along this line, we find that much improvements can be obtained by employing domain or sentiment supervision. By mixing the source and target data in different proportion as training set, Chopra et al.[2013] combined several SDAs together to improve performance of domain adaptation.

Bollegala et al. [2014] proposed a third type called feature projection. Different from feature representation learning which projects different domains to a shared space, they project the source domain feature space to the target domain using Partial Least Squares Regression (PLSR).

Our work is also related to neural network for sentiment classification. Glorot et al. [2011a] showed that rectifier units (hard (max(0, t)) or soft (log(1+et ))) outperform other nonlinear functions for sentiment classification. We use the hard version rectifier since it can produce sparse coding (exactly zero).

3 Method

3.1 Background

Representation learning attempts to project the original features to a new feature space, using shallow or deep learning architectures (usually neural networks). Several forms of representation learning models, like Restricted Boltzmann Machines (RBMs) [Hinton et al., 2006] and auto-encoders [Bengio et al., 2007], have been developed for deep learning. We mainly discuss auto-encoder here. A simplest auto-encoder is composed of two steps: an encoder h maps the input x to a hidden representation h(x), and a decoder g tries to reconstruct x from h(x), i.e. r(x) = g(h(x)). The auto-encoder is trained to minimize the reconstruction loss between r(x) and x. Square error and KL divergence are two commonly used loss functions. The hidden layer h(x) is regarded as a new representation of x after training, and can be concatenated with x to form a new input to task-specific classifiers.

Denoising auto-encoder (Figure 1(a)), referred as denoiser, is a slightly modified version of auto-encoder. It introduces a corruption process before the hidden layer, and tries to reconstruct x from this corrupted input ˜x, i.e. r(˜x) = g(h(˜x)). The corruption process could be a masking noise (each element of x has a probability of p to be set to 0). A denoiser is also trained to minimize the loss between x and its reconstruction form r(˜x). A typical architecture of denoiser is fully connected from the corrupted input ˜x to the hidden layer h(˜x) and from the hidden layer h(˜x) to the output layer r(˜x), and different nonlinear activation functions can be used. Several denoisers can be stacked together to form a stacked denoising auto-encoder (SDA) [Vincent et al., 2008].

3.2 SDA for Domain Adaptation

Glorot et al. [2011b] apply SDA to domain adaptation for sentiment classification. The input x is a zero-one vector indicating the absence or presence of a word in a review (bag-of-words representation), thus the dimension of x is the same as the vocabulary size. They use hard rectifier rec(t) = max{0, t} as nonlinear function for the hidden layer h(˜x) and sigmoid σ(t) = 1 1+e-t for the output layer r(˜x). The objective function is an element-wise KL divergence between r(x) and x:

where xi and r(˜x)i are the i-th element of x and r(˜x). A L1 norm of the hidden layer ||h(˜x)||1 can also be added to the objective to encourage sparsity of the representation.

The training process is to minimize the objective function on a large amount of data across all domains. After the SDA is constructed, new representations are obtained by simply concatenating the input x and the hidden layer h(˜x). A standard sentiment classifier (SVM, logistic regression, etc.) is trained on the source domain using the new representations, and then applied to other target domains.

Glorot et al. [2011b] studied both single-layer and multilayer SDAs ("layer" here refers to "denoiser"), and reported that a single-layer SDA was good enough for domain adaptation. We'll mainly discuss the single-layer SDA here.

3.3 SDAwith Domain Supervision (SDA-DS)

As depicted in Figure 1(b), in addition to reconstructing the input in the output layer, we add a fully connected layer and use softmax as nonlinear function to predict the domain distribution d(˜x) of the input. The hidden layer h(˜x) is required not only to reconstruct the input, but also to predict the domain label. In this way, the proposed model is encouraged to extract domain-specific constituents better in the learned representation. This is what we expected, and several papers have reported that domain adaptation may benefit from such a property [Glorot et al., 2011b; Chen et al., 2012].

The reference domain distribution ˆ d(x) is an indicating vector with 1 corresponding to the domain that x comes from and 0 elsewhere. Thus, the domain supervision can be formulated as the KL divergence between ˆ d(x) and d(˜x),

where ˆ d(x)i and d(˜x)i are the i-th element of ˆ d(x) and d(˜x), respectively. By combining the reconstruction error and domain supervision, we obtain the objective of SDA-DS as follows:

where λ ∈ (0, 1) is used to control the balance of the two terms. Similar to SDA, a L1 norm of the hidden layer ||h(˜x)||1 can be added to the objective function.

The training process is to minimize the objective function across all domains. For domain adaptation, a standard sentiment classifier is trained using the new representations (concatenate x and h(˜x)) on the source domain and then applied to other target domains.

3.4 SDAwith Sentiment Supervision (SDA-SS)

Since our final task is to discover new features for sentiment classification, a simple idea is to guide the representation learning process with sentiment supervision so that the learned representations are sentiment-specialized and can improve performance of sentiment classification.

Figure 1(c) shows how we incorporate sentiment supervision into SDA. Using a fully connected layer and sigmoid nonlinear function, we output y(˜x) indicating the probability of how likely the sentiment of x is positive. Since not all samples have sentiment labels, we construct the sentiment reference ˆ y(x) as follows: for those labeled samples in the source domain, the reference ˆ y(x) is just its sentiment label (1 for positive, 0 for negative), for those unlabeled or target domain samples, we just set ˆ y(x) = 0.5 as a reference distribution 1 . Thus, the sentiment supervision can be formulated as:

The objective function of SDA-SS is a combination of the reconstruction error loss(x, r(˜x)) and sentiment supervision loss(ˆ y(x), y(˜x)):

The training process is the same as SDA-DS.

1 This is equivalent to the maximum entropy principle.

3.5 SDAwith both Domain and Sentiment Supervision (SDA-DSS)

Since we have introduced domain supervision and sentiment supervision in previous sections, a natural idea is to combine them together, enforcing the learned representation domaindistinguishable and sentiment-favorable.This simple combination of SDA-DS and SDA-SS is depicted in Figure 1(d), and we name it as SDA-DSS.

The objective function of SDA-DSS is thus a combination of the reconstruction error loss(x, r(˜x)), domain supervision loss( ˆ d(x), d(˜x)) and sentiment supervision loss(ˆ y(x), y(˜x)):

where λ and η control the weights of the three terms in the objective function. The training process is the same as SDA.

4 Experiments

4.1 Data Preparation

We experiment on a benchmark dataset - products reviews from Amazon, first collected by Blitzer et al. [2007] 2 . This dataset contains about 340,000 reviews of 22 different products. Each review is either unlabeled or labeled as positive or negative. We conduct domain adaptation experiments on reviews of four products (12 pairs for domain adaptation) - book (b), dvds (d), electronics (e) and kitchen (k). Table 1 shows the number of positive and negative reviews, and unlabeled reviews from these four domains.

4.2 Experiment Settings

Parameter Settings

The preprocessing stage of the reviews is the same as [Blitzer et al., 2007]: we use bag-of-words schema, namely a zeroone vector indicating the absence or presence of a word, to represent a review. Only the top 5,000 frequent unigrams and bigrams are kept here. The hidden layer size is 5,000 for all the four networks.

2 http://www.cs.jhu.edu/ mdredze/datasets/sentiment/

We use the standard backward propagation and batch gradient descent method to train all the four neural networks. There are some hyper-parameters to be tuned and we attempt the following settings: masking noise probability p ∈ {0.3, 0.5, 0.8}, L1 regularization penalty l1 ∈ {0, 10-4 , 10-3}, batch size bs ∈ {10, 20}, epoch number ep ∈ {1, 2, 3, 4, 5}, step length sl ∈ {0.1, 0.2, 0.01, 0.02}, weight control coefficient in SDA-DS, SDA-SS, SDA-DSS λ ∈ {0.01, 0.1} and η = {1, 0.5}. All these parameters are selected by minimizing the cross validation error (5-fold) on the source domain training data.

After the networks are constructed, the new representations (concatenation of x and h(˜x)) are used to train a logistic regression classifier 3 on the source domain and this classifier is then applied to the target domains.

Compared Methods

The baseline is a logistic regression classifier trained on the original raw bag-of-words features using the labeled data from the source domain.

The other methods to be compared are including: Structural Correspondence Learning (SCL) which was used in domain adaptation of sentiment classification by Blitzer et al. [2007]; Spectral Feature Alignment (SFA) proposed by Pan et al. [2010]; SDA proposed by Glorot et al. [2011b].

3 Note that we did not choose SVM here as previous work just because SVM training is much slower than LR.

Evaluation Metrics

As described in [Glorot et al., 2011b], transfer error e(S, T) is defined as the test error on the target domain T of a model which is trained on the source domain S. Therefore, e(T, T) which is called in-domain error is the cross validation error on domain T, and eb(T, T) refers to in-domain error of the baseline (the logistic regression model trained with bag-ofwords feature). We compare different models (including the baseline) using transfer loss which is defined as t(S, T) = e(S, T ) - eb(T, T ).

4.3 SystemComparison

The domain adaptation results of the 12 domain pairs and the mean value are shown in Figure 2. As can be seen, the three proposed models significantly outperform other methods. The mean transfer losses are: Baseline (0.0750), SCL (0.0558), SFA (0.0518), SDA (0.0476), SDA-DS (0.0341) SDA-SS (0.0403) and SDA-DSS (0.0353). SDA-DS performs better than other baseline methods in 10 of the 12 tasks, SDA-SS is better in 9 tasks and SDA-DSS is better in 11 tasks.

Our hypothesis is justified by the results - supervising the representation learning process with domain and sentiment information helps to learn better representations and thus significantly improve the performance of domain adaptation.

The results also show that domain supervision is more effective than sentiment supervision, the reason may be that the domain label of all samples are available while only those source domain labeled data have sentiment labels (note that we have much more unlabeled data than labeled data, see Table 1).

4.4 Proxy-A-Distance

Proxy-A-Distance (PAD) [Ben-David et al., 2007] measures the difference between two domains. It is defined as 2(1-2ϵ), where ϵ is the cross validation error when distinguishing the two domains. Ben-David et al. [2007] argued that low PAD measure is favorable for domain adaptation (low PAD ⇒ high ϵ ⇒ the two domains are similar and difficult to distinguish ⇒ favorable for domain adaptation). The PAD measures of SDA and the proposed models are shown in Figure 3 (5-fold cross validation is used). For each domain pair d1 - d2, we report the mean PAD of the two representations - transferring from d1 to d2 and reverse.

The results are shown in Figure 3. On one hand, results show that low PAD measures correlated with better domain adaptation performance. As can be seen, domain pair b - d and e-k have the lowest PAD measures among the 6 domain pairs and we can also see from Figure 2 that the transfer losses of b → d, d → b, e → k and k → e are particularly low in the 12 domain transfer pairs. This observation is consistent with the observation of Ben-David et al. [2007].

On the other hand, the proposed models produce not only high PAD values but also better domain adaption performance (SDA-DS and SDA-DSS against SDA and baselines). A reasonable explanation is that those domain-specific and sentiment-specific constituents are better disentangled in the learned representations, and thus improve both domain prediction and sentiment classification.

The PAD measure of SDA is slightly higher than the baseline while SDA-DS and SDA-DSS are much higher especially in the e-k setting. This is mainly because we introduce domain supervision in SDA-DS and SDA-DSS, and thus their learned representations are well trained for domain classification. This is exactly what we expected at the beginning. We enforce SDA-DS and SDA-DSS to predict the domain of the input so that those domain-specific and sentiment-specific features are better disentangled, and thus can improve the performance of domain adaptation.

4.5 Feature Overlapping Test

The motivation that we incorporate domain supervision into SDA is to guide the representation learning process to better disentangle those domain-specific and sentiment-specific features. In this section, we want to directly analyse the overlap of active features for domain prediction and sentiment prediction.

A feature is called active if it can provide "sufficient" information for classification. Given a classification task (either domain prediction or sentiment prediction), in order to identify these active features, we can train a logistic regression model with L1 sparse regularization (most of the coefficients are driven to 0). After training, each feature fi is associated with several coefficients w(1) i ,w(2) i , ...,w(n) i (n is 1 in binary classification and is the number of classes in multi-class classification). If one of these n coefficients is nonzero, we regard fi as an active feature.

We apply the original input (bag-of-words representation) and the learned representations (only the hidden layer h(˜x)) to two tasks: sentiment classification (binary) and domain classification (4-class). If a feature fi in the representation is active in both tasks, it is called an overlapping feature. Obviously, if those domain-specific and sentiment-specific features are disentangled in the learned representations, the number of overlapping features should be small.

For each method in each source domain, there is an optimal parameter setting which is obtained by minimizing the cross validation error on source domain. The results are shown in Table 2. The number of overlapping features in the learned representations is much smaller than the baseline (original bag-of-words representation). SDA-DS and SDA-DSS also have a much smaller number of overlapping features compared with the original SDA. Without domain supervision, SDA-SS does not reduce the number of overlapping features.

This overlapping test strongly supports our claim that incorporating domain supervision in representation learning helps to learn better disentangled representations for domain adaptation. The domain supervision significantly reduces the number of overlapping features.

4.6 Cross-validation of Sentiment Classification

We propose to learn sentiment-favorable representations by incorporating sentiment supervision. These learned representations are expected to improve sentiment classification performance. We verify this claim in this section.

Using the original input or the learned representations (only the hidden layer), we test the sentiment cross validation error (5-fold) of the logistic regression model on the target domains.

Specifically, given a method m and a source domain d, we report the mean sentiment cross validation error on the other three domains (except d). For example, the cross validation error of SDA-DS on b, is averaged from the cross validation errors of that model on d, e, and k. Thus, m is associated with four errors, one for each domain.

The results are shown in Table 3. As can be seen, sentiment supervision (SDA-SS, SDA-DSS) reduces the cross validation error on the target domains even though it only utilizes labeled data from source domain. The results demonstrate that incorporating sentiment supervision is able to learn sentiment-favorable representations. The cross validation error of SDA-DS is also reduced. This means that disentangling domain-specific and sentiment-specific features can also improve sentiment classification performance.

5 Discussion

We list the number of domain active features in table 4. The results show that with domain supervision, the number of domain active features in our models (SDA-DS, SDA-DSS) is largely reduced, demonstrating that those domain-specific constituents are more centralized with less feature overlapping. We also note that the number of active features in SDA and SDA-SS is increased. The reason may be that when we project the original discrete feature space to a continuous feature space without supervision, domain information will disperse in more dimensions. Unlike SDA which disentangles domain and sentiment features implicitly, our proposed models give more specific and clear signals to the presentation learning process.

The training of SDA and SDA-DS is irrelevant to which domain is the source and which are the targets. In other words, only one SDA or SDA-DS need to be trained to transfer across all domains. But SDA-SS and SDA-DSS do not have such a property since different source domains have different annotated data. However, the source domain is always given in reality, so there is no much difference in the applicability of these three methods.

Glorot et al. [2011b] reported that stacking several denoisers together could learn even better representations and improve domain adaptation performance. But training such a network is extremely time-consuming and further improvement is relatively small, so we just test the one-layer architecture in our work.

6 Conclusion

In this paper, we investigate two key factors (domain/sentiment supervision) for domain adaptation and propose three models (SDA-DS, SDA-SS, SDA-DSS) to learn better representations for this task. We find that domain supervision helps to disentangle domain-specific and sentimentspecific constituents in the learned representations, and that sentiment supervision helps to learn sentiment-favorable representations. Experimental results show that the proposed models obtain significant improvements over some competitive baselines such as SCL, SFA, SDA, etc.

Though we have supplied clear signals to disentangle domain-specific and sentiment-specific constituents in the learned representation, as future work, we are planning to design neural network structures that are able to explicitly represent the two kinds of information.

7 Acknowledgments

This work was partly supported by the National Basic Research Program (973 Program) under grant No.

2012CB316301/2013CB329403, the National Science Foundation of China under grant No. 61272227/61332007, and the Beijing Higher Education Young Elite Teacher Project. The work was also supported by Tsinghua University - Beijing Samsung Telecom R&D Center Joint Laboratory for Intelligent Media Computing.