On the Estimation of Treatment Effect with Text Covariates

1 Introduction

Treatment effect, also known as causal effect, refers to the effect that one variable (i.e., the treatment) exerts on the other variable (i.e., the outcome). Accurate and reliable estimation of treatment effects would largely benefit decision making across various domains, due to its ability to reflect the outcomes of different choices. For example, in the medical domain, doctors can recommend the best therapy for a specific patient if the effect of the therapy on the recovery rate is known; in the education field, teachers can adopt the best teaching method according to the effect of teaching methods on the test score; and in the advertising area, business owners can choose the best platform to advertise based on the effect of the platform on the response rate. In the above examples, the therapy/teaching method/platform is the treatment, and the recovery rate/test score/response rate is the outcome.

The treatment effect is defined as the change of the outcome if the intervention is made on the treatment, supposing the covariates are unchanged (i.e., condition on those covariates), where covariates are the variables/features that are related to the treatment as well as the outcome. For example, in the aforementioned medical case, the patient's demographic information and physical examinations are the covariates. Many methods have been developed for treatment effect estimation [Imbens and Rubin, 2015].

Most of the existing work focuses on numerical covariates, while little attention has been paid to the textual covariates. However, in real world, text data are almost everywhere, such as clinical notes, movie reviews, news, social media posts, and etc. Different from the structured and well-defined numerical covariates, textual covariates contain richer information and can be summarized at different levels, such as word level, topic level, semantics level, and etc. This property of text data brings some new challenges into treatment effect estimation with textual covariates. In particular, some textual covariates that are very predictive to the treatment assignment might not be that predictive to the outcome. Such covariates are referred to as the nearly instrumental variables. In treatment effect estimation, existing work [Pearl, 2012; Wooldridge, 2016] has shown that conditioning on the nearly instrumental variables tends to amplify the bias in the analysis of causal effects. Therefore, the nearly instrumental variables should be excluded when estimating the treatment effect. Thus, the major challenge in estimating the treatment effect with textual covariates is: How to filter out the nearly instrumental variables?

In existing methods, filtering out the nearly instrumental variable is achieved by covariate re-weighting [Kuang et al., 2017a; Chang and Dy, 2017; Diamond and Sekhon, 2013] or feature selection [Kuang et al., 2017b; Tibshirani, 1996; Rassen et al., 2011], when the covariates are numerical. However, when the covariate contains text data, the effectiveness of the re-weighting or feature selection based approaches would be limited, as those methods would be restricted to only one specific level of information contained in the textual variable, which leads to insufficient summarization of text covariates and further leads to insufficiency in filtering out nearly instrumental variables.

To handle the above challenges, we propose the Conditional Treatment-Adversarial learning based Matching method (CTAM), inspired by the conditional adversarial architecture in [Zhao et al., 2017]. CTAMfirst learns the latent representation of all covariates, in which the information contained in text variables can be fully summarized. Then in the learned representation space, we adopt the nearest neighbor matching (NNM), for its interpretability, to estimate the outcome if the treatment had been changed. The key characteristic of CTAM is the conditional treatment adversarial training procedure whose goal is to filter out the information related to nearly instrumental variables in the representation space. In this procedure, the treatment discriminator, along with the representation learner and the outcome predictor, play a minimax game: The treatment discriminator is trained to predict the treatment label correctly, while the representation learner, corporately working with the outcome predictor, aims to fool the treatment discriminator. Through the conditional treatment adversarial training procedure, the learned representation discards the extraneous information specific to treatment assignment, and meanwhile retains the information related to outcome prediction. Consequently, the proposed method benefits the treatment effect estimation with text covariates.

To evaluate the effectiveness of the proposed method, we first conduct experiments on two semi-synthetic datasets. Experimental results show that the proposed method outperforms the state-of-the-art methods. Furthermore, in the real world dataset, we verify the matching quality, and demonstrate that by imposing the conditional treatment adversarial training, the dependency between the treatment assignment and the nearly instrumental variables are removed.

2 Preliminaries

We first introduce some important notations. Let W denote the treatment, and W ∈ RN , where N is the number of records in the dataset, and Wi is the treatment of the ith record. When the treatment is binary, the records with Wi = 1 form the treated group and the others belong to the control group. Let X denote all covariates excluding the textual covariates, and X ∈ RN×d , where d is the number of non-textual covariates. Let T denote the textual covariate, and T = {T1, T2, . . . , TN}, where Ti is the text that belongs to the i-th record, and Ti = [ti,1, ti,2, . . . , ti,NTi ], where ti,j is the j-th word in the i-th record, and NTi denotes the total number of words in Ti. The outcomes of different treatments are the potential outcomes, and let Y ω i denote the potential outcome of the i-th individual/record with the ω-th treatment. The observed outcome (i.e., factual outcome) is denoted as Y F , and Y F ∈ RN .

In this work, we follow the potential outcome framework [Splawa-Neyman et al., 1990; Rubin, 1974] and the following assumptions ensure that the treatment effect can be identified.

Assumption 1: Stable Unit Treatment Value Assumption (SUTVA). The potential outcomes for any unit do not vary with the treatment assigned to other units, and, for each unit, there are no different forms or versions of each treatment level, which lead to different potential outcomes.

Assumption 2: Consistency. The potential outcome of treatment w equals to the observed outcome if the actual treatment received is w.

Assumption 3: Ignorability. Given pre-treatment covari-

ates, i.e., the covariates affect the treatment, treatment assignment is independent of the potential outcomes.

Assumption 4: Positivity. For any set of values of pretreatment covariates X, treatment assignment is not deterministic [D'Amour et al., 2017]: ∀w and x, ∃ η ∈ (0, 0.5), s.t. η < P (W = w|X = x) < 1 - η.

The treatment effect can be measured at the individual, population, and treated group level, which is known as the individual treatment effect (ITE), the average treatment effect (ATE) and the average treatment effect on the treated group (ATT), respectively. With the above four assumptions, the ITE, ATE, and ATT can be identified as 1 :

where Y 1 i and Y 0 i are the potential treated and control outcomes; U is the whole population and U1 is the treated group. With the above knowledge, the problem is defined as

Input: The non-textual covariates X, textual covariates T, treatment assignment W and the observed outcome Y F .

Output: ITE, ATE, and ATT.

3 Methodology

3.1 Motivation

The underlying causal graph of our proposed method is shown in Figure 1. In the figure, Z and Z ′ together are the latent representations of the observed textual covariates T and non-textual covariates X. Among the latent variables, Z ′ denotes the nearly instrumental variables, which is more predictive to the treatment assignment than the outcome Y . As mentioned previously, conditioning on the nearly instrumental variables would amplify the treatment effect estimation bias. Our objective is to learn the latent representations that filter out the information related to nearly instrumental variables.

1 Here we define the treatment effect with binary treatment. For the case of treatment effect with multiple treatments, please refer to [Lopez et al., 2017].

Text Processing

Figure 2: CTAM Framework

Our proposed method introduces the conditional treatmentadversarial learning to eliminate the information related to nearly instrumental variables Z ′ as much as possible in the latent representations.

3.2 Framework

Figure 2 shows the framework of the proposed CTAM method. CTAM contains three major components: text processing, representation learning, and conditional treatment discriminator. Through the text processing component, the original text is transformed into vectorized representation S. After that, S is concatenated with the non-textual covariates X to construct a unified feature vector, which is then fed into the representation neural network to get the latent representation Z. After learning the representation, Z, together with potential outcomes Y , are fed into the conditional treatment discriminator. During the training procedures, the representation learner plays a minimax game with the conditional treatment discriminator: By preventing the discriminator from assigning correct treatment, the representation learner can filter out the information related to nearly instrumental variables. The final matching procedure is performed in the representation space Z. The following sections introduce each component in detail.

3.3 Text Processing and Representation Learning

Text processing procedure converts the text data T to the numeric representation S. Various methods are developed to map the word from the vocabulary to the numerical vector, such as word embedding (GloVe [Pennington et al., 2014], word2vec [Mikolov et al., 2013], etc.), document-term matrix (bag of words [Harris, 1954; Manning et al., 1999], inverse document frequency [Sparck Jones, 1972]). We adopt the word embedding learned by GloVe [Pennington et al., 2014] in this procedure, and S is the average of all wordembeddings in one document.

Following the text processing, in the representation procedure, the learned numerical vector of textual covariate S is first concatenated with the numerical covariate X. The concatenated vector is denoted as C. After concatenation, a representation neural work is adopted to map the concatenated vector C to the latent representation Z: Z = Φrep(C;ΘΦ), where Φrep is a feed-forward neural network with ReLU [Nair and Hinton, 2010] as the activation function and ΘΦ denotes the set of its parameters.

The latent representation Z learned by Φrep contains the information related to nearly instrumental variables, which would amplify the treatment effect estimation bias. In order to eliminate such information, we design the following conditional treatment-adversarial learning procedure.

3.4 Conditional Treatment Discriminator and Conditional Treatment-Adversarial Learning

Conditional Treatment Discriminator

The input of the conditional treatment discriminator is the latent representation Z and the potential outcomes Y , and the output is the treatment assignment W. Our discriminator conditions on the potential outcomes, which allows the latent representation to correlate with the treatment only through the potential outcome distribution. In other words, by playing the minimax game, which is introduced in the following section, with the conditional treatment discriminator, the learned latent representation is capable of eliminating the conditional dependency with the treatment assignment.

The conditional treatment discriminator is a feed-forward neural network, denoted as D(Z, Y ; ΘD), where ΘD is a set of parameters. The goal of this discriminator is to correctly predict the treatment assignment. The loss of the conditional treatment discriminator is measured by the cross-entropy:

where Ψpop(·; ΘΨ) is the pseudo outcome predictor with ΘΨ as its parameters, which is defined as follows.

Pseudo Outcome Prediction

The conditional treatment discriminator requires the potential outcomes of all treatments, which is formulated as:

where nw is the number of total treatments, {Ψi}nw i=1 is outcome prediction model for each treatment, ΘΨ is the set of parameters of Ψpop, and Θ(i) Ψ is the parameter set of i-th outcome prediction model. As the potential outcome here is only for the conditional treatment discriminator and is not the explicit result, we name it as the pseudo potential outcome.

Conditional Treatment-Adversarial Learning

The objective of conditional treatment-adversarial learning is to filter out the information related to the nearly instrumental variables. As the nearly instrumental variables refer to the variables that are more predictive to the treatment assignment instead of the outcome, this filtering strategy is equivalent to removing the conditional dependency between the latent representation and the treatment assignment. Therefore, we train an adversarial learning model to achieve this goal. The discriminator D, along with Φrep and Ψpop, plays a minimax game. The discriminator D aims to minimize Eqn. (2) in order to assign correct treatment. Meanwhile, the representation learner Φrep and the outcome predictor Ψpop are trained to maximize the above loss to filter out the information that benefits the discriminator D. When the conditional treatment discriminator can be successfully fooled, the information that enhances the treatment assignment is eliminated from the latent representation, i.e., the information related to nearly instrumental variables can be successfully filtered out.

3.5 Loss Function and Parameter Training

Loss Function

The final loss of the three-player game is:

where LD is defined in Eqn. (2) and λ is the hyper-parameter. Lp(ΘΦ, ΘΨ) is the sum of the group distance and the pseudo outcome prediction loss, which is defined as:

where NY is the number of label classes in the outcome, NW is the number of treatments, Lpd(·, ·) is the sum of pairwise distance between the two input matrices, Z {i}{m} denotes the representations of records that are assigned with the i-th treatment and their observed outcomes are the m-th label class, and ˆ Y is the corresponding prediction of the observed outcome, which can be obtained from Eqn. (3).

The first term in Lp measures the pairwise distance between the records sharing the observed outcome label under the same treatment, and the second term measures the pairwise distance between the records that have different observed outcomes. Minimizing the difference of two terms makes similar records close to each other, while dissimilar records far from each other in the representation space. The third term is the pseudo outcome prediction loss, and minimizing it allows better potential outcome predictions for conditional treatment discriminator.

Model Training

The training procedure involves optimizing the minimax game among the discriminator D, representation learner Φ and the pseudo outcome predictor Ψ, which can be viewed as:

The three players (D,Φ and Ψ) are alternatively updated as:

where ηΦ, ηΨ, and ηD are the learning rates. To prevent the model collapsing, we can update the ΘD several times before continuing to update the other parameters.

Nearest Neighbor Matching

After the model training, the estimated outcome of the i-th record/individual with the ω-th treatment, denoted as ˆ Y ω i , can be obtained by nearest neighbor matching: ˆ Y ω i = Y F ν , with ν = arg minν∈Uω dist(zi, zν ), where Y F ν is the observed outcome of the ν-th record, Uω is the group with the ω-th treatment, dist(·, ·) is the Euclidean distance, and zi (zν) is the representation of the i-th (ν-th) record. Then the ITE, ATE and ATT can be obtained by Eqn. (1) accordingly.

4 Experiment

In this section, we conduct experiments on both semisynthetic and real world datasets to evaluate the following: 1) our proposed method can work well on both the textual covariates and the non-textual covariates, and 2) the conditional treatment discriminator in our proposed method improves the performance on treatment effect estimation.

4.1 Experiment Settings

Baselines

We compare our proposed CTAM method with the following widely-adopted nearest neighbor matching (NNM) based methods: Mahalanobis distance matching (MDM) [Rubin, 1979], propensity score matching with logistic regression (PSM) [Rosenbaum and Rubin, 1983], dimensionality reduction by random linear projections (DR-RLP) [Li et al., 2016], Hilbert-Schmidt independence criterion based nearest neighbor matching (HSIC-NNM) [Chang and Dy, 2017], and structural topic model based matching (STM) [Mozer et al., 2018]. Besides the NNM based methods, we also compare the proposed method with the following representative baselines: linear regression with ℓ1 regularization (LASSO) [Tibshirani, 1996]; Bayesian additive regression trees (BART) [Chipman et al., 2010]; and causal forest regression (CF) [Wager and Athey, 2017]. Among the above baselines, LASSO and HSIC-NNM take the nearly instrumental variables into consideration by covariates reweighting/selection. For comparison fairness, the baselines share the same text processing procedure with CTAM.

Parameter Setting

The parameters of baselines are set as suggested by the original papers, and the hyper-parameter search of CTAM follows the scheme in [Shalit et al., 2017].

Evaluation Metrics

The following evaluation metrics are adopted to compare the proposed methods with the baselines:

(1) PEHE: precision in estimation of heterogeneous effect [Hill, 2011], which is defined as: PEHE = √ 1n ∑ni=1 (ITEi - ˆ ITEi)2 . (2) EATE: error of ATE estimation. EATE is defined as: EATE = |ATE- ˆ ATE|. (3) EATT: error of ATT estimation. EATT is defined as: EATT = |ATT- ˆ ATT|.

4.2 Experiment on News Dataset

Dataset

The News dataset is first introduced in [Johansson et al., 2016], which studies the effect of viewing devices to the user experience. The text covariate T is represented by the termdocument matrix, and the vocabulary size is 3,477. The treatments are different devices: Wi = 1 denotes the news in the i-th record is viewed in mobile and Wi = 0 denotes the desktop. The generations of treatment assignment and the outcome (reading experience) are the same as [Johansson et al., 2016]. We generate 1, 000 samples for each realization and repeat the sampling procedure 50 times.

Results and Analysis

The results of our proposed method and the baseline approaches are shown in Table 1. Overall, our method has the best performance under PEHE and EATE measurements, and has competing performance compared with the best baseline STM under EATT measurement. This observation demonstrates that the conditional treatment discriminator can effectively filter out information related to the nearly instrumental variables and therefore reduce the bias of treatment effect estimation.

4.3 Experiment on IHDP Dataset

The proposed method is motivated by the existence of nearly instrumental variables which are commonly observed in text covariates, but it can also work well on numerical cases to filter out information that may amplify the bias in the treatment effect estimation and improve the result. To validate this claim, we evaluate the proposed method on the widely used benchmark dataset IHDP.

Dataset

This dataset is from the Infant Health and Development Program [Brooks-Gunn et al., 1992] targeting low-birth-weight, premature infants. The treated group is provided with both intensive high-quality child care and specialist home visits [Hill, 2011]. The dataset also provides 25 covariates related to the children and their mothers. The outcome is the infants' cognitive test score, which is simulated by the setting "A" of NPCI package 2 . Additionally, a biased subset of the treated group is removed to create the correlation between the treatment assignment and the covariates. In all, there are 747 records in the dataset, with 139 in the treated group and 608 in the control group.

Results and Analysis

Table 2 shows the results of our proposed method as well as the baselines. Similar to the performance on News dataset, our proposed method consistently performs best under all three evaluation metrics, which demonstrates the effectiveness of imposing the conditional adversarial-treatment learning.

4.4 Experiment on CFPB Dataset

Dataset

The dataset comes from Consumer Financial Protection Bureau (CFPB) 3 . The CFPB solicits complaints from consumers across a variety of financial products and then addresses those complaints [Egami et al., 2018]. In this dataset, our goal is to analyze the effect of the financial company's public response to the consumers' disputation. In this dataset, we select the following responses as three treatments:

2 https://github.com/vdorie/npci

3 https://www.consumerfinance.gov/data-research/ consumer-complaints/

Type I. The company believes it acted appropriately as authorized by contract or law;

Type II. The company believes the complaint is the result of a misunderstanding;

Type III. The company disputes the facts presented in the complaint.

The treatment represents the company's overall attitude in dealing with the complaint. Different attitudes would affect customers' degree of satisfaction. The outcome is whether the consumer disputed. In total, there are 15, 187 consumers who receive the Type I response, 1, 512 consumers who receive the Type II response, 1, 468 consumers who receive the Type III response. The covariates are consumers' textual complaints, and each complaint is represented as the average of the word embeddings generated by Glove [Pennington et al., 2014].

1

Results and Analysis

Since the ground truth information on treatment effect is not available in the CFPB Dataset, we examine the matching quality of the paired complaints. Figure 3 shows the adjust mutual information (AMI) [Vinh et al., 2010] between the word occurrence and treatment assignment. Each point in the figure represents one word, the value of y-axis represents the AMI before matching, and the value of x-axis represents the AMI after matching. It can be observed from the figure that after matching, almost every word's AMI is close to zero, which indicates that our proposed method can effectively remove the information predictive to the treatment assignment.

Case Study

To better measure the matching quality, we show the matched complaints in Table 3 to validate whether the matched pairs are semantically similar or not. The length of compliants varies, and due to the space limit, we only list three short complaint pairs in the table. It is observed from Table 3 that the matched pairs are similar in our human sense: in the first row of Table 3, the three matched complaints are all about unexpected debt charging or account balancing; in the second row, the claims are all about wrongly placed debt or bill, and in the third row, the complaints are all about an unauthorized account.

5 RelatedWork

Due to its ability of estimating the changes in the outcome after making the intervention on the treatment selection, treatment effect estimation is prevalent across various domains [Guo et al., 2018; Imbens and Rubin, 2015].

Among the existing treatment effect estimation methods, few of them focus on the case where the dataset contains textual covariates. In [Egami et al., 2018], complimentary to our problem setting, the authors consider the case when the treatment or outcome is text, and present a framework to estimate the effect of text or the effect on text. In [Wood-Doughty et al., 2018], the text classifiers are integrated into the causal graph to recover the underlying distribution of other covariates, but the textual covariates are not involved in treatment effects. In [Mozer et al., 2018; Roberts et al., 2018], the authors adopt the topic model to represent the textual covariates and apply matching approaches to estimate the treatment effect. However, the topic model based matching approaches highly rely on the accuracy of the topic model and also ignore the nearly instrumental variables contained in the topic representation. Compared with existing approaches, our proposed CTAM method is flexible to any text representation and is capable of filtering out the information related to nearly instrumental variables, which effectively decreases the estimation bias.

In terms of the nearly instrumental variable related methods, various methods have been developed to handle the numerical covariates, such as covariate re-weighting [Kuang et al., 2017a; Chang and Dy, 2017; Diamond and Sekhon, 2013] and feature selection [Kuang et al., 2017b; Tibshirani, 1996; Rassen et al., 2011]. In our work, instead of restricting the filtering process on certain covariates, CTAM learns the latent representation of all covariates with the information related to the nearly instrumental variables removed.

6 Conclusions

Text data are almost everywhere in real life. However, few of the existing treatment effect estimation methods deal with the textual covariates. The major challenge of treatment effect estimation with textual covariates is how to effectively filter out the nearly instrumental variables which degrade the performance of the treatment effect estimation. Existing covariate selection or covariate re-weighting technologies cannot handle the challenge due to the abundant information contained in the textual covariates. Therefore, we propose a novel conditional treatment-adversarial training based matching method, named as CTAM. By imposing the conditional adversarial training procedure, CTAM can learn the latent representation of all covariates with information related to near instrumental variables discarded. Through the experiments on three datasets, it is demonstrated that our proposed CTAM method can improve the treatment effect estimation with textual covariates.

Acknowledgements

This work was supported in part by the US National Science Foundation under grants NSF-IIS 1747614 and IIS-1514204. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.