Towards Explainable Conversational Recommendation

1 Introduction

Recommender systems, which predict users' personalized preferences to items, have become one of the most effective techniques for overcoming information overload. Recently, researchers are reaching a consensus that both accuracy and explainability are essential for recommendation. Explainability requires that a system provides explanations about why items are recommended. It has been shown that enhanced explainability leads to improved user satisfaction, trust, and efficiency [Tintarev and Masthoff, 2007].

Serving as a bridge between users and recommender systems, explanations not only help users understand the working mechanisms of the models, but also trigger potential user feedbacks, e.g., inspiring users to inform the system when it is wrong [Tintarev and Masthoff, 2007]. Existing methods focus on providing single-turn explanations and lack the capability to incorporate user feedbacks [Sharma and Cosley, 2013; Zhang et al., 2014; He et al., 2015; Li et al., 2017; Wang et al., 2018; Chen et al., 2019; Gao et al., 2019]. For example, the explanation "This is a good documentary about the battle of thermopylae" for recommending a movie may

∗Xiting Wang is the corresponding author

Model: I recommend Pulp Fiction. This is a dark comedy with a great cast.

User: I don't want to watch a comedy right now.

Model: How about Ice Age? It is a very good anime with a lot of action adventure.

User: I don't like anime, but action movie sounds good.

Model: I recommend Mission Impossible. This is by far the best of the action series.

User: Sounds great. Thanks for the recommendation!

Predefined Template Recommended Item Generated Explanation

Figure 1: Conversation excerpts between a user and our explainable conversational recommendation model.

help a user realize why the recommendation is wrong, i.e., the model provides the recommendation based on his/her previous interest documentary. However, the user cannot communicate his/her findings with the system, e.g., his/her interest has recently shifted to thrillers.

In this paper, we introduce explainable conversational recommendation, which integrates user feedbacks into explainable recommendation to enable bidirectional user-model communications through conversations. As shown in Fig. 1, explainable conversational recommendation provides explanations to help users understand the model, and collects user feedbacks to understand and integrate user needs. During the multi-turn user-model collaboration process, both recommendations and explanations are iteratively refined.

Explainable conversational recommendation provides an alternative interaction paradigm between users and recommendation models. Prior works on conversational search and recommendation lack the capability for providing explanations [Christakopoulou et al., 2016; Zhang et al., 2018; Sun and Zhang, 2018; Li et al., 2018; Bi et al., 2019]. Since they cannot trigger feedbacks through explanations, these methods often collect user feedbacks by asking the what questions, e.g., "What category of movies do you like?". While these methods are effective for users who have a clear search target, they are less friendly for users wandering for interesting items. In comparison, users of explainable conversational recommendation can simply inform the system whether they like or dislike the features mentioned in the explanations, which reduces user cognitive load. The features in the explanations may also trigger users to come up with related features that they like or help users understand model imperfections. Both types of findings can be communicated to the system through user feedbacks.

While explainable conversational recommendation is promising, designing such a model is a challenging multiobjective problem. The model needs to effectively integrate user feedbacks to ensure significant and stable improvement of recommendation accuracy (O1) and explainability (O2). Moreover, the model should seldom violate the user requirements reflected in the feedbacks (O3). For example, if a user claims in the conversation that she does not like documentary, the model should take this explicit request seriously and avoids recommending documentaries. While objective O3 requires the model to satisfy specific user requirements (satisfaction), objectives O1 and O2 additionally request effective understanding of users by generalizing feedbacks to similar items and features (generalization).

In this paper, we aim to develop an Explainable Conversational Recommendation (ECR) model that fulfills objectives O1-O3. Our contributions are three-fold.

First, we design an incremental multi-task learning framework for explainable conversational recommendation. In our framework, multiple objectives can be simultaneously achieved through tight collaboration among the recommendation prediction task, the explanation generation task, and the user feedback integration module. The collaboration is achieved through context-aware modeling of item concepts extracted by using Microsoft Concept Graph 1 . Modeling the key concepts that a user likes about an item enables us to derive the cross knowledge between the two tasks (recommendation and explanation), trigger feedbacks about concepts, and integrate the feedbacks for incremental model update.

Second, we propose a multi-view feedback integration method to achieve effective incremental model update. Our method combines two views of incremental learning. The first view focuses on satisfying user requirements through local propagation of user needs (satisfaction), and the second view better generalizes user feedbacks by updating global model parameters, e.g., addressing model imperfections by learning a better concept embedding (generalization).

Third, we evaluate our method with different settings of simulated users and human assessors. The experiments demonstrate that our method consistently improves recommendation accuracy, increases explainability, and seldom violates user requirements (O1-O3).

2 ProblemFormulation

Fig. 2 shows the pipeline of explainable conversational recommendation. The model takes as input a user id u ∈ U, an item id v ∈ V , and side information Iu and Iv about the user and the item. The model then outputs 1) a score ru,v which predicts how much u likes v and 2) an linguistic explanation Yu,v for the recommendation that consists of a word sequence. Item v will be displayed to the user together with the explanation if ru,v is the largest among the candidates V . After the user checks the recommendation and the explanation, s/he will provide a feedback F, which helps to refine the recommendation model. This iterative process continues until 1) the user is satisfied with the recommendation or 2) the maximum number of communication turns is reached.

1 https://concept.research.microsoft.com/

Different choices of side information and user feedbacks result in different types of recommendation models.

Side information Iu and Iv. Many explainable recommendation methods take review comments as inputs to facilitate explanation generation [Zhang et al., 2014; Chen et al., 2018]. Following these methods, we consider Iu and Iv as review comments. Specifically, Iu denotes reviews user u writes and Iv is the set of reviews about item v: Iu = {Du,1, ...,Du,nd }, Iv = {Dv,1, ...,Dv,nd }, where nd is the maximum number of reviews. Each review Du,i or Dv,i is represented by a sequence of words. We also derive the concepts from reviews by utilizing Microsoft Concept Graph [Wu et al., 2012; Wang et al., 2015]. The concepts are a subset of the words that correspond to important explicit features mentioned in the review (e.g., documentary).

User feedbacks F. While free-form natural-language feedbacks empower users with the most flexibility, it is very likely that a model does not have the knowledge to process such feedbacks. For example, a model cannot correctly handle feedback "please recommend a movie that is currently showing in the local cinemas", because it does not know the location of the user or what movies are showing in the cinemas. We believe that a model should clearly define feasible feedbacks and inform the users. In our framework, concepts are the key for connecting different components. Thus, we allow users to provide feedbacks about what concepts they are (not) interested in together with their item interests: F = {c+1 , ..., c+ng } ∪ {c-1 , ..., c-nb } ∪ {v-}. F is the feedback provided at the current turn, c+i is a concept that u likes, and c-i (or v-) is a concept (or an item) that u is not interested in. To collect feedbacks about concepts, we can ask questions like "Are you interested in [CONCEPT]?") and parse feedbacks with aspect-level sentiment analysis tools [Zhang et al., 2014]. We may also collect concept-level feedbacks by requiring users to respond with pre-defined feedback templates. We set v- to the last item recommended to the user. If u likes the last recommended item, the conversation ends and no feedbacks will be provided.

3 Model Description

Fig. 3 shows the incremental multi-task learning framework of our Eexplainable Conversational Recommendation (ECR) model, which consists of two major parts. The first part, incremental cross knowledge modeling, learns the transferred

.

cross knowledge for the recommendation task and the explanation task, and illustrates how the cross knowledge can be updated by using incremental learning. The second part, incremental multi-task prediction, illustrates how we generate explanations based on the cross knowledge, and how we predict recommendation scores based on both the cross knowledge and user feedbacks.

We will introduce two parts in detail and illustrate how they can be jointly optimized to achieve end-to-end learning.

3.1 Incremental Cross Knowledge Modeling

A prior study shows that the transferred cross knowledge for the recommendation task and the explanation task can be modeled by using the key concepts that a user likes about an item [Chen et al., 2019]. We first illustrate how their concept embedding method can be extended to facilitate incremental feedback and improve model performance. Then, we show how concept importance can be computed by combining two views of incremental learning. Finally, we introduce how to select key concepts by considering concept importance from both views. The selected concepts will be included in the explanations to trigger feedbacks and used for recommendation prediction. As shown in Fig. 3, the incremental cross knowledge modeling part consists of four major modules:

Context-aware concept embedding

For each user u, we collect concepts cu,1, .., cu,nc that appear in reviews Du,1, ...,Du,nd and calculate their latent representations. While a concept can be represented by using its word embedding [Chen et al., 2019], we also find its context, i.e., related reviews, to be important for concept modeling. Thus, our embedding cu,i of concept cu,i is a concatenation of its word embedding and context embedding: cu,i = [cwu,i ; cnu,i ], where cwu,i is computed by using a word embedding lookup layer, and cnu,i is calculated by averaging the embeddings of related reviews, i.e., cnu,i = (∑j∈Γu,i du,j)/|Γu,i|. Here, du,j is the embedding of review Du,j, and Γu,i denotes the set of reviews that contains concept cu,i, i.e., Γu,i = {j|cu,i ∈ Du,j}. We follow [Tay et al., 2018] to calculate the review embedding d by summing up its word embeddings: d = ∑ω∈D w.

Given item v, we can similarly collect concepts in the reviews of v and calculate their context-aware embeddings.

Co-attentive concept importance modeling

Given concept embeddings cu,1, ..., cu,nc of user u and concept embeddings cv,1, ..., cv,nc of item v, we calculate coattention weight matrix Φ ∈ Rnc×nc to model deep useritem interactions: φi,j = f(cu,i)TWcf(cv,j), where Wc ∈ Rlw×lw is a learning weight matrix and f(·) is a lf -layered feed-forward neural network with activation function ReLU. We obtain user (item) concept importance vectors a and (b) to find concepts with the maximum co-attentions:

Here, ζ(·) is a Softmax function that ensures ∑i ai = 1 and ∑j bj = 1. ai (or bj) represents the probability that the concept in the reviews of u (or v) is important for the user-item pair. Concepts with larger ai or bj values have a larger probability to be selected and included in the explanations.

User feedbacks {c+i } and {c-j } provide ground-truth labels for important and unimportant concepts. These labels can be naturally incorporated by using a concept-level feedback loss:

Here, a+i , b+i (or a-j , b-j ) are the entries in a and b that correspond to c+i (or c-j ) and denote the probabilities for c+i (or c-j ) being considered important.

We consider minimizing LFc as a global view of incremental learning, since it fixes model imperfections that affect all user-item pairs, e.g., it refines concept embeddings. While this method is intuitive and useful for parameter refining, it usually fails to ensure satisfaction of user needs, e.g., significantly reduce the importance of c-j and remove it from the explanations. This is because LFc 's impact on ai and bj is indirect, i.e., can only be achieved by changing model parameters. Since the number of feedbacks is small, the influence of this indirect method is limited.

Local propagation of user-concept interest

To solve the aforementioned issue of indirect feedback integration, we use local label propagation, in which importance of concepts are measured according to their distance with c+i and c-j (the labels). To this end, we first calculate the aggregated embedding ̂c+ = αt ng (∑ng i=1 c+i ) + (1 - αt)p+, where p+ records the value of ̂c+ at the last turn, and 0< αt ≤1 balances current and previous feedbacks. Similarly, we have ̂c- = αt nb (∑nb i=1 c-)+(1-αt)p-. We then compute a preference score βc of the concept c based on the 2-norm distances between its embedding and the aggregated embeddings:

Accordingly to the triangle inequality, -||̂c+-̂c-||≤(||ĉc-||-||c-̂c+||)≤||̂c+-̂c-||. Thus, βc ∈ [-1, 1] and obtains the maximum (or minimum) value when c = ̂c+ (or c = ̂c-). The local concept importance vectors can be obtained by: a′i = ζ(βcu,i ) and b′j = ζ(βcv,j ).

Multi-view concept selection

The multi-view concept importance are computed with:

where αc ∈ R is a learnable parameter. We further ensure that the multi-view importance match user-concept interests reflected in the feedbacks by using a loss LFd similar with LFc

Based on ¯ai and ¯bj, we select a set of key concepts that will be included in the explanations and considered in recommendation prediction. To avoid the model being nondifferentiable, we use the Straight-Through Gumbel-Softmax function Gumbel(·) [Tay et al., 2018]:

Here, zu = (zu,1, ..., zu,nc ) ∈ {0, 1}nc . zu,i is 1 (or 0) if cu,i is (or is not) selected. Similarly, zv = (zv,1, ..., zv,nc ) ∈ {0, 1}nc denotes whether concepts {cv,j} are selected. Each time we calculate Gumbel(·), only one concept will be selected, i.e., ∑i zu,i = 1. However, a user may consider multiple concepts of an item. To select np concepts, we run Gumbel(·) np times with different Gumbel noises and coattention weight matrices Wc. As a result, we obtain multiple selected concepts {c(1) u , ..., c(np) u } and {c(1) v , ..., c(np) v }.

3.2 Incremental Multi-Task Prediction

We predict recommendation scores and generate explanations based on the selected concepts. Considered as concepts that user u likes about item v, the selected concepts can be used to 1) calculate user and item embeddings and 2) determine the key words that must appear in the explanations.

We first leverage the context-aware embeddings of the selected concepts to learn user and item embeddings eu = σ(Wp[c(1) u , ..., c(np) u ] +bp) and ev = σ(Wp[c(1) v , ..., c(np) v ] + bp). Here Wp ∈ Rlp×2nplw , bp ∈ Rlp and σ is the sigmoid function. Since explicit factors such as concepts and related reviews may fail to include all information about a user or an item, we additionally compute implicit user and items representations. In particular, a lookup layer is used to transform u (v) into implicit representations hu ∈ Rlu (hv ∈ Rlv ). The final user and embeddings are xu = [eu; hu] and xv = [ev; hv], which capture both explicit and implicit factors about a user or an item.

Multi-view recommendation

We predict recommendation scores based on two views. The global view learns a recommendation model based on all training samples. The local view ensures satisfaction of user feedbacks by considering local user-item interest.

Global FM-based recommendation. Factorization machine (FM) [Rendle, 2010] is used to predict recommendation score

where qi ∈ R is the i-th entry of q = [xu; xv]. wi ∈ R and mi ∈ Rlm are parameters to be learned.

The FM can be pre-trained with offline training instances Ωu by using the BPR loss [Rendle et al., 2009].

v′ ∈ Ωu denotes an item that u likes. For each v′ , an item v′′ that u does not like is obtained through negative sampling. Items v- provided in F can be integrated similarly:

Local estimation of user-item interest. Given F, we can compute a recommendation score based purely on whether feedbacks are satisfied. Specifically, estimated score ̂ ru,v is set to 0 if v=v- or if none of c+1 , ...c+ng is contained in the reviews of v. Otherwise, ̂ ru,v is set to 1. We then calculate the local recommendation score r′u,v with: r′u,v = αt̂ ru,v + (1-αt)τu,v, where τu,v is the r′u,v score at the previous turn. Multi-view combination. The final recommendation score is obtained by combining the two views: ¯ ru,v = (1 - αr)ru,v + αrr′u,v , where αr is a learnable parameter. We use LFs to learn ¯ ru,v :

Constrained explanation generation

We generate an linguistic explanation Yu,v by considering two types of constraints. The first type is a hard constraint, which requires that the first selected concept c(1) v must appear in the explanations. This better ensures that a concept that u likes will be included in the explanations. Note that c(1) v can only be selected if it appears in the item reviews. Thus, requiring c(1) v to appear in the explanation will not introduce false claims about item v in the explanation. The second type is soft constraints, which punish the model if other selected concepts are not included.

To satisfy the first constraint, we use a bi-directional generation method based on Gated Recurrent Unit (GRU) [Chung et al., 2014]. In particular, two GRUs are learned: a backward GRUb and a forward GRUf . GRUb considers c(1) v as the first generated word, and outputs all words before c(1) v . When GRUb reaches the start of the explanation (<SOS>), GRUf starts to generate words after c(1) v by considering all words output by GRUb. In both GRUs, the initial state is set to tanh(Wuxu +Wvxv + bs) ∈ Rls , where Wu, Wv, bs are parameters to be learned. The two GRUs can be trained by using the widely-adopted negative log-likelihood loss

where yu,v′ ,t∗ is the ground truth word at time t for the (u, v′ ) pair and ot,y is the probability of word y being generated.

To satisfy the second constraint, we use the concept relevance loss [Chen et al., 2019]. Let us denote the selected concepts as s ∈ R|V| , where |V| is vocabulary size. sk is 1 when the k-th word is a concept and has been selected for the (u, v′ ) pair and is 0 otherwise. The concept relevance loss is

3.3 Learning Process

As shown in Fig. 3, the model is first pre-trained with offline data Ωu. This is achieved by minimizing the following loss

LΩ = ∑u∈U (LΩu r +λnLΩu n +λcLΩu c ) +λθ‖Θ‖22 (12) where λs denote the importance of different losses and Θ represents the model parameters. During pre-training, local views are disabled, i.e., αc and αr are set to 0. We choose Adam [Kingma and Ba, 2014] as the optimizer.

When a feedback F is provided by u, we incrementally update the previous model parameters and learn local use interests. This is achieved by minimizing

LF = LFc +λdLFd +λrLFr +λsLFs +λ′θ ‖Θ-Θ′ ‖22 (13) Here, Θ′ represents the previous model parameters. When a sequence of feedbacks F1, ...,FT are given, we incrementally update the model by processing Ft one by one.

4 Experiment

4.1 Experimental Settings

Dataset. As shown in Table 1, we use three publicly available datasets. Electronics and Movie&TV are two categories of the Amazon dataset 2 , and Yelp contains restaurant reviews from Yelp Challenge 2016 3 . Each dataset is split into a training set (80%), a validation set (10%) and a test set (10%). The training set is used to derive Ωu , which consists of items that u provided reviews for. Following [Li et al., 2017; Chen et al., 2019], we consider the ground-truth explanations as the first sentence in the reviews. The validation set is used for model hyperparameter tuning and the test set is leveraged for simulating conversations. The goal of the model is to correctly predict the ground-truth items and explanations in the test set. If it fails to recommend the ground-truth item, user feedbacks will be provided to help it refine the results.

Feedback simulation. Most works on conversational search and recommendation directly use ground-truth concepts (or features) as user feedbacks [Bi et al., 2019]. In our setting, the ground-truth concepts are the ones mentioned in the ground-truth explanations (reviews). Different from previous works, we assume that a user will only give a groundtruth concept c+i when the generated explanation contains a concept that is similar with c+i . We consider the cosine similarity between BERT embeddings [Devlin et al., 2018] of the concepts and assume that the 20 most similar concepts of c+i will trigger c+i . We further assume that the user will point out a unimportant concept c-i if it appears in the explanation. Effects of different simulation settings are evaluated in Sec. 4.3.

2 http://jmcauley.ucsd.edu/data/amazon/

3 https://www.yelp.com/dataset/challenge

Baselines. We find that no existing works can be used directly for explainable conversational recommendation. To evaluate our method, we design two baselines by extending existing explainable recommendation methods NRT [Li et al., 2017] and CAML [Chen et al., 2019]. We train both models with the BPR loss. NRT is updated with λrLFr + λ′θ ‖Θ-Θ′ ‖22 . Other feedback losses cannot be used because NRT does not model concepts and cannot perform multi-view incremental learning. CAML is updated similarly, except that the concept-level loss LFc is also used.

Evaluation measures. We evaluate recommendation accuracy by using three widely-adopted measures: HR (hit ratio), NDCG (normalized discounted cumulative gain), and MRR (mean reciprocal rank). Following [Zhang et al., 2018], we calculate HR, NDCG and MRR based on the top 1, 10 and 100 recommended item(s), respectively. Following [Chen et al., 2019], the explanation quality is evaluated by using BLEU [Papineni et al., 2002] and ROUGE-L [Lin, 2004], which are widely adopted to measure the similarity between ground truth and generated texts. We also propose criterion CSR to measure the Concept-level feedback Satisfcation Ratio. We consider c+i to be satisfied if it appears in the generated explanation, and c-i to be satisfied if the concept is removed from the explanation.

Implementation details. For all methods, the maximum number of conversation turns is set to 5 and negative sampling is used to reduce the candidate size to 256. Most hyperparameters are set and tuned by following the papers of the baselines [Li et al., 2017; Chen et al., 2019]. The λs for balancing different losses are set to 1 except for λc=0.05. We tune np by performing grid search over {1, 2, .., 5}. The learning rates of LΩ and LF are set to 10-3 and 10-2 , respectively. αt is set to 0.8, and αc, αr are initialized by using 0.9 and automatically tuned during the learning process.

4.2 Overall Performance

Performance after 5 turns of conversation. Table 2 compares our method with the baselines in terms of objectives O1-O3. To facilitate comparison, we also test Ours-G. Similar with CAML, Ours-G considers only the global view (αc=αr=λd=λs=0). Note that all the evaluated methods take concept feedbacks as inputs except for NRT. The results show that our method significantly outperforms the baselines in terms of all three objectives. For example, compared with CAML, our method achieves 41.9% to 121.8% better HR (recommendation accuracy) and 42.0% average ROUGE-L improvement (explainability). Moreover, our method can almost always satisfy concept-level user feedbacks, with the CSR score ranging from 94% to 97%.

We draw three conclusions from the results. First, conceptlevel feedbacks are useful for model improvement, since the other methods perform better than NRT. Second, our model structure better facilitates feedback integration compared with CAML, since Ours-G outperforms CAML even when they use the same feedback integration method. This may be caused by our design of context-aware concept embeddings. Third, Ours outperforms Ours-G, which illustrates the importance of combining the global view with the local view.

Performance gain during conversation. Fig. 4 shows the recommendation accuracy and explainability of four methods at each conversation turn. As the number of conversation turns increases, the number of feedbacks increases, and the performance of all methods improves. Among all methods, ours increases the fastest, followed by Ours-G, CAML, and NRT. The explanation quality is highly related to the accuracy of selected concept and may become saturated once we find the ground-truth concept.

Human evaluation. Fig. 5 shows the human evaluation results on explanation quality, which illustrates that our explanations are considered more fluent and useful than the baselines. We hire three experienced human assessors to label the explanations generated after 5 turns of conversation. 100 test cases are sampled from the Electronics dataset, and each assessor labels whether an explanation is fluent and whether it is useful. Explanations of NRT, CAML, and Ours are provided to the assessors in random order. The Fleiss Kappa score is 0.423 for fluency and 0.303 for usefulness, which shows a moderate agreement and a fair agreement among assessors.

4.3 Ablation Analysis

Effectiveness of multi-view learning. Table 3 shows that our final model (Ours) outperforms the model with only the global (Ours-G) or local (Our-L) view in terms of recommendation accuracy and explainability. This demonstrates the effectiveness of multi-view learning. The reason is that Ours-G has not fully integrated user feedbacks while Ours-L ignores users' personal interests. The CSR of Ours-L is 1.0 since it considers only user feedbacks.

Effect of different user simulations settings. Table 3 shows the result of our model when only c+1 , ...c+ng (OursP) or c-1 , ...c-nb (Ours-N) are provided as feedbacks. Ours outperforms Ours-P and Ours-N, which indicates that our method can leverage both types of user feedbacks to improve performance. By comparing this result with the result at conversation turn 0 (Fig. 5), we can find that Ours-N successfully improves HR and explainability by using only 5 negative feedbacks (no ground-truth concepts provided).

5 Conclusion

We propose a framework for explainable conversational recommendation, which enables tight collaboration between the recommendation task, the explanation generation task, and the incremental feedback integration module. A multi-view method is also proposed to effectively incorporate user feedbacks. Experiments show that our approach achieves stable and significant improvement of both recommendation and explainability, and can effectively satisfy user requirements.

Acknowledgments

This research was partially supported by grants from the National Natural Science Foundation of China (Grants No. U1605251, 61727809).