Explainable Recommendation via Interpretable Feature Mapping and Evaluation of Explainability

1 Introduction

Since the inception of the Netflix Prize competition, latent factor collaborative filtering (CF) has been continuously adopted by various recommendation tasks due to its strong performance over other methods [Koren et al., 2009], which essentially employs a latent factor model such as matrix factorization and/or neural networks to learn user or item feature representations for rendering recommendations. Despite much success, latent factor CF approaches often suffer from the lack of interpretability [Zhang and Chen, 2018]. In a contemporary recommender system, explaining why a user likes an item can be as important as the accuracy of the rating prediction itself [Zhang and Chen, 2018].

Explainable recommendation can improve transparency, persuasiveness and trustworthiness of the system [Zhang et al., 2019]. To make intuitive explanation for recommendations, recent efforts have been focused on using metadata such as user defined tags and topics from user review texts or item descriptions[Lu et al., 2018; Chen et al., 2018] to illuminate users preferences. Other works such as [Hou et al., 2019; He et al., 2015; Zhang et al., 2014] use aspects to explain recommendations. Although these approaches can explain recommendation using external metadata, the interpretability of the models themselves and the interpretable features enabling the explainable recommendations have still not been systematically studied and thus, are poorly understood. It is also worth mentioning that the challenges in explainable recommendation not only lie in the modeling itself, but also in the lack of a gold standard for evaluation of explainability.

∗Corresponding author

Here we propose a novel feature mapping strategy that not only enjoys the advantages of strong performance in latent factor models but also is capable of providing explainability via interpretable features. The main idea is that by mapping the general features learned using a base latent factor model onto interpretable aspect features, one could explain the outputs using the aspect features without compromising the recommendation performance of the base latent factor model. We also propose two new metrics for evaluating the quality of explanations in terms of a user's general preference over all items and the aspect preference to a specific item. Simply put, we formulate the problem as: 1) how to find the interpretable aspect basis; 2) how to perform interpretable feature mapping; and 3) how to evaluate explanations.

We summarize our main contributions as follows: 1) We propose a novel feature mapping approach to map the general uninterpretable features to interpretable aspect features, enabling explainability of the traditional latent factor models without metadata; 2) Borrowing strength across aspects, our approach is capable of alleviating the trade-off between

propose new schemes for evaluating the quality of explanations in terms of both general user preference and specific user preference.

2 RelatedWork

There are varieties of strategies for rendering explainable recommendations. We first review methods that give explanations in light of aspects, which are closely related to our work. We then discuss other recent explainable recommendation works using metadata and knowledge in lieu of aspects.

2.1 Aspect Based Explainable Recommendation

Aspects can be viewed as explicit features of an item that could provide useful information in recommender systems. An array of approaches have been developed to render explainable recommendations at the aspect level using metadata such as user reviews. These approaches mostly fall into three categories: 1) Graph-based approaches: they incorporate aspects as additional nodes in the user-item bipartite graph. For example, TriRank [He et al., 2015] extract aspects from user reviews and form a user-item-aspect tripartite graph with smoothness constraints, achieving a review-aware top-N recommendation. ReEL [Baral et al., 2018] calculate useraspect bipartite from location-aspect bipartite graphs, which infer user preferences. 2) Approaches with aspects as regularizations or priors: they use the extracted aspects as additional regularizations for the factorization models. For example, AMF [Hou et al., 2019] construct an additional useraspect matrix and an item-aspect matrix from review texts, as regularizations for the original matrix factorization models. JMARS [Diao et al., 2014] generalize probabilistic matrix factorization by incorporating user-aspect and movie-aspect priors, enhancing recommendation quality by jointly modeling aspects, ratings and sentiments from review texts. 3) Approaches with aspects as explicit factors: other than regularizing the factorization models, aspects can also be used as factors themselves. [Zhang et al., 2014] propose an explicit factor model (EMF) that factorizes a rating matrix in terms of both predefined explicit features (i.e. aspects) as well as implicit features, rendering aspect-based explanations. Similarly, [Chen et al., 2016] extend EMF by applying tensor factorization on a more complex user-item-feature tensor.

2.2 Beyond Aspect Explanation

There are also other approaches that don't utilize aspects to explain recommendations. For example, [Lee and Jung, 2018] give explanations in light of the movie similarities defined using movie characters and their interactions; [Wang et al., 2019] propose explainable recommendations by exploiting knowledge graphs where paths are used to infer the underlying rationale of user-item interactions. With the increasingly available textual data from users and merchants, more approaches have been developed for explainable recommendation using metadata. For example, [Chen et al., 2019b; Costa et al., 2018; Lu et al., 2018] attempt to generate textual explanations directly whereas [Wu et al., 2019; Chen et al., 2019a] give explanations by highlighting the most important words/phrases in the original reviews.

Overall, most of the approaches discussed in this section rely on metadata and/or external knowledge to give explanations without interpreting the model itself. In contrast, our Attentive Multitask Collaborative Filtering (AMCF) approach maps uninterpretable general features to interpretable aspect features using an existing aspect definition, as such it not only gives explanations for users, but also learns interpretable features for the modelers. Moreover, it is possible to adopt any latent factor models as the base model to derive the general features for the proposed feature mapping approach.

3 The Proposed AMCFModel

In this section, we first introduce the problem formulation and the underlying assumptions. We then present our AMCF approach for explainable recommendations. AMCF incorporates aspect information and maps the latent features of items to the aspect feature space using an attention mechanism. With this mapping, we can explain recommendations of AMCF from the aspect perspective. An Aspect s [Bauman et al., 2017] is an attribute that characterizes an item. Assuming there are totally m aspects in consideration, if an item has aspects si1 , ..., sik simultaneously, an item can then be described by Ii = {si1 , si2 , ..., sik }, k ≤ m. We say that an item i has aspect sj, if sj ∈ Ii.

3.1 ProblemFormulation

Inputs. The inputs consist of 3 parts: the set of users U, the set of items V , and the set of corresponding multi-hot aspect vectors for items, denoted by S.

Outputs. Given the user-item-aspect triplet, e.g. user i, item j, and aspect multi-hot vector sj for item j, our model not only predicts the review rating, but also the user general preference over all items and the user specific preference on item j in terms of aspects, i.e., which aspects of the item j that the user i is mostly interested in.

3.2 Rationale

The trade-off between model interpretability and performance states that we can either achieve high interpretability with simpler models or high performance with more complex models that are generally harder to interpret [Zhang and Chen, 2018]. Recent works [Zhang and Chen, 2018; He et al., 2015; Zhang et al., 2014] have shown that with adequate metadata and knowledge, it is possible to achieve both explainability and high accuracy in the same model. However, those approaches mainly focus on explanation of the recommendation, rather than exploiting the interpretability of the models and features, and hence are still not interpretable from modeling perspective. Explainability and interpretability refer to "why" and "how" a recommendation is made, respectively. Many above-referenced works only answer the "why" question via constraints from external knowledge without addressing "how". Whereas our proposed AMCF model answers both "why" and "how" questions, i.e., our recommendations are made based on the attention weights (why) and the weights are learned by interpretable feature decomposition (how). To achieve this, we assume that an interpretable aspect feature representation can be mathematically derived from the corresponding general feature representation. More formally:

Assumption 1. Assume there are two representations for the same prediction task: u in complex feature space U (i.e. general embedding space including item embedding and aspect embedding), and v in simpler feature space V (i.e. space spanned by aspect embeddings), and V ⊂ U. We say that v is the projection of u from space U to space V, and there exists a mapping M(·, θ), such that v = M(u, θ), with θ as a hyper-parameter.

This assumption is based on the widely accepted notion that a simple local approximation can give good interpretation of a complex model in that particular neighborhood [Ribeiro et al., 2016]. Instead of selecting surrogate interpretable simple models (such as linear models), we map the general complex features to the simpler interpretable aspect features, then render recommendation based on those general complex features. We give explanations using interpretable aspect features, achieving the best of both worlds in keeping the high performance of the complex model as well as gaining the interpretability of the simpler model. In this work, the interpretable simple features are obtained based on aspects, hence we call the corresponding feature space as aspect space. To map the complex general features onto the interpretable aspect space, we define the aspect projection.

Definition 1. (Aspect Projection) Given Assumption 1, we say v is an aspect projection of u from general feature space U to aspect feature space V (Figure 1).

To achieve good interpretability and performance in the same model, from Definition 1 and Assumption 1, we need to find the mapping M(·, θ). Here we first use a latent factor model as the base model for explicit rating prediction, which learns general features, as shown in Figure 2 (left, Lpred), where we call the item embedding u as the general complex feature learned by the base model. Then the remaining problem is to derive the mapping from the non-interpretable general features to the interpretable aspect features.

3.3 Aspect Embedding

To design a simple interpretable model, its features should be well aligned to our interest, e.g. the aspects is a reasonable choice. Taking movie genre as an example: if we use 4 genres (Romance, Comedy, Thriller, Fantasy) as 4 aspects, the movie Titanic's aspect should be represented by (1, 0, 0, 0) because it's romance genre, and the movie Cinderella's aspect is (1, 0, 0, 1) because it's genre falls into both romance and fantasy.

From Assumption 1 and Definition 1, to make the feature mapping from a general feature u to an aspect feature v, we need to first define the aspect space V. Assuming there are m aspects in consideration, we represent the m aspects by m latent vectors in general space U, and use these m aspect vectors as the basis that spans the aspect space V ⊂ U. These aspects' latent vectors can be learned by neural embedding or other feature learning methods, with each aspect corresponding to an individual latent feature vector. Our model uses embedding approach to extract m aspect latent vectors of ndimension, where n is the dimension of space U. In Figure 2, the vertical columns in red (ψ1, ...,ψm) represent m aspect embeddings in the general space U, which is obtained by embedding the aspect multi-hot vectors from input.

3.4 Aspect Projection of ItemEmbedding

In Assumption 1, u is the general feature representation (i.e. the item embedding) in space U, and v is the interpretable aspect feature representation in space V. The orthogonal projection from the general space U to the aspect space V is denoted byM, i.e. v =M(u).

From the perspective of learning disentangled representations, the item embedding u can be disentangled as u = v+b (Figure 1), where v encodes the aspect information of an item and b is the item-unique information. For example, movies from the same genre share similar artistic style (v) yet each movie has its own unique characteristics (b). With this disentanglement of item embeddings, we can explain recommendation via capturing user's preference in terms of aspects.

Let's assume that we have m linearly independent and normalized aspect vectors (ψ1, ...,ψm) in space U, which span subspace V. For any vector v =M(u) in space V, there exists an unique decomposition such that v = ∑mi=1 viψi. The coefficients can be directly calculated by vi = v ·ψi = u·ψi, (i = 1, ...,m, ψi is normalized). Note that the second equality comes from the fact that v is the orthogonal projection of u on space V.

Generally speaking, however, (ψ1, ..., ψm) are not orthogonal. In this case, as long as they are linearly independent, we can perform Gram-Schmidt orthogonalization process to obtain the corresponding orthogonal basis. The procedure can be simply described as follows: ˜ψ1 = ψ1; and ˜ψi = ψi - ∑i-1 j=1 〈ψi, ˜ψj〉˜ψj, where 〈ψi, ˜ψj〉 denotes inner product. We can then calculate the unique decomposition as in the orthogonal cases. Assume the resulting decomposition is v = ∑mi=1 ˜ vi ˜ψi, the coefficients corresponding to the original basis (ψ1, ...,ψm) can then be calculated by: vi = ˜ vi - ∑mj=i+1 〈ψi, ˜ψj〉; and vm = ˜ vm.

Hence, after the aspect feature projection and decomposition, regardless of orthogonal or not, we have the following unique decomposition in space V: v = ∑mi=1 viψi.

Aspect Projection via Attention. As described above, any interpretable aspect feature v can be uniquely decomposed as v = ∑mi=1 viψi, which is similar to the form of attention mechanism. Therefore, instead of using Gram-Schmidt orthogonalization process, we utilize attention mechanism to reconstruct v directly. Assume we can obtain an attention vector a = (a1, ..., am), which can be used to calculate ˆv = ∑mi=1 aiψi, with the fact that the decomposition is unique, our goal is then to minimize the distance ||ˆv-v||2 to ensure that ai ≈ vi.

However, as the interpretable aspect feature v is not available, we cannot minimize ||ˆv - v||2 directly. Fortunately, the general feature u is available (obtained from a base latent factor model), with the fact that v is the projection of u, i.e. v =M(u), we have the following lemma:

Lemma 1. Provided that v is the projection of u from space U to space V, where V ⊂ U, we have

where ˆv = ∑mi=1 aiψi, a = (a1, ..., am), and || · ||2 denotes l2 norm.

Proof. Refer to the illustration in Figure 1, and denote the difference between u and v as b, i.e. u = v + b. Hence

Note that b is perpendicular to ˆv and v, the right hand side can then be written as

From the above proof, we know that attention mechanism is sufficient to reconstruct v ≈ ˆ v = ∑mi=1 aiψi by minimizing ||ˆv -u||2. Note that from the perspective of disentanglement u = v + b, the information in b, i.e., the item specific characteristics, is not explained in our model. Intuitively, the item specific characteristics are learned from the metadata associated with the item.

3.5 The Loss Function

The loss function for finding the feature mapping M(·) to achieve both interpretability and performance of the recommender model has 2 components:

• Lpred prediction loss in rating predictions, corresponding to the loss function for the base latent factor model.
• Lint interpretation loss to the general feature u. This loss is to quantify ||ˆv -u||2.

We calculate the rating prediction loss component using RMSE: Lpred = √ 1N ∑(i,j)∈Observed (rij - ˆ rij)2 , where ˆ rij represents the predicted item ratings. We then calculate the interpretation loss component as the average distance between u and ˆv: Lint = 1N ∑(i,j)∈Observed ||ˆv-u||2. The loss component Lint encourages the interpretable feature ˜v obtained from the attentive neural network to be a good approximation of the aspect feature representation v (Lemma 1). Hence the overall loss function is L = Lpred + λLint, where λ is a tuning parameter to leverage importance between the two loss components.

Gradient Shielding Trick. To ensure that interpretation doesn't compromise the prediction accuracy, we allow forward propagation to both Lint and Lpred but refrain the backpropagation from Lint to the item embedding u. In other words, when learning the model parameters based on backpropagation gradients, the item embedding u is updated only via the gradients from Lpred.

3.6 User Preference Prediction

Thus far we attempt to optimize the ability to predict user preference via aspect feature mapping. We call the user overall preference as general preference, and the user preference on a specific item as specific preference.

General preference. Figure 3 illustrates how to make prediction on user general preference. Here we define a virtual item ˆ vk, which is a linear combination of aspect embeddings. For general preference, we let ˆvk = ψk to simulate a pure aspect k movie, the resulting debiased (discarded all bias terms) rating prediction ˆ pk indicates the user's preference on such specific aspect (e.g., positive for 'like', negative for 'dislike'). Formally: ˆ pk = f(q, ˆvk), where q is the user embedding, ˆvk = ψk is the aspect k's embedding, and f(·) is the corresponding base latent factor model without bias terms.

Specific preference. Figure 3 also shows our model's ability to predict user preference on a specific item, as long as we can find how to represent them in terms of aspect embeddings. Fortunately, the attention mechanism is able to help us find the constitution of any item in terms of aspect embeddings using the attention weights. That is, for any item, it is possible to rewrite the latent representation u as a linear combination of aspect embeddings: u = ˆ v + b = ∑k akψk + b, where ak and ψk are the k-th attention weight and the k-th aspect feature, respectively. The term b reflects interpretation loss. For aspect k of an item, we use ˆ vk = akψk to represent the embedding of a virtual item which represents the aspect k property of the specific item. Hence, the output ˆ pk indicates the specific preference on aspect k of a specific item.

Model Interpretability. From specific preference, a latent general feature can be decomposed into the linear combination of interpretable aspect features, which would help interpret models in a more explicit and systematic manner.

4 Experiments and Discussion

We design and perform experiments to demonstrate two advantages of our AMCF approach: 1) comparable rating predictions; 2) good explanations on why a user likes/dislikes an item. To demonstrate the first advantage we compare the rating prediction performance with baseline approaches of rating prediction only methods. The demonstration of the second advantage, however, is not a trivial task since currently no gold standard for evaluating explanation of recommendations except for using real customer feedback[Chen et al., 2019b; Gao et al., 2019]. Hence it's necessary to develop new schemes to evaluate the quality of explainability for both general and specific user preferences.

4.1 Datasets

MovieLens Datasets. This data set [Harper and Konstan, 2016] offers very complete movie genre information, which provides a perfect foundation for genre (aspect) preference prediction, i.e. determining which genre a user likes most. We consider the 18 movie genres as aspects.

YahooMovies Dataset. This data set fromYahoo Lab contains usual user-movie ratings as well as metadata such as movie's title, release date, genre, directors and actors. We use the 29 movie genres as the aspects for movie recommendation and explanation. Summary statistics are shown in Table 1.

Pre-processing. We use multi-hot encoding to represent genres of each movie or book, where 1 indicates the movie is of that genre, 0 otherwise. However, there are still plenty of movies with missing genre information, in such cases, we simply set them as none of any listed genre, i.e., all zeros in the aspect multi-hot vector: (0, 0, ..., 0).

4.2 Results of Prediction Accuracy

We select several strong baseline models to compare rating prediction accuracy, including non-interpretable models, such as SVD [Koren et al., 2009], Neural Collaborative Filtering (NCF) [He et al., 2017] and Factorization Machine (FM) [Rendle, 2010], and an interpretable linear regression model (LR). Here the LR model is implemented by using aspects as inputs and learning separate parameter sets for different individual users. In comparison, our AMCF approaches also include SVD, NCF or FM as the base model to demonstrate that the interpretation module doesn't compromise the prediction accuracy. Note that since regular NCF and FM are designed for implicit ratings (1 and 0), we replace their last sigmoid output layers with fully connected layers in order to output explicit ratings.

In terms of robustness, we set the dimension of latent factors in the base models to 20, 80, and 120. The regularization tuning parameter λ is set to 0.05, which demonstrated better performance compared to other selections. It is worth noting that the tuning parameters of the base model of our AMCF approach are directly inherited from the corresponding non-interpretable model. We compare our AMCF models with baseline models as shown in Table 2. It is clear that AMCF achieves comparable prediction accuracy to their noninterpretable counterparts, and significantly outperforms the interpretable LR model.

4.3 Evaluation of Explainability

Despite the recent efforts have been made to evaluate the quality of explanation by defining explainability precision (EP) and explainability recall (ER)[Peake and Wang, 2018; Abdollahi and Nasraoui, 2016], the scarcity of ground truth such as a user's true preference remains a significant obstacle for explainable recommendation. [Gao et al., 2019] make an initial effort in collecting ground truth by surveying real customers, however, the labor intense, time consuming and sampling bias may prevent its large-scale applications in a variety of contexts. Other text-based approaches [Costa et al., 2018; Lu et al., 2018] can also use natural language processing (NLP) metrics such as Automated Readability Index (ARI) and Flesch Reading Ease (FRE). As we don't use metadata such as text reviews in our AMCF model, user review based explanation and evaluation could be a potential future extension to our model.

Here we develop novel quantitative evaluation schemes to assess our model's explanation quality in terms of general preferences and specific preferences, respectively.

General Preference

Let's denote the ground truth of user general preferences as pi for user i, and the model's predicted preference for user i is ˆpi, we propose measures inspired by Recall@K in recommendation evaluations.

TopM recall at K. (TM@K): Given theM most preferred aspects of a user i from pi, top M recall at K is defined as the ratio of the M aspects located in the top K highest valued aspects in ˆpi. For example, if pi indicates that user i's top 3 preferred aspects are Adventure, Drama, and Thriller, while the predicted ˆpi shows that the top 5 are Adventure, Comedy, Children, Drama, Crime, the top 3 recalls at 5 (T3@5) is then 2/3 whereas top 1 recall at 3 (T1@3) is 1.

Bottom M recall at K. (BM@K): Similarly defined as above, except that it measures the most disliked aspects.

As the ground truth of user preferences are usually not available, some reasonable approximations are needed. Hence we propose a method to calculate the so-called surrogate ground truth. First we define the weights wij = (rij - bui - bvj - ¯ r)/A, where the weight wij is calculated by nullifying user bias bui , item bias bvj , and global average ¯ r, and A is a constant indicating the maximum rating (e.g. A = 5 for most datasets). Note that user bias bui and item bias bvj can be easily calculated by bui = ( 1 |Vi | ∑j∈Vi rij)-¯ r, and bvj = ( 1 |Uj | ∑i∈Uj rij) - ¯ r. Here Vi represents the sets of items rated by user i, and Uj represents the sets of users that have rated item j. With the weights we calculate user i's preference on aspect t using the following formula: pti = ∑j∈Vi wijstj , where stj = 1 if item j has aspect t, 0 otherwise. Hence a user i's overall preference can be represented by an l1 normalized vector pi = (p1i , ..., pti , ..., pTi )/||pi||1. As our model can output a user preference vector directly, we evaluate the explainability by calculating the average of TM@K and BM@K. The evaluation results are reported in Table 3. We observe that the explainability of AMCF is significantly better than random interpretation, and is comparable to the strong interpretable baseline LR model with a much better prediction accuracy. Thus our AMCF model successfully integrates the strong prediction performance of a latent factor model and the strong interpretability of a LR model.

Specific Preference

Our approach is also capable of predicting a user's preference on a specific item, i.e. ˆpij, showing which aspects of item j are liked/disliked by the user i. Compared to user general preference across all items, the problem of which aspect of an item attracts the user most (specific preference) is more interesting and more challenging. There is no widely accepted strategy to evaluate the quality of single item preference prediction (except for direct customer survey). Here we propose a simple yet effective evaluation scheme to illustrate the quality of our model's explanation on user specific preference. With the overall preference pi of user i given above, and assuming sj is the multi-hot vector represents the aspects of item j, we say the element-wise product pij = pi ⊙sj reflects the user's specific preference on item j.

Note that we should not use the TM@K/BM@K scheme as in general preference evaluation, both pij and predicted ˆ pij's entries are mostly zeros, since each movie is only categorized into a few genres. Hence the quality of specific preference prediction is expressed using a similarity measure. We use s(pij, ˆpij) to represent the cosine similarity between pij and ˆpij, and the score for specific preference prediction is defined by averaging over all user-item pairs in the test set: scores = 1N ∑ij s(pij, ˆpij). We report the results of specific user preferences in the scores column of Table 3. As the LR cannot give specific user preferences directly, we simply apply ˆ pij = ˆ pi ⊙ sj where ˆ pi represents the general preference predicted by LR.

An insight. Assume that for a specific user i, our AMCF model can be simply written as ˆ rij = fi(uj) to predict the rating for item j. Note that our AMCF model can decompose the item in terms of aspects. Lets denote these aspects as {ψ1, ...ψm}. Then the prediction can be approximated by ˆ rij ≈ fi(∑mk=1 ajkψk), where ajk denote the k-th attention weights for item j. In the case of LR, the rating is obtained by ˆ rij = gi(∑mk=1 bkxk), where gi is the LR model for user i, bk is the k-th coefficient of it, and xk represents the indicator of aspect k, xk = 1 when the item has aspect k, xk = 0 otherwise. The similarity between AMCF formula and LR formula listed above indicates that the coefficients of LR and the preference output of AMCF share the same intrinsic meaning, i.e., both indicate the importance of aspects.

An example. For specific explanation, given a user i and an item j, our AMCF model predicts a vector p, representing the user i's specific preference on an item j in terms of all predefined aspects. Specifically, the magnitude of each entry of p (i.e. |pi|) represents the impact of a specific aspect on whether an item liked by a user or not. For example, in Figure 4, the movie 120 is high-rated by both users 65 and 74, however, with differential explanations: the former user preference is more on the Action genre whereas the latter is more on Sci-Fi and War. On the other hand, the same movie is low-rated by user 67 mainly due to the dislike of Action genre.

5 Conclusion

Modelers tend to better appreciate the interpretable recommender systems whereas users are more likely to accept the explainable recommendations. In this paper, we proposed a novel interpretable feature mapping strategy attempting to achieve both goals: systems interpretability and recommendation explainability. Using extensive experiments and tailormade evaluation schemes, our AMCF method demonstrates strong performance in both recommendation and explanation.

Acknowledgements

This work is supported by the National Science Foundation under grant no. IIS-1724227.