Interpretable Recommendation via Attraction Modeling: Learning Multilevel Attractiveness over Multimodal Movie Contents

1 Introduction

Nowadays, new media, e.g., social websites and online video stream, dominates the traffic of Internet. Huge amount of new contents including news, blogs and videos, are produced in every second. As a result, a lot of content-sharing platforms have emerged in recent years. For example, Twitter is an online news and social networking service where users post and interact with messages. Users will be recommended the latest news that most potentially attract them when they login. By 2010, Netflix's streaming business has become the largest source of Internet streaming traffic in North America in the evening. As reported, "About 75 percent to 80 percent of what people watch on Netflix comes from what Netflix recommends, not from what people search for" [Timothy, 2013]. Obviously, recommendation becomes the first choice for users to consume contents in the new media age.

Classical recommender systems (RS) may help users to find interesting contents according to similar users' history with collaborative filtering (CF) technique. However, CF cannot work in the cold-start cases, e.g., a news article has not been rated by any users. Content-based filtering (CBF) [de Gemmis et al., 2015] finds contents by the semantic similarity so it does not suffer from the above issue. Intuitively, the user selection of contents is often determined by some attractive points, e.g., an actor of a movie. However, current CBF approaches cannot capture the attractive points leading to user selection. In recent years, more and more researchers argue that only focusing on improving the accuracy may hurt RS [McNee et al., 2006; Hu et al., 2017b]. Following this argument, instead of aiming at higher recommendation accuracy, we pay more attention to finding and interpreting the attractive points in available contents, although our model can still achieve comparable accuracy performance.

Specially, we aim to model user attractiveness over contents to interpret user selection by assuming that attraction is one of the strongest motivations to make the final decision. For example, a person may choose a movie due to the attraction on a movie star ignoring other factors of that movie. Similarly, we may like a song due to one or two heart-touching lyrics even though we cannot remember the whole song. Obviously, these attractive points instead of the whole movie or song result in the selection. Furthermore, attraction is a subjective feeling which is different from person to person. For example, a person may like a movie as s/he is attracted by an actress while another person selects this movie as attracted by the director. Therefore, modeling attraction is a critical task to interpret user behavior, which is not limited to interpret user selection in RSs. Attention mechanism can assign focus on selective parts where a related context is given. It has been shown effective in various tasks such as machine translation [Bahdanau et al., 2014] and image captioning [Vinyals et al., 2015]. Obviously, attention mechanism shares some common ideas with attraction modeling. The main difference is that attention mechanism focuses on salient parts of an object in an objective way without considering user difference whereas attraction modeling aims to find personal focus on a content in a subjective way. In this work, we incorporate user context into attention mechanism for modeling subjective attraction.

Since the internet video stream has accounted for the major traffic in this new media age, in this paper, we take online movies as the representative case to study attraction modeling. In particular, the story and the cast members, e.g., actors, directors and writers, are two most important aspects of a movie to attract audience. On one hand, when a person reads the story of a movie, s/he may be caught by some attractive words in a sentence. Moreover, only a few sentences of the core plot instead of all sentences may actually attract user's attention. Accordingly, we build a multilevel neural model on the story (textual content) to capture word-level, sentencelevel, and story-level attraction. On the other hand, cast members (categorical content) of a movie are another important factor to attract users so we build another neural model to weight the attraction over each cast member and generate a representation of the whole cast. At the top level, we create a joint representation of story and cast representation to score attractiveness. Due to the complementation of story (textual data) and cast members (categorical data), we build a multimodal neural model to integrate the information from both types of data to comprehensively capture user attraction.

The contributions of this paper are summarized as follows:

• We model human attraction on new contents, using movie content as the study case, to capture and interpret the motivation on user selections.
• A multimodal neural network model with subjective attention mechanism is designed to learn the multilevel personal attraction on the story and the attraction on the cast members of a movie.
• Extensive experiments on a real-world dataset are conducted to evaluate the above design. All results show that our approach can achieve comparable performance with the state-of-the-art methods. Moreover, we demonstrate the statistical user attractiveness on movies to interpret the recommendation results.

2 RelatedWork

Briefly, we present the two most relevant aspects of our work: (1) the content-based recommender systems, and (2) the attention mechanism on contents.

CF is not capable of recommending new content because there is no other users' feedback on this new item. In comparison, the CBF approach can recommend the latest contents, but it may lead to overspecialization when a user is associated with very limited contents [Balabanovi´c and Shoham, 1997]. Kompan et al. [2010] proposed a content-based news recommendation method, where users are assigned into a cluster of similar news articles according to their browse history. Since each user is assigned to a news cluster, this method lacks of personalization for each user and fails to model the diversity of user's tastes across multiple news clusters. To take advantage of both CF and CBF, some hybrid recommendation methods were proposed. fLDA [Agarwal and Chen, 2010] generalized the supervised topic model (sLDA) [Mcauliffe and Blei, 2008] by using the latent topics learning from textual content for recommendation. Similar to fLDA, Wang et al. [2011] proposed a hybrid method for scientific article recommendation by combining matrix factorization and LDA. Musto et al. [Musto et al., 2016] proposed to learn word embedding from Wikipedia, and represent user profile as the centroid of the embedding vectors of the items the user previously liked. The goal of these methods is recommendation other than interpreting user attraction as in this work.

Attention mechanism has been shown effective in various tasks such as machine translation [Bahdanau et al., 2014] and image captioning [Vinyals et al., 2015]. Recently, some researchers have employed attention mechanism to model textual content. Yang et al. [2016] proposed hierarchical attention networks for document classification, where attention mechanisms are respectively applied at the word and sentence level, enabling it to attend to more or less important content when constructing the document representation. Denil et al. [2014] use convolutional neural networks (CNN) to transform word embedding in each sentence into the embedding for the entire sentence. At the document level, another CNN is used to transform sentence embedding into a document embedding vector. These methods try to find salient words or sentences from documents without considering user factors. To relieve the workload of editors for selecting articles, Wang et al. [2017] proposed a dynamic attention deep model (DADM) to recommend articles, where each article is represented by a vector using character-level text modeling [Kim et al., 2016]. However, these attentive words and sentences do not mean the attractive points to all users since each user may have quite different attention. Moreover, the content of an item, e.g., a movie, often consists of multiple types of data, not limited to text. Therefore, we design a multilevel attraction neural network to model personal attention on multimodal contents.

3 Problem Statement

We denote the movie set as M = {m1, · · · ,mN}. For each movie m ∈ M, it consists of a textual story, St, and a set of cast members Cm = {c1, · · · , cNm }. For each story St, it consists of Nt sentences, St = {s1, · · · , sNt }. For each sentence s ∈ St, it consists of a set of Ns words, {ws,1, · · · ,ws,Ns }. We denote the user set as U to model their attraction on movies. Given a user u ∈ U, her user profile about previously liked movies is denoted Mu = {mu,1, · · · ,mu,Nu }.

Given a movie m ∈ Mu, one of our tasks is to learn the attractiveness over words for each sentence, and the attractiveness over sentences of the story St from user u's perspective and generate story-level representation ht . Another task is to weight the attractiveness on the cast members Cm and generate attraction-based cast-level representation hc . Then, we can use ht and hc to score the attractiveness on movie m. When the parameters of attraction model are learned, we can compute personal attractiveness scores over a set of candidate moviesMC for recommendation and interpret the recommendation with the highlighted sentences of the story and which actors may attract the target user.

4 Multilevel AttractionModel

The overview of the architecture of our model is illustrated in Figure 1. This model consists of three main parts: cast attraction module (left), user module (middle) and story attraction module (right). In the story module, we build a multilevel attraction model to score attractiveness over words and sentences for each user. Similarly, we build another attraction model to score attractiveness over cast members in the cast module. Finally, we compute the personal attractiveness score of a movie on the top of these modules using the castlevel embedding and the story-level embedding.

Different from classical attention mechanism without considering subjective user personalization, we design a bottomup multilevel attraction model over the text of a movie story to learn personally attractive embeddings. At the word level, our model scores the attractiveness over words in a sentence, and then encodes a sentence embedding based on the word embedding vectors weighted by their attractiveness scores. Recursively, our model encodes the story embedding based on the sentence embedding vectors weighted by their attractiveness. In the following subsections, we give more technical details about these modules.

4.1 Multilevel Story AttractionModule Sentence Encoder with Word Attraction Filters

Given a set of words {w1, · · · , wNs } of a sentence s in story St of movie mu,i liked by user u, we aim to score the attractiveness over words from u's perspective. First, we map each word wi into a word embedding vector wi ∈ RL, where L denotes the length of the embedding vector. Then, we input these word embedding vectors {w1, · · · ,wNs } to the word attractiveness scoring module (illustrated in the overlapped part of user module and story attraction module). We use uwu ∈ RL to denote the word-level attraction filter of user u. Then, we compute the attractiveness score awu,i in terms of the inner product between uwu and each word embedding wi

The normalized weight can be computed by softmax(ai) = eai / ∑j eaj . However, awu,i in Eq. 1 can be arbitrarily large, which makes softmax to output a weight close to 1 on the maximum awu,i . Furthermore, a large ai easily makes the exponential function overflow in implementation. To resolve this problem, we impose an inverse squared root function isrα to bound the value of ai.

We can use the parameter α to control the upper bound and lower bound. A large α makes the upper bound and lower bound close to 0; as a result, the softmax tends to output uniform weights. A small α = 0.001 has the range (-31.6, 31.6) which guarantees the exponential function not overflow and the softmax tends to output a single large weight. In practice, α needs to be tuned with data. Accordingly, the normalized attractiveness score on word wi is

We find that α = 4 performs good through our experiments. Since ¯awu,i scores user attractiveness on each word, we create a personal attraction sentence embedding su by weighted sum over word embedding vectors using ¯awu,i .

su largely encodes the information from the most attractive words and discards the information from unattractive ones. To measure the overall attractiveness of the sentence s at the sentence level, we need to preserve the major information of a sentence apart from the most attractive part encoded by su. Therefore, we use another attraction-free filter gw to extract the major information over all words.

The filter weights are computed similar to user attractiveness (cf. Eq. 3), where the major difference is that the weight vector g and the bias bw are user independent and α of isr is set to 32 to keep relatively uniform information for all words. As a result, the attraction-free sentence embedding sg is encoded:

Then, we concatenate the attraction-encoded sentence embedding and the attraction-free sentence embedding, [su, sg], and input it into the tanh neural network layer to jointly encode su and sg into a comprehensive sentence encoding hsu with the parameters Ws and bs

Story Encoder with Sentence Attraction Filters

Once we obtain the sentence embeddings (cf. Eq. 7) for all sentences of the story St from the sentence encoder as presented above, we can build a story encoder over these sentence embedding vectors at the sentence level. As shown in the right-hand of Figure 1, the structure of story encoder is very similar to the sentence encoder. Therefore, we briefly introduce this story encoder in this subsection.

Given the sentence-level user attraction filter, usu , of user u, and the sentence embedding vectors {hsu,1 , · · · , hsu,Nt }, the attractiveness scores over usu and each hsu,i is given by

Accordingly, we can obtain the normalized attractiveness score ¯asu,i on each sentence embedding and the corresponding attraction-based story embedding tu:

where α = 2 is set throughh experiments. Apart from encoding the most attractive sentences, we use the attraction-free story embedding to preserve other information of the story for the follow-up movie-level attraction scoring.

Then, the comprehensive story embedding htu is encoded with the parameters Wt and bt :

4.2 Multilevel Cast AttractionModule

The architecture of cast attraction module is similar to the story attraction module. First, we map the cast members {c1, · · · , cNm } of the movie mu,i into embedding vectors {c1, · · · , cm}. Given the user attraction filter ucu , the attractiveness score over each cast members is:

Accordingly, the normalized attractiveness score ¯acu,i and the attraction-based cast embedding cu w.r.t. user u are given:

For the follow-up movie-level attraction scoring, we need to preserve the overall attraction-free cast information besides the attractive cast member embedding as done in the story attraction module.

Finally, we obtain the comprehensive cast embedding hcu , which is encoded with the parameters Wc and bc

Algorithm 1 The learning procedure for a mini-batch

• 1: B ←GetMinibatch({Mu}) from all user-movie pairs
• 2: N ←Sample contractivemoviesmu,j for eachmu,i ∈ B
• 3: Compute mini-batch loss using Eq. 21: LB ← 1 |B| ∑〈mu,i ,mu,j 〉∈〈B,N 〉 Lmu,i⪰mu,j
• 4: Update parameters: Θ←Θ-ΓAdam(∇ΘLB)

4.3 Optimization Objective and Training

Movie Attraction Scoring

After we obtain the comprehensive story embedding htu from story attraction module and the comprehensive cast embedding hcu from cast attraction module. We concatenate them as the joint multimodal movie embedding [htu , hcu ]. Then, the attraction scores on the movie m can be computed with the user's movie-level filter umu over [htu , hcu ]:

Ranking Loss

In real-world scenarios, explicit like/dislike data are often not available; instead, data like watch records and click logs are much more easily obtained. In such cases, we only have oneclass data [Hu et al., 2016; 2017a] which cannot be directly used to differentiate user preferences. Learning from oneclass preference data is often treated as a ranking problem [Rendle et al., 2009]. Given a user u, we can construct a contrastive pair to specify the attractiveness order, that is, we have the order mu,i ⪰ mu,j over a movie (mu,i ∈ Mu) explicitly selected by u and an unselected movie (mu,j /∈ Mu). Then, we use the following max-margin loss [LeCun et al., 2006] to optimize the ranking order over pairs:

where the parameter margin needs to be tuned over data.

Training Procedure

Our model is implemented using Keras [Chollet, 2015] with Tensorflow as the backend. We initialize the word embeddings with the pre-trained GloVe vectors [Pennington et al., 2014]. Due to the limited space, we only list a brief scheme of the learning procedure on a mini-batch in Algorithm 1, where ΓAdam(·) denotes Adam [Kingma and Ba, 2014] based gradient descent optimizer. The code for more detail is available at: https://github.com/rainmilk/ijcai18mlma.

5 Experiments

The experiments are conducted on the real-world movie watch dataset MovieLens 1M [Harper and Konstan, 2016]. We demonstrate our model from three aspects: (1) recommendation accuracy; (2) newmovie recommendation, and (3) interpretation of attraction on movies.

5.1 Data Preparation

We collect user watch records from the MovieLens 1M dataset. However, this dataset does not contain any story and cast data. Fortunately, researchers have provided good mapping from MovieLens ID to DBPedia URI [Noia et al., 2016].

We queried all available story abstract and cast data from DBPedia. The statistics of the data are reported in Table 1.

For testing the performance on released movie recommendation, we randomly held out 20% user watch records as the testing set, and the remainder were served as the training set. One of the most important tasks is to recommend newmovies without knowing any watch record. To simulate this case, we randomly selected 10% movies and held out all their watch records from the dataset, and the remainder of 90% movies and their watch records were used for training. For each holdout test sample in above two testing sets, we randomly draw ten noisy samples to test whether the testing methods can rank the true sample at a top position out of noisy samples.

5.2 ComparisonMethods and EvaluationMetrics

The following state-of-the-art content-based methods are compared for movie recommendation. CF methods cannot deal with rich contents and new movies as the study focus in this paper so they are not included for comparison.

• CENTROID: We create user profiles using the centroid [Musto et al., 2016] of all word embedding vectors from the users' movie stories. Then, we rank recommendations by the similarity between the user profile and the controid of word embedding vectors of movie story.
• CTR: Collaborative topic regression [Wang and Blei, 2011] performs user regression over the latent topic distribution of movie stories learned from LDA.
• CWER: Similar to CTR, we create the collaborative word embedding user regression (CWER) to perform regression over the centroid word embedding vector of each movie story initialized by GloVe embeddings.
• MLAM: This is the full multilevel attraction model proposed in this paper.
• MLAM-S: This is the single-modal version MLAM that only has the story attraction module.
• MLAM-C: This is the single-modal version MLAM that only has the cast attraction module.

To evaluate the recommendation quality, the following metrics are used:

• R@K: denotes the mean Recall from the top-K recommended items over all testing users.
• MAP@K: denotes the Mean Average Precision of the topK recommended items over all testing users.
• MRR@K: denotes the Mean Reciprocal Rank of the top-K recommended items over all testing users.

5.3 Recommendation Accuracy Evaluation

Recommendation for Released Movies

We evaluate the recommendation performance on released movies associated with users' watch records, that is, people have known the story and cast members in these movies. Table 2 reports the recommendation accuracy. CTR scores the user preference according to the topic distribution over a movie story. However, there are many uninformative words which may obscure the core topic distribution of the story. CENTORID and CWER are built on the story embedding derived from centroid of word embeddings. Since word embedding is an unnormalized vector, it allows large elements to specify the significance. As a result, CENTORID and CWER outperform CTR but they still suffer the obscure from uninformative words. MLAM-S leads CENTORID and CWER with a large margin, the MAP and the MRR are at least 200% higher than baselines. This highlights the design of our model, that is, we place two types of filters in MLAM-S, one is to extract the most attractive words and sentences and the other is to filter out noisy words and sentences. MLAM-C surprisingly performs well, this discloses the fact that the attractiveness of a movie is heavily related to the attractiveness of its cast. Thanks to the multimodal modules over story and cast to comprehensively capture users' attraction from different aspects, MLAM achieves the best performance out of all comparison methods.

Figure 3 depicts the recall of all comparison methods. We find that the plots of MLAM-C, MLAM-S and MLAM are above the plots of baselines with apparent margins, i.e., MLAM can more accurately retrieve the attractive movies for each user in top positions. MLAM combines the information from both modules, which leads to the best recall.

Recommendation for New Movies

We apply the above design to recommend attractive new movie, which cannot be handled by pure CF methods, to demonstrate the goal and value of attraction modeling. Content-based methods are more capable of tackling such cases widely seen in this new media age. The ranking performance is reported in Table 3 and the recall is illustrated in Figure 3 (b). CTR underperforms other methods for the reason analyzed in the above subsection. CENTROID and CTR achieve similar performance to the first experiment, which proves the effectiveness of content-based matching using word embedding for new movies even without any user watch record. Similarly, MLAM-S achieves comparable performance with the above case. However, MLAM-C is the special case. We find that the performance drops drastically when comparing with Table 2. In fact, the reason behind is quite straightforward. We can find most cast only appeared in two movies (cf. Table 1). Accordingly, users cannot tell whether they will be attracted by an unknown cast. Figure 4 shows two testing samples of a user. The left movie is associated with a high attractiveness score due to the known cast members (in red) in this user's training set, whereas the cast members in the right movie are absent from user's training set, which results in low attractiveness scores. As a result, MLAM-C tends to assign low rank on these movies. This also proves the factor that the attractiveness of a movie is heavily dependent on its cast members. Accordingly, the multimodal model MLAM slightly underperforms MLAM-S due to the ineffectiveness of MLAM-C.

5.4 Interpretation and Visualization

The most important value of attraction modeling is not only for recommendation but for obtaining insight into the underlying causes of user selection by disclosing the attractive points. In this experiment, we pick two case studies to visualize the user attractiveness scores output by MLAM. Figure 2 illustrates the statistical attractiveness, according to the weights (cf. Eqs. 3, 9 and 15), over the sentences of movie story, words in the most attractive sentence, and the cast members points for User 156 and User 2163. The results show that we can easily find the attraction difference between two users. User 156 is attracted by the first sentence which highlights the genre of this movie, i.e., comedy-drama, while User 2163 is attracted by the last sentence which highlights the award of this movie. Similarly, we find User 156 is attracted by the director Alexander Payne while User 2163 is attracted by the star Reese Witherspoon. Therefore, we can easily use MLAM to analyze user selection and tell the insight about the recommendation made.

6 Conclusion

In this paper, we propose a multilevel attraction model (MLAM) over multimodal contents to learn user attraction on movies. MLAM can provide the interpretation of user selection w.r.t. the attractive points. Moreover, it can conduct content-based recommendation for new movies. The results prove the effectiveness and merits of MLAM. Textual data, e.g. news, papers and categorical data, e.g. writers, authors, are the most common data in content-based RS, so our model can be directly adapted to these recommendations. For other content like music, image, we can apply CNN to learn their embeddings and then build attraction over them.

Acknowledgements

The paper is partially supported by the National Natural Science Foundation of China (61572325); Shanghai Key Programs of Science and Technology(16DZ1203603); Shanghai Engineering Research Center Project (GCZX14014 and C14001).