Transfer Learning to Predict Missing Ratings via Heterogeneous User Feedbacks

1 Introduction

Data sparsity is a major challenge in collaborative filtering methods [Goldberg et al., 1992; Pan et al., 2010] used in recommender systems. Sparsity refers to the fact that some observed ratings, e.g. 5-star grades, in a user-item rating matrix are too few, such that overfitting can easily happen when we predict the missing values. However, we observe that, some auxiliary data of the form "like/dislike" may be more easily obtained; e.g. the favored/disfavored data in Moviepilot 1 , the love/ban data in Last.fm 2 and the "Want to see"/"Not Interested" data in Flixster 3 . It is more convenient for users to express such preferences instead of numerical ratings. The question we ask in this paper is: how do we take advantage of our knowledge in the form of binary ratings to alleviate the sparsity problem in numerical ratings when we build a prediction model?

1 http://www.moviepilot.de

To the best of our knowledge, no previous work answered this question of how to jointly model a target data of numerical ratings and an auxiliary data of like/dislike. There are some work on using both the numerical ratings and implicit data of "whether rated" [Koren, 2010; Liu et al., 2010] or "whether purchased" [Zhang and Nie, 2010] to help boost the prediction performance. Among the previous works, Koren ([Koren, 2010]) uses implicit data of "rated" as offsets in a factorization model, Liu et al. [Liu et al., 2010] adapt the collective matrix factorization (CMF) approach [Singh and Gordon, 2008] to integrate the implicit data of "rated", and Zhang et al. [Zhang and Nie, 2010] convert the implicit data of simulated purchases to a user-brand matrix as a user-side meta data representing brand loyalty and a user-item matrix of "purchased". However, none of these previous works consider using auxiliary data of both like and dislike in collaborative filtering in a transfer learning framework.

Most existing transfer learning methods in recommender systems consider auxiliary data from several perspectives, including user-side transfer [Cao et al., 2010; Ma et al., 2011; Vasuki et al., 2012], item-side transfer [Singh and Gordon, 2008], two-side transfer [Pan et al., 2010], or knowledgetransfer using related but not aligned data [Li et al., 2009a; 2009b]. In this paper, we consider the situation where the auxiliary data of like/dislike is such that users and items of the target rating matrix and the auxiliary like/dislike matrix are both aligned. This gives us more precise information on the mapping between auxiliary and target data, which can lead to higher performance. Under this framework, the following questions can be addressed.

1. What to transfer and how to transfer, as raised in [Pan and Yang, 2010], can be answered. Previous works that address this question include approaches that transfer the knowledge of latent features in an adaptive way [Pan et

2 http://www.last.fm

3 http://www.flixster.com

• al., 2010] or collective way [Singh and Gordon, 2008], transfer cluster-level rating patterns in an adaptive manner [Li et al., 2009a] or collective manner [Li et al., 2009b].
• How to model the data-dependent effect of numerical ratings and like/dislike when sharing the dataindependent knowledge? This question is important since clearly the auxiliary and target data may be with different distribution and semantic meaning.

In this paper, we propose a principled matrix-based transfer-learning framework referred as Transfer by Collective Factorization (TCF), which jointly factorizes the data matrices in three parts: a user-specific latent feature matrix, an item-specific latent feature matrix, and two data-dependent core matrices. Technically, our main contributions include:

1. We construct a shared latent space (what to transfer) via matrix tri-factorization in a collective way (to address the how to transfer question).
2. We model the data-dependent effect of like/dislike and numerical ratings by learning the core matrices of trifactorizations separately.
3. We introduce orthonormal constraints to the latent feature matrices in TCF to enforce the effect of noise reduction in singular value decomposition (SVD), and thus only transfer the most useful knowledge.

2 Transfer by Collective Factorization

2.1 ProblemDefinition

In the target data, we have a matrix R = [rui]n×m ∈ {1, 2, 3, 4, 5, ?}n×m with q observed ratings, where the question mark "?" denotes a missing value (unobserved value). Note, the observed rating values in R are not limited to 5-star grades, instead, they can be any real numbers. We use a mask matrix Y = [yui]n×m ∈ {0, 1}n×m to denote whether the entry (u, i) is observed (yui = 1) or not (yui = 0). Similarly, in the auxiliary data, we have a matrix ˜R = [˜ rui]n×m ∈ {0, 1, ?}n×m with ˜ q observations, where 1 denotes the observed 'like' value, and 0 denotes the observed

'dislike' value. The question mark is the missing value. Similar to the target data, we have a corresponding mask matrix ˜Y = [˜ yui]n×m ∈ {0, 1}n×m. Note that there is an one-one mapping between the users and items of R and ˜R. Our goal is to predict the missing values in R by transferring knowledge from ˜R. Note that the implicit data in [Koren, 2010; Liu et al., 2010; Zhang and Nie, 2010] is different in the form of {1, ?}, since implicit data corresponds to positive observations only.

2.2 Model Formulation

We assume that a user u's rating on item i in the target data, rui, is generated from the user-specific latent feature vector Uu· ∈ R1×du , item-specific latent feature vector Vi· ∈ R1×dv , and some data-dependent effect denoted as B ∈ Rdu×dv . Note that it's different from the PMF model [Salakhutdinov and Mnih, 2008], which only contains Uu· and Vi·. Our graphical model is shown in Figure 1, where Uu·, u = 1, . . . , n and Vi·, i = 1, . . . ,m are shared to bridge two data, while B, ˜B are designed to capture the datadependent information. We fix d = du = dv for notation simplicity in the sequel. We denote the tri-factorization in the target domain as F(R∼UBVT ) = ∑nu=1 ∑mi=1 yui[ 1 2 (rui - Uu·BV T i· )2 + αu 2 ||Uu·||2F + αv 2 ||Vi·||2F ] + β2 ||B||2F , where regularization terms ||Uu·||2F , ||Vi·||2F and ||B||2F are used to avoid overfitting. Similarly, in the auxiliary data, we have F( ˜R ∼ U ˜BVT ). To factorize R and ˜R collectively, we obtain the following optimization problem for TCF,

where λ > 0 is a tradeoff parameter to balance the target and auxiliary data and D is the range of the latent variables. D can be DR = {U ∈ Rn×d ,V ∈ Rm×d} or D⊥ = DR ∩ {UT U = I,VT V = I} to resemble the effect of noise reduction in SVD [Keshavan et al., 2010]. Thus we have two variants of TCF, CMTF (collective matrix trifactorization) for DR and CSVD (collective SVD) for D⊥. Although 2DSVD or Tucker2 [Ding and Ye, 2005] can factorize a sequence of full matrices, it does not achieve the goal of missing value prediction in sparse observation matrices, which is accomplished in our proposed system.

To solve the optimization problem in Eq.(1), we first collectively factorize two data matrices of R and ˜R to learn U and V, and then estimate B and ˜B separately. The knowledge of latent features U and V is transferred by collective factorization of the rating matrices R and ˜R, and for this reason, we call our approach Transfer by Collective Factorization.

2.3 Learning the TCF

Learning U and V in CMTF Given B and V, we have gradient on the latent feature vector Uu· of user u,

Input: R, ˜R, Y, ˜Y. Output: U, V, B, ˜B. Step 1. Scale ratings in R. Step 2. Initialize U,V. Step 3. Estimate B and ˜B. repeat repeat Step 4.1.1. Fix B and V, update U in CMTF or CSVD. Step 4.1.2. Fix B and U, update V in CMTF or CSVD. until Convergence Step 4.2. Fix U and V, update B and ˜B. until Convergence

Figure 2: The algorithm of Transfer by Collective Factorization (TCF).

λ˜ yui ˜ ruiVi· ˜BT ). Thus, we have an update rule similar to alternative least square (ALS) approach in [Bell and Koren, 2007],

Note that Bell et al. [Bell and Koren, 2007] consider bifactorization in a single matrix, which is different from our tri-factorization of two matrices. We can obtain the update rule for Vi· similarly.

LearningUandV inCSVD Since the constraintsD⊥ have similar effect of regularization, we remove the regularization terms in Eq.(1) and reach a simplified objective function g = 1 2 ||Y⊙(R-UBVT )||2F + λ2 || ˜Y ⊙ ( ˜R-U ˜BVT )||2F , where the variables U and V can be learned via gradient descent on the Grassmann manifold [Edelman et al., 1999; Buono and Politi, 2004],

where ∂g ∂U = (Y⊙(UBVT -R))VBT +λ( ˜Y⊙ (U ˜BVT - ˜R))V ˜BT and γ = -tr(tT1 t2)-λtr(˜ tT1 ˜ t2) tr(tT2 t2)+λtr(˜ tT2 ˜ t2) with t1 = Y ⊙ (R - UBVT ), ˜ t1 = ˜Y ⊙ ( ˜R - U ˜BVT ), and t2 = Y ⊙ (∇UBVT ), ˜ t2 = ˜Y ⊙ (∇U ˜BVT ). Note that [Buono and Politi, 2004; Keshavan et al., 2010] study a single-matrix factorization problem and adopt a different learning algorithm on the Grassmann manifold for searching γ. We can obtain the update rule for V similarly.

Learning B and ˜B Given U, V, we can estimate B and ˜B separately in each data, e.g. for the target data,

where the data-dependent parameter B can be estimated exactly in the same way as that of estimating w in a corresponding least square SVM problem, where w = vec(B) = [BT ·1 . . . BT ·d ]T ∈ Rd2×1 is a big vector concatenated from the columns of matrix B. The instances can be constructed as {(xui, rui)} with yui = 1, where xui = vec(U T u·Vi·) ∈

Rd2×1 . Hence, we obtain the following least-square SVM problem,

where X = [. . . xui . . .]T ∈ Rp×d2 (with yui = 1) is the data matrix, and r ∈ {1, 2, 3, 4, 5}p×1 is the corresponding observed ratings from R. Setting ∇w = -XT (r - Xw) + βw = 0, we have

Note that B or w can be considered as a linear compact operator [Abernethy et al., 2009] and solved efficiently using various existing off-the-shelf tools.

Finally, we can solve the optimization problem in Eq.(1) via alternatively estimating B, ˜B, U and V, all in closed forms. The complete algorithm is given in Figure 2. Note that we scale the target matrix R with rui = rui-1 4 , yui = 1, u = 1 . . . , n, i = 1 . . .m, to remove the value range difference of two data sources. We adopt random initialization for U, V in CMTF and SVD results of ˜R for that in CSVD.

Each of the above sub-steps of updating B, ˜B, U and V will monotonically decrease the objective function in Eq.(1), and hence ensure convergence to local minimum. The time complexity of TCF and other baseline methods (see Section 3) are: (a) AF: O(q), (b) PMF: O(Kqd2 +Kmax(n,m)d3 ), (c) CMF: O(Kmax(q, ˜ q)d2 + K max(n, m)d3 ), (d) TCF: O(K max(q, ˜ q)d3+Kd6), where K is the iteration number, q, ˜ q (q, ˜ q > n,m) is the number of non-zeno entries in the matrix R and ˜R, respectively, and d is the number of latent features. Note that TCF can be sped up via stochastic sampling or distributed computing.

3 Experimental Results

3.1 Data Sets and Evaluation Metric

We evaluate the proposed method using two movie rating data sets, Moviepilot and Netflix 4 , and compare to some state-ofthe-art baseline algorithms.

Moviepilot Data The Moviepilot rating data contains more than 4.5×106 ratings with values in [0, 100], which are given by more than 1.0×105 users on around 2.5×104 movies [Said et al., 2010]. The data set used in the experiments is constructed as follows,

1. we first randomly extract a 2, 000 × 2, 000 dense rating matrix R from the Moviepilot data, and then normalize the ratings by rui 25 +1, and the new rating range is [1, 5];
2. we randomly split R into training and test sets, TR, TE, with 50% ratings, respectively. TR, TE ⊂ {(u, i, rui) ∈ N × N × [1, 5]|1 ≤ u ≤ n, 1 ≤ i ≤ m}. TE is kept unchanged, while different (average) number of observed ratings for each user, 4, 8, 12, 16, are randomly sampled from TR for training, with different sparsity levels of 0.2%, 0.4%, 0.6% and 0.8% correspondingly;

4 http://www.netflix.com

1. we get the auxiliary data ˜R (sparsity 2%) from favoured/disfavoured records of users expressed on movies. The overlap between ˜R and R (∑i,j yij ˜ yij/n/m) is 0.035%, 0.070%, 0.10% and 0.14% correspondingly.

Netflix Data The Netflix rating data contains more than 108 ratings with values in {1, 2, 3, 4, 5}, which are given by more than 4.8 × 105 users on around 1.8 × 104 movies. The data set used in the experiments is constructed as follows,

1. we first randomly extract a 5, 000 × 5, 000 dense rating matrix R from the Netflix data;
2. we randomly split R into training and test sets, TR, TE, with 50% ratings, respectively. TE is kept unchanged, while different (average) number of observed ratings for each user, 10, 20, 30, 40, are randomly sampled from TR for training, with different sparsity levels of 0.2%, 0.4%, 0.6% and 0.8% correspondingly;
3. we randomly pick 100 observed ratings on average from TR for each user to construct the auxiliary data matrix ˜R. To simulate heterogenous auxiliary and target data, we adopt the pre-processing approach [Sindhwani et al., 2009] on ˜R, by relabeling 1, 2, 3 ratings in ˜R as 0 (dislike), and then 4, 5 ratings as 1 (like). The overlap between ˜R and R (∑i,j yij ˜ yij/n/m) is 0.035%, 0.071%, 0.11% and 0.14% correspondingly.

The final data sets are summarized in Table 1.

Evaluation Metric We adopt the evaluation metric of Mean Absolute Error (MAE),

where rui and ˆ rui are the true and predicted ratings, respectively, and |TE| is the number of test ratings. In all experiments, we run three random trials when generating the required number of observed ratings from TR, and averaged results are reported. The results on RMSE are similar.

3.2 Baselines and Parameter Settings

We compare our TCF method with two non-transfer learning methods: the average filling method (AF), PMF [Salakhutdinov and Mnih, 2008], as well as one transfer learning method: CMF [Singh and Gordon, 2008].

For the average filling (AF) method, we use the empirically best approach [Pan et al., 2010], ˆ rui = ¯ r + bu· + b·i, where ¯ r = ∑u,i yuirui/ ∑u,i yui is the global average rating, bu· = ∑i yui(rui - ¯ r·i)/ ∑i yui is the bias of user u, and b·i = ∑u yui(rui - ¯ ru·)/ ∑u yui is the bias of item i. For PMF, CMF and TCF, we fix the latent feature number d = 10. For PMF, different tradeoff parameters of αu = αv ∈ {0.01, 0.1, 1} are tried; for CMF, different tradeoff parameters αu = αv ∈ {0.01, 0.1, 1}, λ ∈ {0.01, 0.1, 1} are tried; for CMTF, β is fixed as 1, and different tradeoff parameters αu = αv ∈ {0.01, 0.1, 1}, λ ∈ {0.01, 0.1, 1} are tried; for CSVD, different tradeoff parameters λ ∈ {0.01, 0.1, 1} are tried.

To alleviate the data heterogeneity of {0, 1} and {1,2,3,4,5}-1 4 or [1,2,3,4,5]-1 4 , a logistic link function σ(Uu·V T i· ) was embedded in the auxiliary data matrix factorization of CMF, where σ(x) = 1 1+e-γ(x-0.5) , and different parameters γ ∈ {1, 10, 20} are tried.

3.3 Results

We randomly sample n ratings (one rating per user on average) from the training data R and use them as the validation set to determine the parameters and convergence condition for PMF, CMF and TCF. The results on test data (unavailable during training) are reported in Table 2. We can make the following observations:

1. TCF performs significantly better than all other baselines at all sparsity levels;
2. For the transfer learning method of CMF, we can see that it is significantly better than PMF at almost all sparsity levels (except the extremely sparse case of 0.2% on Moviepilot), but is still worse than AF, which can be explained by (1) the heterogeneity of the auxiliary binary rating data and target numerical rating data, and (2) the usefulness of smoothing (AF) for sparse data;
3. For the transfer learning methods of CMF and CMTF, we can see that CMTF performs better than CMF in all cases, which shows the advantages of modeling the datadependent effect in CMTF.
4. For the two variants of TCF, we can see that introducing orthonormal constraints (CSVD) improves the performance over CMTF in all cases, which shows the effect of noise reduction, and thus selectively transfer the most useful knowledge from the auxiliary data.

To further study the effectiveness of selective transfer via noise reduction in TCF, we compare the performance of CMTF and CSVD at different sparsity levels with different auxiliary data of sparsity 1%, 2% and 3% on Netflix. The results are shown in Figure 3. We can see that CSVD performs better than CMTF in all cases, which again shows the advantage of CSVD in transferring the most useful knowledge.

4 RelatedWorks

PMF Probabilistic matrix factorization (PMF) [Salakhutdinov and Mnih, 2008] is a recently proposed method for missing value prediction in a single matrix. The RSTE model [Ma et al., 2011] generalizes PMF and factorizes a single rating matrix with a regularization term from the user-side social data. The PLRM model [Zhang and Nie, 2010] generalizes PMF to incorporate numerical ratings, implicit purchasing data, meta data and social network information, but does not consider the explicit auxiliary data of both like and dislike. Mathematically, the PLRMmodel that only considers numerical ratings and implicit feedback can be considered as a special case of our TCF framework, CMTF for D = DR, but the learning algorithm is different (CMTF has closed-form solutions for all steps). CSVD (with D = D⊥) performs better than CMTF via selectively transferring the most useful knowledge.

CMF Collective matrix factorization (CMF) [Singh and Gordon, 2008] is proposed for jointly factorizing two matrices with the constraints of sharing one-side (user or item) latent features. However, in our problem setting as shown in Figure 1, both users and items are aligned. To alleviate the data heterogeneity in CMF, we embed a logistic link function in the auxiliary data matrix factorization in our experiments.

DPMF Dependent probabilistic matrix factorization (DPMF) [Adams et al., 2010] is a multi-task version of PMF based on Gaussian processes, which is proposed for incorporating homogeneous, but not heterogeneous, side information via sharing the inner covariance matrices of latent features.

CST Coordinate system transfer (CST) [Pan et al., 2010] is a recently proposed transfer learning method in collaborative filtering to transfer the coordinate system from two auxiliary CF matrices to a target one in an adaptive way.

Parallel to the PMF family of CMF and DPMF, there is a corresponding NMF [Lee and Seung, 2001] family with nonnegative constraints:

1. Tri-factorization method of WNMCTF [Yoo and Choi, 2009] is proposed to factorize three matrices of useritem, item-content and user-demographics, and
2. Codebook sharing methods of CBT [Li et al., 2009a] and RMGM [Li et al., 2009b] share cluster-level rating patterns of two rating matrices.

Models in the NMF family usually have better interpretability, while the top ranking models [Koren, 2010] in collaborative filtering are from the PMF family. We summarize the above related work in Table 3, in the perspective of whether having non-negative constraints on the latent variables, and what & how to transfer in transfer learning [Pan and Yang, 2010].

5 Conclusions and Future Work

In this paper, we presented a novel transfer learning framework, Transfer by Collective Factorization (TCF), to transfer knowledge from auxiliary data of explicit binary ratings (like and dislike), which alleviates the data sparsity problem in numerical ratings. Our method constructs the shared latent space U,V in a collective manner, captures the datadependent effect via learning core matrices B, ˜B separately, and selectively transfer the most useful knowledge via noise reduction by introducing orthonormal constraints. The novelty of our algorithm includes generalizing transfer learning methods in recommender systems in a principled way. Experimental results show that TCF performs significantly better than several state-of-the-art baseline algorithms at various sparsity levels.

In the future, we will extend the TCF framework to include more theoretical analysis and large-scale experiments.

Acknowledgement

We thank the support of Hong Kong RGC/NSFC N HKUST624/09 and Hong Kong RGC grant 621010.