Disparity-preserved Deep Cross-platform Association for Cross-platform Video Recommendation

1 Introduction

Recommender systems are playing an important role in our life. With the emergence of various online services, people are now getting used to engaging on different platforms simultaneously in order to meet their increasing diverse information needs [Chen et al., 2012b]. The complementary information from various platforms jointly reflects user interests and preferences, providing us with a great opportunity to tackle the data sparsity problem and improve the recommendation accuracy through associating information across platforms. In order to improve user experience and increase user adoption, many online service providers release new features to encourage cross-platform data associations. For example, Google+ encourages users to share their homepages on other platforms, enabling different accounts of the same person to be linked together. To date, quite a number of various platforms tend to associate with each other, which provides chances for cross-platform information transfer and association analysis possible.

∗Corresponding Author

Existing cross-platform recommendation works try to improve their cross-platform performances mainly by investigating the ablity of cross-platform association. Yan et al. propose a user-centric topic association framework to map cross-platform data in a common latent space [2014]. They then introduce a predefined micro-level metric to adaptively weight data while doing the data integration [2015]. Man et al. propose an embedding and mapping framework, EMCDR, to represent and associate users across different platforms [2017]. However, existing works on cross-platform video recommendation ignore the inconsistencies in crossplatform association and differences in semantic granularities, two challenging phenomena discovered in this paper.

The first challenge, inconsistency in cross-platform association, results in a reconsideration for the current assumption that all cross-platform information of the same user is consistent and can be aligned, which will be empirically validated on a real-world cross-platform dataset later. Our discovery demonstrates that the inconsistency is mainly caused by the platform-specific disparity, i.e., in addition to a user's inherent personal preferences, its interests shown on different platforms also contain platform-specific factors due to different focuses of various platforms. The second challenge is that different platforms may have different semantic granularities. Take Twitter and YouTube as an example, tags on Twitter are mainly hot event topics, which are generally finer-grained than video categories on YouTube. As such, a good crossplatform video recommendation model based on the correlated information across platforms should have the ability to reveal characteristics at different semantic granularities.

To address these challenges, we propose the Disparitypreserved Deep Cross-platform Association (DCA) model for cross-platform recommendation, which employs a partiallyconnected multi-modal autoencoder to explicitly capture and preserve platform-specific disparities in latent representation. Besides, we also introduce nonlinear mapping function, which is superior to their linear competitors in terms of handling granularity difference. To validate the advantages of our proposed DCA model, we further design a cross-platform video recommendation approach based on the proposed DCA model, which is capable of automatically targeting and transferring useful cross-platform information from a comprehensive semantic level. The contributions of this paper can be summarized as follows:

• We recognize the inconsistency phenomenon, which is caused by platform-specific disparity, as well as the granularity difference problem in cross-platform association.
• We propose a novel Disparity-preserved Deep Crossplatform Association (DCA) model for cross-platform recommendation through employing a modified multimodal autoencoder, which is able to handle the crossplatform inconsistency issue by preserving the platformspecific disparities and solve the granularity difference problem via nonlinear functions.
• We present a cross-platform video recommendation approach based on the proposed DCA model to test its effectiveness compared with other approaches.
• We conduct experiments on a real-world cross-platform dataset on Twitter and YouTube to demonstrate the superiority of our proposed DCA model over several state-ofthe-art methods in terms of various evaluation metrics.

2 RelatedWork

Existing cross-platform recommendation works can be categorized in two ways.

Categorization by what to associate. One option is to take advantage of different platform characteristics towards collaborative applications. Particularly, Qian et al. propose a generic cross-domain collaborative learning framework based on nonparametric Bayesian dictionary learning model for cross-domain data analysis [2015]. Min et al. propose a cross-platform multi-modal topic model which is capable of differentiating topics and aligning modalities [2015]. An alternative option is to associate in a user-centric way, which focuses on integrating multiple sources of overlapped user's activities. In particaular, XPTrans [Jiang et al., 2016] optimally bridges different platforms through exploiting the information from a small number of overlapped crowds. Yan et al. propose an overlapped user-centric topic association framework based on latent attribute sparse coding, and prove that bridging information from different platforms in common latent space outperforms explicit matrix-oriented transfer [2014]. Man et al. propose an embedding and mapping framework, EMCDR, where user representations on different platforms are first learnt through matrix factorization and then mapped via multi-layer perceptron [2017]. Although the user-centric works above are based on different premises, they share the same core idea that all cross-platform information is consistent and should be aligned. However, several works have pointed out the data inconsistency phenomenon in cross-platform data association, and attempted to solve this

(b) User groups clustered on YouTube

Figure 1: Visualization of users on Twitter and YouTube in the CASIA-crossOSN subset. Orange and purple dots represent two groups of aligned users across platforms. Concentrated areas are circled with dashed lines.

problem by data selection. To be concrete, Lu et al. find that selecting consistent auxiliary data is important for crossdomain collaborative filtering [2013]. They propose a novel criterion to assesses the degree of consistency, and embed it into a boosting framework to selectively transfer knowledge. Yan et al. divide users into three groups and introduce a predefined micro-level user-specific metric to adaptively weight data while integrating information across platforms [2015]. Our proposed DCA model associates information from different platforms in the user-centric way.

Categorization by entire model structure. One group of methods [Chen et al., 2012a; Jiang et al., 2016; Jing et al., 2014; Qian et al., 2015] build the model in a unified framework, where the former two works adopt matrix factorization and the latter two employ the probabilistic models. Another group of works [Yan et al., 2014; Yan et al., 2015; Man et al., 2017] adopt a two-step framework. These works first map users from different platforms into corresponding latent spaces for representations, and then associate these representations. Our proposed model in this work utilizes the two-step structure in a topic-based way.

3 Measurement and Observation

To validate the existence of the inconsistency, we first visualize users in CASIA-crossOSN, a cross-platform dataset linking user accounts between YouTube and Twitter (whose detailed description will be given in Section 6), following three steps as follows. 1) For users who engage in both Twitter and YouTube, we extract their interest representations by Latent Dirichlet Allocation (LDA). 2) We adopt the interest representations as inputs to the visualization tool tSNE [Maaten and Hinton, 2008] so that users are represented as two-dimensional vectors and visualized as dots - the closer two dots are in distance, the more similar their corresponding users are in topic space. 3) We cluster users into different groups by their Twitter (resp. YouTube) interest representations, select two groups, and mark them with orange and purple respectively in Figure 1 such that users who share a same color are regarded as similar.

We observe from Figure 1 that only a part of users who are similar on Twitter (resp. YouTube) still stay similar on YouTube (resp. Twitter) (circled with dashed line) and the rest actually becomes quite different. Moreover, in Figure 1(b), orange user group clustered on YouTube splits into two subset on Twitter, indicating that two platforms may have different semantic granularities.

In addition to visualizing the interest representations, Table 1 further confirms the existence of the inconsistency phenomenon through a measurement study on CASIAcrossOSN. i) For each clustered user group on Twitter and YouTube, we calculate the average Euclidean distance from the group center to group members. ii) We do the same thing for randomly sampled groups on both platforms and use the calculated average Euclidean distances as normalizers to obtain the "concentration ratio" such that a larger concentration ratio indicates a less clustered pattern.

It can be observed from Table 1 that when we cluster users by their interest representations on Twitter (resp. YouTube), the concentration ratio is small on the origin platform while becomes quite large (close to 1) on the other platform, indicating that users having similar interests on one platform tend to behave quite similarly to randomly sampled users on the other - they no longer share similar interests.

We close this section by giving a conclusion on the existence of inconsistency in cross-platform association: users' interests on different platforms may be diverse and inconsistent.

4 DCA: Disparity-preserved Deep Cross-platform Association

In this section, we employ multi-modal autoencoder to present our disparity-preserved deep cross-platform association (DCA) model.

4.1 ProblemFormulation

We first introduce the problem of user-centric cross-platform association. Given a set of aligned users U, for each user u ∈ U, his or her representations on different platforms (uT on Twitter and uY on YouTube in the experiment) are known.

The goal is to find the association among these representations (direct mapping function or unified latent representation), so that cross-platform applications can be carried out (specifically inferring uY for video recommendation in the experiment).

4.2 Disparity-preserved Deep Cross-platform Association

Multi-modal autoencoder is an extension to basic autoencoder. It is trained to reconstruct multiple inputs from a unified hidden layer when given multiple modalities of the same entity, thus is able to discover correlations across modalities. It has been widely applied to various multi-modal tasks since being proposed [Ngiam et al., 2011; Zhang et al., 2014; Wang et al., 2015]. Hong et al. extend the concept of "multimodal" to "multi-source" [2015]. In this paper, we treat representations of the same user on different platforms as multiple modalities of his or her unified latent representation, and modify the structure of multi-modal autoencoder to conduct disparity-preserved cross-platform association. The detailed structure of multi-modal autoencoder is formulated as:

where i ∈ {T, Y } denotes different platforms (e.g., Twitter and YouTube). xi is the "multi-modal" input layer, specifically the representations ui of the user u on different platforms. h is the hidden layer, i.e., the derived unified user latent representation. ˆxi is the output layer, i.e., the reconstructions of inputs. Weight matrices W and bias units b serve as parameters of the multi-modal autoencoder, denoted as θ together. g(·) is the activation function. The loss function of multi-modal autoencoder is defined as follows:

where the first term is reconstruction error. The second term is regularizer to avoid overfitting. The third term is a sparsity constraint to encourage sparse latent representations. We minimize the loss function and train the parameters through backpropagation.

Multi-modal autoencoder takes representations of the same user on different platforms as inputs, mapping them to a unified hidden layer h and expecting accurate reconstructions. It learns mapping functions from representations on different platforms to the unified representation and then vice versa, which automatically completes the process of cross-platform association. Moreover, it is able to handle the granularity difference through introduction of nonlinearity.

We note that all multi-modal inputs should be given for the multi-modal autoencoder to work properly. However, in many cross-platform applications, we need to infer unknown data on other platforms, which is extremely difficult. One straightforward solution is to train several networks with a certain combination of several platforms as inputs and output the results for all platforms, setting every decoding weight equally. However, such an approach does not scale well as we will need an exponential number of models. Ngiam et al. propose to train bi-modal autoencoder using augmented but noisy data with additional examples that have only one single modality as inputs [2011]. In this paper, we augment data with average values of different platforms, and still expect the network to "reconstruct" real representations for all modalities. To make the model become more robust to absent inputs and being capable of inferring unknown data in crossplatform association task, we split the training data as follows: one-third of the training data includes user representations on Twitter and average user representation on YouTube, one-third consists of average user representation on Twitter and user representations on YouTube, the remaining one-third contains groundtruth representations.

As aforementioned, platform-specific disparity may cause inconsistencies in cross-platform association. We need to envisage it and avoid messing up the association. To achieve this goal, we divide the hidden layer into three parts and cut off certain links, through which the shared and platformspecific parts are explicitly captured and preserved. Specifically, the hidden layer is divided into h = [hT , hC , hY ], where hT and hY are Twitter and YouTube platform-specific parts and hC is the common part. As is shown in Figure 2, we assume that Twitter representations are derived from hT and hC without hY while YouTube representations are derived from hY and hC without hT . The fully-connected structure is modified through cutting off links between units: from xT to hY , from xY to hT , from hT to ˆxY and from hY to ˆxT . The model is able to automatically map representations of the same user on different platforms to a unified representation while preserving the information in platform-specific parts at the same time. It does not require cross-platform data to be thoroughly consistent by allowing the existences of inconsistencies. Thanks to the usage of shared common structure, cross-platform association becomes clearer and tighter.

In addition, most traditional linear methods associate representations on different platforms by utilizing a linear transfer matrix, which is formulated in Eq. (3) and Eq. (4). The linear combination limits the power in modeling complex relations across different semantic granularities. Our modified multi-modal autoencoder naturally introduces nonlinear activation function g(·), possessing great advantages in handling the challenging granularity differences over linear methods. We choose Sigmoid as the activation function in this paper and other activation functions such as ReLU or tanh can also be used.

The time complexity is O(MNk(n1 + n2)) for training and O(k(n1 + n2)) for testing. M is the number of training iterations, N is the number of users for training, k is the hidden layer dimension, n1 and n2 are the input dimensions of two platforms. Our proposed DCA model, whose structure is presented in Figure 2, automatically captures common information and preserves platform-specific disparities through its partially-connected structure, and meanwhile naturally adapts to granularity difference via nonlinear functions.

5 Cross-platformVideo Recommendation

In this section, we present the cross-platform video recommendation approach based on the proposed DCA model.

5.1 ProblemFormulation

We first formulate the cross-platform video recommendation problem. Given a set of aligned users U, for each user u ∈ U, his or her behaviors on social platform are observed (i.e., tweet text t in collection Tu on Twitter). The goal of crossplatform video recommendation is to recommend videos for those aligned users on video platform (i.e., YouTube) through making use of the cross-platform information.

5.2 DCA-based Cross-platformVideo Recommendation

As is shown in Figure 3, our proposed cross-platform video recommendation method includes preprocessing and matching stages in addition to the DCA model. The capability of DCA model in tackling the cross-platform inconsistency issue through explicitly capturing shared information and preserving platform disparities enables the cross-platform recommendation method to automatically concentrate on useful information and discard useless one. Moreover, nonlinear mapping functions in our proposed approach enhance the method's ability to conduct association across platforms at a deeper and more comprehensive semantic level.

In order to apply our proposed DCA model for the associations between Twitter and YouTube, we first obtain user representations by Latent Dirichlet Allocation (LDA) [Blei et al., 2003]. On Twitter, we regard each user's tweeting data together as a document in Latent Dirichlet Allocation (LDA). Then each user u ∈ U can be represented by a topic distribution uT . On YouTube, as we focus on the relevance between videos and users in the recommendation problem, we extract representations for both users and videos by two steps: (1) treat the surrounding textual information (title, category, tags and description) of each video together as a document and to get video representation v through online LDA [Hoffman et al., 2010], (2) take the average v of the interacted videos consumed by each user to get uY . We remark that v and uY are in the same YouTube topic space, which enables similarity matching for recommendation.

Then the representation pairs of the same user as well as the augmented examples are fed to the DCA model to learn cross-platform association. Because of the input augmenting, the trained model is able to predict user representations on YouTube when only information from Twitter is available. By setting the input from YouTube to its average value, the corresponding output can be used to predict user representations ˆuY .

We finally adopt Euclidean similarities between the representations of the predicted user and candidate videos as the properness measure. The recommendation results can be obtained according to the rank of candidate videos.

6 Experiments

In this section, we carry out extensive experiments on a real-world cross-platform dataset and compare our proposed method with several state-of-the-art algorithms to show the advantages of the proposed approach.

6.1 Dataset

CASIA-crossOSN 1 is a cross-network user dataset with account linkages between YouTube and Twitter, created by Institute of Automation, Chinese Academy of Sciences. It contains 11,687 aligned users across platforms and 2,280,129 YouTube videos. On YouTube, three kinds of user behaviors (i.e., uploading, favoring and adding to playlist) as well as videos' rich metadata are collected. On Twitter, only users' representations extracted from their tweets via standard topic modeling (LDA) process [Blei et al., 2003] are provided because of privacy concerns. We filter aligned users and YouTube videos by keeping users who interacted with at least 3 videos and videos which were consumed by at least 3 users. We finally obtain a subset containing 7,338 users and 34,144 videos, and randomly select 80% users for training, and the rest 20% for testing.

1 CASIA-crossOSN dataset: http://www.nlpr.ia.ac.cn/mmc/ homepage/myan/dataset.html

6.2 Experimental Settings

On Twitter, CASIA-crossOSN offers a 60-dimensional topical distribution for each user. On YouTube, we resort to perplexity to determine the number of topics. We select 80, which leads to the smallest perplexity on a 5% held-out YouTube video subset, in our experiments.

To evaluate the effectiveness of the proposed DCA model, we implement the following baselines:

Linear Regression-based Association (LR)

LR treats cross-platform association as a linear transfer problem, and pursues an explicit transfer matrix based on regression. The objective function is:

Latent Attribute-based Association (LA)

Instead of pursuing hard transfer, Yan et al. introduce another method by discovering the shared latent structure behind two topic spaces [2014], whose objective function is:

The problem can be efficiently solved by sparse coding algorithm [Lee et al., 2007] after a few transformations.

MLP-based Nonlinear Mapping (MLP)

Man et al. employ MLP to cope with the latent space matching problem [2017]. MLP is a flexible nonlinear transformation. The optimization problem can be formulated as:

To train the above neural network-based model and our DCA model, we adopt Adam optimizer to minimize the loss functions.

6.3 Evaluation of Cross-platformAssociation

As for the proposed model in this paper, multi-modal autoencoder with L1-norm but without disparity-preserved structure (MA) and the DCA model are examined.

We perform evaluations in two scenarios: (1) Infer YouTube from Twitter. Given users with their Twitter representation uT , to estimate their YouTube representation uY . (2) Infer Twitter from YouTube in turn. As is discussed before, by setting the unknown input to zeros, we can get the corresponding output as the predicted representation ˆu. In both scenarios, we utilize Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) as evaluation metric:

where i ∈ {T, Y } denotes platform, and Ki is the dimension of representation. ui and ˆui are the real and predicted user representation.

For model parameters, in LR model as Eq. 3, λ is selected by line search. In LA model as Eq. 4, λ and the number of latent attributes m are selected by two-dimensional grid search. In the DCA model as Eq. 2, parameters consist of λ, µ and m, which further divides into [mT ,mC ,mY ]. We first tune the MA model (equivalent to set mT = mY = 0,m = mC in the DCA model) by multi-dimensional grid search, and get an optimal setting for λ, µ and m. We then carefully fine-tune [mT ,mC ,mY ] by a combined line-search strategy. Specifically for all randomly initialized autoencoder model, we conduct 6 experiments and take the average result.

Performances are shown in Figure 4 . Several observations can be made: (1) LA and MLP achieve better result than LR, indicating that the shared latent structure and nonlinear mapping function both benefit the association. (2) MA outperforms LA and MLP through combining the two strategies. (3) Compared with MA, DCA further improves the performance, demonstrating the contribution of the disparitypreserved structure. We also notice that compared with nonlinear models (MLP, MA and DCA), the optimal parameters of the linear models vary greatly between two scenarios. We think it is the granularity differences that cause the parameter gaps. Non-linear mappings can naturally tackle this issue, and have more advantages over linear ones. In total, the proposed DCA model outperforms LA by reducing relatively 9.3% on MAEY , 8.6% on RMSEY , 7.2% on MAET and 7.3% on RMSET , and outperforms MLP by reducing relatively 7.5% on MAEY , 7.8% on RMSEY , 6.5% on MAET and 6.6% on RMSET . It greatly improves the performance by preserving the disparities and introducing nonlinearity, validating that envisaging the inconsistency caused by platform-specific disparity and granularity difference jointly contributes to better cross-platform association performance.

6.4 Evaluation of Cross-platformVideo Recommendation

Then we evaluate the proposed DCA-based cross-platform video recommendation method for new YouTube users, following the above "infer YouTube from Twitter" scenario.

For each test user u, we randomly select as many as groundtruth interacted videos as additional candidates. We perform a top-k recommendation task: to recommend the top k YouTube videos with the highest topic-based Euclidean similarity, and adopt top-k precision, recall and F-score as evaluation metrics [Herlocker et al., 2004]. The evaluation metrics are calculated by examining whether the recommended videos are included in u's groundtruth video set Vu. Final results are averaged over all test users.

As is shown in Figure 5 and Table 2, the proposed DCAbased cross-platform video recommendation method outperforms LA-based method by enhancing relatively 6.1% on F-score@10, and outperforms MLP-based method by enhancing relatively 5.3% on F-score@10. The proposed DCA-based method improves over existing cross-platform association-based recommendation approaches.

7 Conclusion

In this paper, we discover the existence of inconsistency in cross-platform recommendation. We propose the DCA model, which tackles the inconsistency issue, as well as the granularity difference problem. We further present a crossplatform video recommendation method based on the proposed DCA model. Extensive experiments demonstrate the superiority of the DCA model and the DCA-based crossplatform recommendation approach over several state-of-theart methods.

Acknowledgments

This research is supported by National Program on Key Basic Research Project No. 2015CB352300, National Natural Science Foundation of China Major Project No. U1611461, China Postdoctoral Science Foundation No. BX201700136 and Shenzhen Nanshan District Ling-Hang Team Grant under No.LHTD20170005.