DA-GCN: A Domain-aware Attentive Graph Convolution Network for Shared-account Cross-domain Sequential Recommendation

1 Introduction

Cross-domain Sequential Recommendation (CSR) is a recommendation task that aims at recommending the next item via leveraging a user's historical interactions from multiple domains. CSR is gaining immense research attention nowadays as users need to sign up for different platforms to access domain-specific services, e.g., music subscription and food delivery. In this work, we study CSR in an emerging yet challenging scenario, SCSR, where multiple individual users share a single account and their interaction behaviors are recorded in multiple domains [Lu et al., 2019]. We consider the shared-account scenario because it is increasingly common that people share accounts with others in many applications. For example, members of a family tend to share an account for watching movies (e.g., Netflix) and online shopping (e.g., Amazon). Consequently, generating accurate recommendations is much more challenging in this case because of the mixture of diverse users' interests within an interaction sequence [Yin et al., 2020; Yin et al., 2019; Guo et al., 2020]. Furthermore, though in a single-user scenario, data from two domains might collaboratively help uncover her/his preferences, the existence of shared accounts will instead amplify the noise in the interaction data and impede the sequential recommendation accuracy.

∗Corresponding author

Recently, several studies have been focused on recommendation with either shared-account or cross-domain settings, very few of them address SCSR that simultaneously considers both aspects. In prior work on shared accounts [Bajaj and Shekhar, 2016], a common approach is to capture the user relationships under the same account with latent representations, but none of them consider the cross-domain context, and are hence inapplicable to SCSR. Despite that cross-domain recommenders [Zhuang et al., 2018], partially fit this scenario, the cross-domain knowledge is implicitly transferred in the latent space, the explicit structural information bridging two domains are largely unexplored. Zhao et al. [2019] proposed a graph-based model as a solution, but their work ignores the important sequential information and relies on explicit user ratings that are not usually available in both domains. However, those cross-platform methods are not designed to cope with entangled user preferences on shared accounts. One prior work on SCSR is the π-net method [Ma et al., 2019], which formulates SCSR as a parallel sequential recommendation problem that is solved by an information-sharing network. Another related work is the PSJNet method [Ren et al., 2019], which improves the π-net using a split-join strategy. However, these RNN-based methods overwhelmingly target on discovering sequential dependencies, and have limited capability for capturing the complex relationships among associated entities (i.e., users and items) in both domains. As a result, this limits the expressiveness of learned user and item representations, and also overlooks the explicit structural information (e.g., item-useritem paths) linking two domains.

To address the above challenges, we propose a novel graphbased solution, namely Domain-Aware Graph Convolutional Network (DA-GCN), for SCSR. Specifically, to model the complicated interaction relationships, we first construct a Cross-Domain Sequence (CDS) graph to link different domains, where users and items in each domain are nodes and their associations are edges. Thereafter, to adapt to the scattered user preferences on shared accounts, we suppose there are H latent users under each account and leverage the message-passing strategy in a domain-aware graph convolution network to aggregate the information passed from directly linked neighbors. Then, node embeddings of users and items are learned by passing information from connected neighbor nodes. Though interactions with items are recorded at the account level, items in different domains will have different attraction to different users under the same account. In a similar vein, different neighbor users/items may have varied importance when learning the properties of an item. Hence, we further design two specialized attention mechanisms to discriminatively select relevant information during the message passing. Consequently, we are capable of modeling the multifaceted interactions as well as transferring fine-grained domain knowledge by considering the structure information.

The main contributions of this work are summarized as:

• We investigate an emerging yet challenging recommendation task, namely SCSR. After pointing out the defects of existing RNN-based solutions, we bring a new take on the SCSR problem with a graph-based approach.
• We firstly construct an innovative CDS graph that explicitly links accounts and items from two domains, and build a novel graph model named DA-GCN to attentively learn expressive representations for items and account-sharing users for recommendation.
• We conduct extensive experiments on two real-word datasets, and the experimental results demonstrate the superiority of DA-GCN compared with several state-ofthe-art baselines.

2 RelatedWork

2.1 Cross-domain Recommendation

As Cross-domain Recommendation (CR) concerns data from multiple domains, it has been proven useful in dealing with cold-start [Abel et al., 2013] and data sparsity issues [Pan et al., 2010]. Existing studies can be categorized into traditional methods and deep learning-based methods. In traditional methods, there are two main ways in dealing with CR. One is to aggregate knowledge between two domains [Hu et al., 2018]. Another is to transfer knowledge from the source domain to the target domain [Tang et al., 2012]. Deep learning-based methods are well suited to transfer learning, as they can learn high-level abstractions among different domains [Zhao et al., 2019; Liu et al., 2020]. For example, Zhao et al. [2019] and Liu et al. [2020] studied CR by leveraging the Graph Convolution Network (GCN)-based techniques, but their methods all rely on explicit user ratings, and can not be directly applied to SCSR. π-net [Ma et al., 2019] and PSJNet [Ren et al., 2019] are two recently proposed methods for SCSR, but their methods are all based on RNNs, which are neither expressive enough to capture the multiple associations nor can model the structure information that bridges two domains.

2.2 Shared-account Recommendation

The task of recommending items for a shared-account is to make recommendations by modeling the mixture of user behaviours, which usually performs user identification first, and then makes recommendations [Zhao et al., 2016; Jiang et al., 2018]. For example, Wang et al. [2014] are the first to study user identification in a novel perspective, in which they employ user preference and consumption time to indicate user's identity. Jiang et al. [2018] introduced an unsupervised framework to judge users within a same account and then learned the preferences for each of them. In their method, a heterogeneous graph is exploited to learn the relationship between items and the metadata. Ma et al. [2019] and Ren et al. [2019] argued that this task can be treated in an end-to-end fashion, and can also be improved by simultaneously considering the cross-domain information.

2.3 GCN-based Recommendation

The development of the graph neural networks [Wang et al., 2019b; Wang et al., 2019c] has attracted a lot of attention to exploring graph-based solutions for recommender systems [Guo et al., 2021; Wang et al., 2021]. E.g., PinSage [Ying et al., 2018] combines random walks with multiple graph convolutional layers on the item-item graph for Pinterest image recommendation. NGCF [Wang et al., 2019b] exploits high-order proximity by propagating embeddings on the user-item interaction graph to simultaneously update the representations for all users and items in an efficient way by implementing the matrix-form rule. Qiu et al. [2020] exploits the session-based recommendation problem via proposing a novel full graph neural network to the item dependencies between different sessions. However, none of them can be directly applied to SCSR as they either fail to capture the important sequential information [Wang et al., 2020] or focus on the recommendation problem in a single domain.

3 Methodologies

3.1 Preliminaries

Let SA = {A1, A2, . . . , Ai} and SB = {B1, B2, . . . , Bj} be the behaviour sequences from domain A and B respectively, where Ai ∈ A(1 ≤ i ≤ p) is the item index in domain A; A is the item set in domain A; Bj ∈ B(1 ≤ j ≤ q) is the item index in domain B; and B is the item set in domain B. U = {U1,U2, ...,Uk, ...,U|U|} is a subset of accounts, where Uk ∈ U(1 ≤ k ≤ n) denotes the account index and U represents the whole account set. Given SA and SB, the task of SCSR is to recommend the next item based on a user's historical behaviours. The recommendation probabilities for all candidate items in domain A and B are:

where P(Ai+1|SA, SB) is the probability of recommending the next item Ai+1 in domain A, given SA and SB. fA(SA, SB) is the learned function to estimate P(Ai+1|SA, SB). Similar definitions are applied to P (Bj+1|SB, SA) and fB(SB, SA).

3.2 An Overview of DA-GCN

DA-GCN tries to learn a graph-based recommendation framework that can model the multiple associations and the structure-aware domain knowledge via recursively aggregating features from local graph neighbors in different domains, so as to better capture the diverse user preferences in a sharedaccount. Fig. 1 presents an overview of DA-GCN. Specifically, to model the sophisticated associations among users and items, a Cross-Domain Sequence (CDS) graph is first constructed. Four types of associations are considered: a) user-item interactions in domain A; b) user-item interactions in domain B; c) sequential transitions among items in domain A; d) sequential transitions among items in domain B. Then, we subsume H latent users under the same account and leverage GCN to learn node representation for each latent user, which collects information from the connected neighbors in different domains. In our case, as multiple users interact with different items with a joint account, an interacted item is actually of interest to only some specific account users. Hence, a domain-aware attention mechanism is further proposed to weight the passed messages from different domains. Analogously, we also design another attention scheme on the item side during information aggregation. Thereafter, we generate the sequence-level embedding of each domain via concatenation and pooling operations, which facilitates prediction of the next item. Finally, to mutually maximize the knowledge, a joint training paradigm is leveraged.

3.3 The ProposedMethod

Motivated by Wang et al. [2019a] and Zhao et al. [2019], in this work we employ the message-passing strategy to learn the node representations from the CDS graph.

Representation Learning with Latent Users

Since in SCSR, an account is usually shared by multiple users, behaviors under the same account are usually generated by different users. Moreover, as users under the same account often have different interests, uniformly treating an account as a virtual user is inappropriate. Because the number and identity of all users under an account are both unknown, we assume there are H latent users (Uk,1, Uk,2, ...., Uk,h, ..., Uk,H) under each account (Uk), and the embedding of user Uk,h is denoted by eUk,h ∈ Rd , which is learned by accumulating all the information passed from connected items in domain A and B. Intuitively, by defining latent users under each account, we can learn diversified user representations from their cross-domain interaction sequences to encode multifaceted personalities and preferences.

Message Passing. Suppose Ai ∈ N Uk,h Ai and Bj ∈ N Uk,h Bj are the item neighbors of Uk,h in domain A and B respectively, where N Uk,h Ai is the item neighbor set of Uk,h in domain A, and N Uk,h Bj is the item neighbor set in domain B. Specifically, we define the message passed from Ai ∈ N Uk,h Ai to Uk,h as:

where mUk,h←Ai is the message representation; W1,W2 ∈ Rd′×d are the trainable weight matrices. The information interaction between Ai and Uk,h is represented by the elementwise product eAi ⊙ eUk,h , where eAi ∈ Rd and eUk,h are the embedding vectors of item Ai and user Uk,h respectively. γ Uk,h Ai is a learnable parameter that controls how much information can be passed from item Ai to user Uk,h. We will detail the learning process of this attentive weight in the subsequent section. Similar to domain A, the message passed fromBj to Uk,h in domain B can be defined as:

Besides, to retain the information carried by the target user, a self-connection is further added to her/him via:

where γ Uk,h Bj and γ Uk,h Uk,h are attentive weights guiding the strength of each message to be passed, which are also computed by an attention mechanism.

Domain-aware Attention Mechanism. On both domains, to fully capture each latent user's distinct preferences over different interacted items, we devise a domain-aware attention mechanism to compute the importance of different item nodes to the target user. The importance of Ai to Uk,h in domain A is defined as:

where f(·) is a pairwise similarity metric. In this work, cosine similarity is applied. Similarly, the importance of Bj to Uk,h in domain B can be defined as:

Then, the attentive weight on each item from domain A is obtained by the following normalization operation:

where sUk,h Uk,h weighs the self-connection within user Uk,h. Analogously, by replacing the corresponding user/item embeddings, γ Uk,h Bj and γ Uk,h Uk,h can be obtained in a similar way.

Message Aggregation. The embedding of user Uk,h (eUk,h ) is then updated via:

where all the messages passed from her/him-self and items in both domains are aggregated, followed by a LeakyReLU activation function. Eventually, we merge all latent users' representations into the account-level representation Uk:

User-specific Item Representation Learning

For an item, its representation is learned by aggregating the information from two types of nodes, i.e., the connected users and items within the same domain. As the item representations are learned in a similar way in both domains, we take domain A as an example, and the process is also applicable for domain B.

Message Passing. Let Ai-1 ∈ N Ai Ai-1 andUk,h ∈ N Ai Uk,h be the connected item and user nodes respectively, where N Ai Ai-1 is the item neighbor set of Ai in domain A and N Ai Uk,h is the user neighbor set of Ai. Then, the message passed from item Ai-1 to Ai is formulated as:

where the parameter γAi Ai-1 controls how much information can be passed from Ai-1 to Ai, as we will explain shortly. The message passed from user Uk,h to Ai is defined as:

Similarly, the retained message of Ai is formulated as:

where γAi Uk,h and γAi Ai are both learnable weights.

Sequence-aware Attention Mechanism. Due to varied relevance of linked users/items to the target item, we develop another attention method to assess the importance of the connected users and the items that have sequential relationships to the target item. The importance of item Ai-1 to Ai is formulated as:

The importance of user Uk,h on item Ai is defined as:

Then, we normalize sAi Ai-1 by the following operation:

where sAi Ai is the importance of the self-connection within itemAi. γAi Uk,h and γAi Ai are obtained in the same way.

Message Aggregation. The user-specific item representation is updated by aggregating all messages passed from the neighbor user/item nodes within domain A:

For better expression, we use ghAi = eAi to denote the embedding of item Ai w.r.t the h-th user. Then, the final representation of item Ai (gAi ) can be defined as:

Matrix-form Propagation Rule

To efficiently update the representations for all users and items, we formulate the propagation rule in a layer-wise matrix-form, which is defined as:

where I denotes an identity matrix; and El ∈ R(p+n+q)×d is the representation of all user and item nodes in domain A and B. L ∈ R(p+n+q)×(p+n+q) is the Laplacian matrix for the CDS graph, which is defined as:

where YAiAi-1 ∈ Rp×p and YAiUk ∈ Rp×n are the attention matrices carrying the weights from item and user neighbors to the target item in domain A, respectively; YUkAi ∈ Rn×p represents the weights from item neighbors in domain A to user nodes; YUkBj ∈ Rn×q denotes the weights from item neighbors in domain B to the user nodes; YBj Uk ∈ Rq×n and YBjBj-1 ∈ Rq×q represent the weights from user nodes and item nodes to the target item in domain B, respectively.

The Prediction Layer

Through the max pooling operation on the item embeddings within the sequence (gA1 , gA2 , ..., gAi ) (or (gB1 , gB2 , ..., gBj )), we can get the sequence-level embedding h′SA (or h′SB ) for SA (or SB). Then, we feed the concatenation of h′SA (or h′SB ) and eUk into the following prediction layer:

where WA, WB are the embedding matrices of all items in domain A and B, respectively; bA, bB are the bias terms. Then, we leverage a negative log-likelihood loss function to train DA-GCN in each domain:

where θ represents all the parameters of DA-GCN and S denotes the training sequences in both domain A and B. Both objectives are joined via a multi-task training scheme:

All the parameters in DA-GCN are learned via gradient backpropagation algorithm in an end-to-end fashion.

4 Experiments

4.1 Experimental Setup

ResearchQuestions. We intend to answer the following research questions:

• RQ1. How does our proposed DA-GCN method perform compared with other state-of-the-art methods?
• RQ2. Is it helpful to leverage the sequence information? Is it helpful to learn the node representations by incorporating the attention mechanism?

RQ3. Is it helpful to model the shared-account characteristic? How does the hyper-parameter H (the number of users) affect the performance of DA-GCN?

RQ4. How is the training efficiency and scalability of DA-GCN when processing large-scale data?

Datasets and Evaluation Protocols. We evaluate DA-GCN on two real-world datasets that are released by Ren et al. [2019], i.e., HVIDEO and HAMAZON. HVIDEO is a smart TV dataset that contains the watching logs from two platforms, i.e., the V-domain (the videos of TV series, movies, etc.) and E-domain (the educational videos based on textbooks and instructional videos on sports food, medical, etc.). HAMAZON is the data from two Amazon domains, that is, the movie watching and rating records (M-domain) and the book reading and rating behaviors (B-domain). We randomly select 75% of all the sequences as the training data, 15% as the validation set and the remaining 10% as the test set. For evaluation, we use the last watched item in each sequence for each domain as the ground truth and report the results measured by the commonly used metrics MRR@5, Recall@5, MRR@20 and Recall@20.

Baselines. We compare DA-GCN with the following baseline methods: 1) Traditional recommendations: POP [He et al., 2017], Item-KNN [Linden et al., 2003], and BPRMF [Hidasi et al., 2016]. 2) Shared-account recommendations: VUI-KNN [Wang et al., 2014]. 3) Cross-domain recommendations: NCF-MLP++ [He et al., 2017], and Conet [Hu et al., 2018]. 4) Sequential recommendations: GRU4REC [Hidasi et al., 2016], HGRU4REC [Quadrana et al., 2017]. 5) Shared-account cross-domain sequential recommendations: π-net [Ma et al., 2019], and PSJNet [Ren et al., 2019].

Implementation Details. We implement DA-GCN using the Tensorflow framework accelerated by NVidia RTX 2080 Ti GPU. For the initialization strategy, we randomly initialize the model parameters by the Xavier method [Glorot and Bengio, 2010]. We take Adam as our optimizing algorithm and also apply gradient clipping with range [-5,5] during the training period. For hyperparameters, the learning rate is set to 0.001. The embedding size is set to 100 on HVIDEO and 200 on HAMAZON. To speed up the training process, the batch size are both set to 128. The latent user number H is searched in [1-5] with a step size of 1. All these parameters are tuned on the validation set.

4.2 Experimental Results (RQ1)

Table 1 shows the comparison results of DA-GCN over other baselines on HVIDEO and HAMAZON. We have the following observations: 1) Our DA-GCN method achieves the best performance on both domains of HVIDEO and outperforms other baselines on HAMAZON in most metrics, which demonstrates the capability of DA-GCN in modeling the multiple associations and the user-specific representations, as well as the structure information of the domain knowledge. 2) The methods developed for SCSR are significantly better than other baselines, demonstrating the importance of considering the shared-account and cross-domain characteristics simultaneously. 3) DA-GCN outperforms the shared account and cross-domain baselines (i.e., VUI-KNN, NCF-MLP++ and Conet) indicating the importance of the sequential information and the effectiveness of our method in modeling users' sequential patterns. 4) From Table 1, we also observe that our GCN-based solution performs better than RNN-based methods (π-net and PSJNet). This result demonstrates the capability of our GCN-based method in modeling the sequential information and the complicated interaction relationships.

4.3 Ablation Studies of DA-GCN (RQ2)

To further illustrate the impact of different components to DA-GCN, we conduct ablation studies on both datasets. Due to space limitation, only results on HVIDEO are presented and similar results are obtained on HAMAZON. The experimental results are shown in Table 2, where GCNOS is the method that disables the sequential information when constructing the CDS graph. GCNOA is the method that disables the attention mechanisms when aggregating the passed messages. GCNOSA is a variant of DA-GCN that removes both components. From Table 2 we can find that: 1) DA-GCN outperforms GCNOSA and GCNOS demonstrating the importance of leveraging the sequential information in modeling users' preferences. 2) DA-GCN outperforms GCNOA indicating the importance of weighting the passed messages differently and our model can learn a better node representation via the proposed attention mechanisms.

4.4 Impact of H andModel Training Efficiency

Impact of H (RQ3). To explore the importance of modeling the shared-account characteristic, we further conduct experiments on HVIDEO to show the impact of H. The experimental results shown in Fig. 2 demonstrate that viewing an account as a virtual user can not get better results than modeling it as H latent users, which is more in line with reality.

Training Efficiency (RQ4). To investigate the model training efficiency and scalability of our graph-based solution, we further conduct experiments via measuring the time cost for the model training with varying the training ratio of the HVIDEO data in [0.1, 1.0] with step size 0.1. From the experimental results shown in Fig. 2, we can find that DA-GCN has lower training time cost than π-net and PSJNet, demonstrating its better scalability to large scale datasets.

4.5 Conclusions

In this work, we propose DA-GCN for SCSR to fully learn user and item representations from their interactions, as well as the explicit structural information. Specifically, to model the multiple associations among users and items, we first link them in a CDS graph. To model the structure information of the transferred knowledge, we then develop a domain-ware GCN to learn user-specific node representations, where two attention mechanisms are devised to weight the local neighbors of the target. The experimental results on two real-world datasets demonstrate the superiority of our graph-based solution.

Acknowledgments

This work was supported by National Natural Science Foundation of China (Nos. 61602282, 61802236), ARC Discovery Project (No. DP190101985) and ARC Training Centre for Information Resilience (No. IC200100022).