JUMP: a Jointly Predictor for User Click and Dwell Time

1 Introduction

Session-based Prediction (SBP) is an emerging problem in recent recommendation system research. For services like ecommerce, media streaming, or classified site, previous user behavior sequence is usually available. In such circumstance, a session-based predictor is adopted to forecast user's next behavior, either the next user click or the dwell time.

SBP tasks can be solved by classical recommendation algorithms like item-to-item methods and matrix factorization based ones [Hidasi and Tikk, 2016; Linden et al., 2003; Musto et al., 2015]. Despite of their efficiency, such methods fail to exploit the session's sequential nature, which precludes their use in practical problems. Alternatively, many researchers resort to Markov Decision Processes (MDP) based technique [Shani et al., 2005; Tavakol and Brefeld, 2014] for its capability in capturing the ordering information. A major issue for applying MDP is that the state space becomes unmanageable quickly when trying to include all possibilities.

Recently, deep learning methods have brought a tremendous potential to the area of recommender system. Of these methods, Recurrent Neural Networks (RNN), especially, Gated Recurrent Unit (GRU), have emerged as powerful methods for modeling sequential data in SBP [Hidasi et al., 2015; Dallmann et al., 2017; Hidasi and Karatzoglou, 2017; Zhang et al., 2014; Wu et al., 2016a]. Essentially, most of these methods can be formulated as learning to rank problems, which use ranking losses to achieve optimal choices.

∗Corresponding author

Although remarkable progress has been made for SBP with RNN methods, current SBP techniques as a productivity lever are still challenging. (a) First, the prediction of dwell time is not taken into consideration. Existing studies have proved that dwell time is an important metric to measure user engagement on the content and should be used as a proxy to user satisfaction [Yi et al., 2014; Yin et al., 2013]. On the other hand, sophisticated neural networks as RNNs are prone to overfit due to the extreme sparsity of user clicks. It is illustrated that by coercing the RNN with an extra task, the risk of overfitting will plummet [Baxter, 1997; Ruder, 2017]. In SBP, the dwell time prediction is a natural choice. (b) Second, the input sequence may include many noisy or irrelevant parts, especially when the session is long. Most RNN based SBP solvers treat the information in the whole sequence equally, thus the noisy or irrelevant parts may grievously degrade the recommendation performance. (c) Third, existing RNN based SBP solvers are sub-optimal when the input sequence is too short. Many researchers observed that very short history interactions significantly deteriorate the GRU's performance [Quadrana et al., 2017; Ruocco et al., 2017]. Since short sessions comprise a significant portion of the practical situations, improving recommendation results for short sessions can be highly propitious.

To bridge these gaps, we propose a joint predictor, JUMP 1 , for the session-based settings. The main contributions are summarized as follows.

1 JUMP is for Joint User-click and dwell tiMe Predictor.

• Dwell time is modeled by survival analysis, which has sound underlying mathematical principles. Based on this, a joint predictor is designed for both user click and dwell time.
• A novel three-layered RNN structure is constructed to map the session input to feature vector. Instead of forcing it to encode all information into one fixed-length vector, we use an attention layer to allow the network to refer back to the input sequence, which improves the robustness against noise.
• We also use redesigned fast-slow cells to enhance the performance for very short sessions. In the meanwhile, Recurrent Highway sub-cells are embedded in the fastslow cell to keep gradient flow stable.

In the experiments, we evaluate the performance of JUMP by recommendation tasks. Empirical results show that the proposed model has better performance in prediction of both user click and dwell time than the state-of-the-arts.

2 RelateWork

Recurrent Neuron Network The ordering nature of SBP makes it suitable to be formulated as a sequential learning problem. Highly successful in natural language modeling[Lin et al., 2017; Sukhbaatar et al., 2015], RNNs have become the most popular tool to approach such tasks. Traditional RNN architectures suffer from vanishing and exploding gradients [Bengio et al., 1994], which renders the optimization difficult and prohibits RNNs from learning long dependency tasks. For a long time, LSTM has been the de facto way to address the gradient vanishing/exploding problem [Hochreiter and Schmidhuber, 1997]. Later GRU has been proposed as a simplification of LSTM using fewer gates, but with competitive performance [Chung et al., 2014]. Recently, a new architecture named Recurrent Highway (RH) has been proposed [Zilly et al., 2016]. Suarez suggests that RH has better gradient flow than both LSTM and GRU [Suarez, 2017]. Besides, the attention mechanism [Bahdanau et al., 2014] has become an inseparable part of modern RNN structures, which significantly boosts the performance of RNNs, especially for long sequences. Additionally, a fast-slow RNN framework has shown that interconnecting different RNN cells and making them evolve at distinct speed are helpful to learn complex transition functions [Mujika et al., 2017].

Dwell Time in Other Domain Dwell time has been modeled in many domains for a better understanding of user actions. In social media analysis, Yin et al. convert dwell time of items to pseudo ratings for prediction of user votes [Yin et al., 2013]. In personalized recommendation, Yi et al. directly model normalized dwell time to help estimate the user satisfaction [Yi et al., 2014]. In web searching, Liu et al. model the probability of user satisfaction on one click as a product of the click's relevance and the dwell time's information gain [Liu et al., 2017]. However, in the domain of SBP, dwell time is only used to augment the feature vector for user click prediction [Dallmann et al., 2017]. None of the existing methods has jointly predicted user click and dwell time to enhance the performance of recommendation.

3 Dwell time Analysis

3.1 Preliminaries on Survival Analysis

We use capital letters like X, Y,Z . . . to denote a random variable and their instances are denoted by lower cases x, y, z . . .. Survival analysis aims to model the survival time which is the latency until the occurrence of a certain event. Conceptually, dwell time can be viewed as the survival time of leaving the item page and hence are often modeled by survival analysis.

We review some basic concepts and methods in survival analysis [Wang et al., 2017]. Let O being a random variable representing survival time. One way to model O is to suppose the existence of some invertible function g(·) such that

where f is some simple distribution, e.g. Gaussian and Gamma. The parameters of g(O) can be estimated by maximum likelihood estimation if samples of O is available. However, sampling from an unknown survival model by waiting for the occurrence of the event is unpractical, as the event may not happen after a long observation period. Thus in practice, one predefines a maximum observation duration c and records a tuple (τ, δ) with δ being a boolean value (that takes 0 when the event occurs) and τ being a sample of random variable T

According to [Li et al., 2016], the distribution of g(T) is

where F is the Cumulated Density Function (CDF) of f .

3.2 Estimation of Dwell Time

Most e-commerce systems record user clicks along with click timestamps. Let ik and ik+1 be two consecutively clicked items of a session with click time stamps tk and tk+1. Roughly, the dwell time on ik is approximated by ∆k = tk+1 - tk. Such approximation has two drawbacks. Firstly, for the last click of each session, its dwell time is not defined. Secondly, ∆k may greatly over-estimates the dwell time because a user may be distracted by other activities when viewing the item page. Indeed, in Recsys15 datasets, ∆k can be as large as 3 hours, which is rarely possible to be the actual dwell time. Moreover, the last dwell time ∆l cannot be computed from the timestamps as tl+1 does not exist. To obtain a more realistic dwell time estimation, we make the following assumptions:

• Users do not lose their attention in c1 seconds after clicking an item.
• Dwell time of the last click in each session is greater than c2 seconds.

Here c1 and c2 are hyper-parameters. By such assumptions, we obtain a censored dwell time estimation via a tuple (τk, δk) where δk = 1{ik is the last click or ∆k ≥ c1} and

We refer τk as the censored dwell time.

3.3 ProbabilityModeling of Dwell Time

Let D be a random variable representing the dwell time. To motivate our choice in modeling D, we give the histogram of the dwell time of RecSys15 in Fig. 2. This figure shows that the possibility of staying in a single item page decays exponentially fast over the dwell time, and therefore directly modeling D using Gaussian distribution is inappropriate. Surprisingly, the histogram of log dwell time is nearly symmetric around its mode as Fig. 2 shows. Thus, we assume logD has Gaussian distribution, i.e.

Suppose T is the censored dwell time. According to (3), its density can be formulated as

with Ψ(·) the CDF of the standard normal distribution. Note that the density of logT is not Gaussian.

4 Methodology

Formally, we consider the current session of the form sk = {(ij, log τj, δj)}kj=1 (1 ≤ k ≤ l), where ij is the j-th clicked item, τj is the censored dwell time defined in (4), and δj is a boolean value indicating whether τj is censored or not. Assuming that, the session is sampled from the density P(S) which is factorized as follows.

where Sk-1 is the historical clicks and dwell time sequence before the k-th click. P(Ik|Sk-1) and P(logTk|Sk-1, Ik) are the conditional probability distributions of the next user click and the dwell time respectively.

The probability distribution of the next user click is assumed to be a soft-max distribution

where v(Sk-1) is the feature vector of Sk-1 and v(Ik) = V(Ik, :) is the embedding vector of item Ik with V being an embedding matrix and V(Ik, :) being its Ik-th row. According to (6), the conditional density of log-dwell time is as follows.

where µ(Ik) (v(Sk-1)) is the mean of logTk. We define µ(Ik) (v(Sk-1) as the following linear model 2

in which W, b are parameters of the model. Now the remaining problem is how to extract the feature factor v(Sk-1) from a session Sk-1.

4.1 Extract Session's Feature Vector with RNN

In this section, we will present the RNN that maps a session Sk(1 ≤ k ≤ l) into a latent vector space. The proposed neural architecture has three layers. From top to bottom, they are referred as the attention layer, the fast-slow layer, and the embedding layer (see Fig. 3).

Attention Layer The attention layer outputs a session's feature vector via performing linear combination

2 Naturally, the function µ(Ik) (v(Sk-1) can also be a DNN. However, we found that using a simple linear model is good enough in predicting the dwell time.

where hi's are the outputs of the previous fast-slow layer, and the averaging weight α(i, k) is called the attention signal. The attention signal is formulated as

where d is the dimension of the hidden state hi. Note that ∑ki=1 α(i, k) = 1, and hence the extracted feature vector vk is equal to a weighted average of the outputs of the fast-slow layer.

Fast-Slow Layer The fast-slow layer is a novel recurrent network. When processing the j-th input (ij, log τj, δj), it uses ⌈τk/s⌉ sequentially connected fast cells on the lower hierarchical layer and one slow cell on the top hierarchical layer to model the transition function, as depicted in Fig. 4. The update rule of fast-slow RNN is given as follows.

where F(·, ·) and S(·, ·) are fast unit and slow unit respectively, z is the scale parameter and x(j,τj) is the output of the embedding layer. The output of the j-th step of the fast-slow RNN is hF⌈τj/z⌉ . Note that the slow cells comprise a sequence which has much longer length than the session. And hence, for a long session, the gradient vanishing/exploding problem would be more severe than GRU.

To attack the above problem, we proposed to use a Recurrent Highway (RH) unit for both slow and fast unit since it has been showed that RH units have better gradient flow than both LSTM and GRU units [Zilly et al., 2016]. Moreover, for capturing the more non-linear relationship between two steps, we use multiplicative information integration [Wu et al., 2016b] which can be used to improve the capacity of RNN cell without increasing model complexity.

Specifically, assuming that h is the previous state and x is the current input, the proposed RH unit updates the state via the following equations:

where ⊙ is the point-wise product, α is multiplicative gate unit, β, γ are linear gate unit, and h′ is the output of RH unit. For units without input, all the terms containing x will be omitted.

Embedding Layer The bottom layer is the embedding layer which maps a tuple (ik, log τk) to a vector. We jointly embed dwell time since it have been found that such approach is is beneficial for enhancing the prediction accuracy [Dallmann et al., 2017]. Specifically, the embedding is done by batchnormalizing

where V is the item embedding matrix and T is the time embedding matrix. Note that, in the considered datasets, the dwell time of users are less than 6000 seconds. Thus, the row number of the embedding matrix T is less than 6000.

4.2 Composition of Likelihood

Given dataset of N sessions D = {s(n) }Nn=1 with each session being sequences s(n) = {(i(n) k , log τ (n) k , δ(n) k )}l(n) k=1 , the log-likelihood of the dataset is

where θ is the parameters of JUMP and the conditional densities P(i(n) k |s(n) k-1 ) and P(log τ (n) k |i(n) k , s(n) k-1 )) are defined in (9) and (10). The parameter θ can be trained by maximizing the log-likelihood.

5 Experiments

To valid the performance of our model, we perform the task of user click prediction and dwell time estimation. Stateof-the-art session-based recommendation methods, which include GRU [Hidasi et al., 2015], IGRU [Hidasi and Karatzoglou, 2017], NARM [Li et al., 2017], DTGRU [Li et al., 2016], are compared. We also compare ours with neural temporal process models including RMTP [Du et al., 2016], ATRP [Xiao et al., 2017], NSR [Jing and Smola, 2017]. All the compared methods are performed on the same PC with i7-7820HK CPU, 16GB RAM, and GTX1080 GPU. All the compared methods are optimized by Adam with the batch size set to 100.

5.1 Datasets

We conduct experiments on three publicly available datasets including RecSys15, CIKM16, and REDDIT. RecSys15 is used for RecSys Challenge 2015. It contains click streams with timestamps collected from a commerce site. CIKM16 is from CIKMCup 2016. It contains sequences of anonymous transactions provide by DIGINETICA. REDDIT contains a log of user interactions on different subreddits (sub-forums) at different timestamps. The original REDDIT dataset does not organize user interactions into sessions. To re-organize the dataset, we group the consecutive actions that happened within the time of one hour into a session. For all the datasets, items or subreddits that appear less than 3 times are discarded, and the sessions with length equal to one are filtered out. After such process, we split all sessions of the datasets into 80% for training and 20% for testing. Note that each session is assigned to either training or testing. We list the basic statistics of the datasets in Tab. 1.

5.2 Experiments on ItemPrediction

To compare the top-k recommendation performance of our method with the baselines, we evaluate recall@20, MRR@20 and NDCG@20 of these methods on RECSYS15, CIKM16, and REDDIT datasets. For all the methods in our comparison, the dimension of item embedding vectors is set to 100 to make the number of their free parameters to be the same order. Besides, the dimension of the hidden states in the recurrent units are chosen to be the same with the word dimension for simplicity. For our model, we set σ to 10, c1 = 2000 and c2 = 30. c1 and c2 is selected from a candidate set 20, 30, 200, 1000, 2000, 20000 which lead to best prediction accuracy on randomly sampled 10% data from Recsys15. The results are showed in Tab. 3. From Tab. 3 we can see that JUMP outperforms all the baselines on all the datasets. This can be explained by that our model integrates the dwell time more effectively than the baselines and that our model uses a more refined RNN structure.

To verify the performance of these methods under different session lengths, we evaluate recall@20 for short sessions (length from 2 to 6), moderate sessions (length from 10-30), and long sessions (length greater than 150). We report the results in Fig. 7. From the figure, we can see that all the methods perform nearly equally well at modest length sessions while for short and long sessions, the performance of all the methods deteriorate to some extent. For short sessions, our method outperforms NARM and IGRU significantly, and this is because our model takes dwell time information in neural network designation. For long sessions, the performance of JUMP and NARM are similar, and both outperform IGRU, this is because JUMP and NARM use the attention mechanism to stabilize the performance of long step RNN. To check the performance of JUMP when different state dimensions are used ( the item dimension is set to the same value as the state dimension), we compare it with NARM and IGRU under different dimension settings in Fig.6. Since dimension d controls the expressive power of the models, it has a huge impact on the performance. The results show that our method has better click prediction accuracy than NARM and IGRU under all the dimension settings.

5.3 Experiments on Dwell Time Prediction

We validate the accuracy of dwell time modeling of JUMP via comparing the performance with neural temporal process models including RMTP, ATRP, and NSR. The accuracy of the time prediction is measured by Mean Absolute Error (MAE). The time prediction accuracy results are reported in Tab. 2. We can see that our model has much smaller MAE than all the other models. We attribute the performance gain to the ability of our model to utilize user clicks of the SBP datasets. The hyperparameter σ controls the trade-off between the accuracy in predicting dwell time and the accuracy of click forecasting. To depict this, we vary 1/σ2 from 10-4 to 100 and report the prediction accuracies of next click and dwell time in Fig. 5. From the figure, we can see that, when 1/σ2 is relatively small, both the accuracy of click prediction and time prediction grows as 1/σ2 increases. In this phase, dwell time prediction task and click prediction task cooperate with each other to find a better representation for the sessions. When 1/σ2 grows relatively large, time prediction accuracy still grows while click prediction accuracy decays from the peak. In this phase, the model treats the time prediction as a more important task and compromises the click prediction accuracy for the time prediction performance.

6 Conclusion

We explore using survival analysis and sequential modeling techniques to jointly predict user clicks and dwell time for the session-based recommendation. We propose a new predictor, JUMP. JUMP contains a new three-layered neuron network to extract a feature vector from input session. The network utilizes attention mechanism to improve the robustness against noise. And the fast-slow RNN structure is embedded to boost performance for very short sessions. Empirical results validate the performance of the proposed method.

Acknowledgments

This work is supported by National Natural Science Foundation of China (Grant No: 61472347, 61672376, 61751209), and Zhejiang Provincial Natural Science Foundation of China under Grant No. LZ18F020002.