User Retention: A Causal Approach with Triple Task Modeling

1 Introduction

In the information revolution era, the Internet industry has become highly competitive. The scale of active users is the foundation of a company's business success and the retention ratio of users is one of the most important indicators. Therefore it's of great significance to design personalized strategies aimed at improving user retention.

Improving retention ratio not only involves reducing customer churn rate but also enhancing customer activities. Previous researches on reducing churn rate focus on predicting which users are more likely to churn [Sabbeh, 2018]. Machine learning has recently emerged as an effective way for churn prediction, including ensemble-based methods [Idris and Khan, 2014], neural networks based methods [Kasiran et al., 2014], and data certainty based methods [Amin et al., 2019]. Customer activity promotion based approaches are often incorporated into recommender systems. For example, news/video websites promote user interactions by personalized recommendation, which focuses on optimizing CTR via deep neural networks [Covington et al., 2016; Wang et al., 2017].

∗Both authors contributed equally.

†Work done while interning at Ant Group.

Despite the rich studies in user retention, there is still no specific solution targeting both reducing user churn rate and enhancing user activities. We model user retention in the framework of recommender systems and focus on recommending the most suitable services or boons. The objective is to maximize the probability of explicit interaction (e.g., click) and revisit. For easier understanding, in the rest part we will take the click behavior as an example to represent explicit interactions. Thus, the objective is to "guide" users to behave like this sequential path "impression → click → revisit". It can be decomposed into two parts: 1) a conventional CTR estimation task, 2) post-click revisit rate (CRR) estimation task, as shown in Fig. 2. However, revisit modeling faces the following challenges:

Challenge 1: Implicit impression-revisit effect. Even if users don't click or explicitly interact with the recommended services, they may still revisit the APP (as shown in Fig. 1, space within red except for green). According to our statistics from different service scenarios, the number of non-click revisit observations is 10 to 50 times the number of post-click revisit observations. It will lose a lot of user login preference information if we simply treat such observations as non-click samples.

Challenge 2: Selection bias. Recommender systems usually suffer from the selection bias issue. Selection bias is the phenomenon that the distribution of the observed group is not representative of the group we are interested in. As shown in Fig. 1, CRR estimation is under the click space which is only part of the whole impression space. This problem often leads to severe performance degradation.

To overcome these two challenges, we propose UR-IPW, a novel method leveraging extra abundant supervised signals from both click and non-click actions. We model the user retention problem as three subproblems, including clickthrough rate (CTR) task, post-click revisit rate (CRR) task, and non-click revisit rate (NCRR) task. To improve the efficiency of leveraging data, we solve the triple task optimization problem in a multi-task paradigm. The NCRR task serves as a way to learn user revisit preference from implicit interaction behaviors and benefits the CRR task through a shared bottom embedding lookup table. Especially for revisit modeling (both CRR task and NCRR task), we point out the label noise issue. In some cases, revisit is only dominated by the characteristics of the user himself. Motivated by this, we employ a gating structure for dynamically filtering item signals.

Moreover, we address the selection bias problem from a causal perspective. To simplify the debiasing task of CRR estimation, we assume the exposure space is the entire item space we are interested in (see Fig. 1) [Ma et al., 2018]. We adopt the Inverse Propensity Weighing (IPW) based method [Rosenbaum and Rubin, 1983] to reduce the bias introduced by user's self-selection. And the IPW based method is well incorporated with the aforementioned triple task modeling. The CTR task can exactly serve as inverse propensity weighting for both CRR and NCRR tasks to obtain debiasing estimators.

The main contributions are summarized as follow:

• To the best of our knowledge, this is the first paper to formulate user retention problem as improving the probability of user's explicit interaction and revisit behavior in the framework of recommender systems, which not only addresses reducing user churn rate but also promoting user activities.
• We propose UR-IPW which optimizes user retention by making good use of sequential patterns of user actions in a triple task learning framework. Moreover, we tackle the selection bias problem in revisit modeling from a causal perspective.
• We conduct extensive experiments on both offline and online environments. The results demonstrate the effectiveness and robustness of proposed UR-IPW over previous methods.

2 Preliminary

In this section, we first introduce the basic concepts used through this paper, and then formally define our problem. Let U = {u1, . . . , uN} be a set of users and I = {i1, . . . , iM} be a set of items, and D = U × I be all user-item pairs, R ∈ RN×M be the true revisit label matrix where each entry ru,i ∈ {0, 1}, and ˆR ∈ RN×M be the predicted revisit probability matrix where each entry ˆ ru,i ∈ [0, 1]. Then the Prediction inaccuarcy P over all user-item pairs can be formulated as follows

where e(ru,i, ˆ ru,i) = -ru,i log (ˆ ru,i) - (1 - ru,i) log (1 - ˆ ru,i). Let C ∈ {0, 1}U×I be the indicator matrix where each entry cu,i is an observation indicator: cu,i = 1 if a user u explicitly interacts with item i, e.g., user u clicks item i. cu,i = 0 if user u doesn't interact with item i explicitly, e.g., user u is exposed to item i while he does not click it. But such interaction is subjective to user's self selection and lead to data that is missing not at random (MNAR) [Enders, 2010; Little and Rubin, 2019].

Post-click revisit rate (CRR) modeling is to estimate the probability of pCRR = p (r = 1|c = 1, x)1 for any given user-item pair (u, i) ∈ D, where x represents feature vector of user field and item field. Two associated probabilities are: post-view click-through rate (CTR) with pCTR = p (c = 1|x) and post-view click & revisit rate (CTCRR) with pCTCRR = p (c = 1, r = 1|x). Given impression useritem pair x, these probabilities models the sequential user actions, i.e., impression→ click → revisit. They follow Eq.(2).

The most intuitive approach to estimate CRR is to use naive estimators trained only in the click space C = {(u, i)|cu,i = 1, (u, i) ∈ D}. We evaluate these naive CRR models by averaging the cross-entropy loss over the observed data [Schnabel et al., 2016]

Similarly, we define non-click revisit rate (NCRR) as pNCRR = p (r = 1|c = 0, x) to model the sequential pattern of implicit user actions, i.e., impression → non-click → revisit. Naive NCRR estimators are trained only in the nonclick space ˜ C = {(u, i)|cu,i = 0, (u, i) ∈ D}. We evaluate these naive NCRR models by averaging the cross-entropy

1 We omit subscript u, i without loss of generality.

→

→ Click

→ Revisit

Figure 3: The Overall Architecture of UR-IPW, which consists of a core CRR task and two auxiliary tasks, namely CTR and NCRR.

loss over the observed data

But such naive estimators are biased due to users' selfselection. Thus, the subspace C and ˜ C are not representative of the whole impression space. This bias usually causes performance degradation.

3 Our Method

In this section, we provide details of our proposed method, UR-IPW (User Retention Modeling with Inverse Propensity Weighting). We first go through the overall architecture of UR-IPW and demonstrate the details of model design in each of the three tasks. Moreover, we address the selection bias problem and deal with it from a causal perspective.

3.1 Overall Architecture

Fig. 3 illustrates the overall architecture of UR-IPW, which consists of a core CRR task and two related auxiliary tasks namely CTR and NCRR. In the middle of triple tasks is the CTR task which models the probability whether user u will click item i in the whole impression space D. The CRR task on the left models revisit in the click space C while the NCRR task models it in the non-click space ˜ C. We adopt the philosophy of multi-task learning and share embedding lookup table of CRR network with CTR and NCRR networks. The paradigm of triple task learning contributes to CRR estimation in two aspects.

• The number of non-click revisit observations is 10 to 50 times the number of post-click revisit observations. The design of the NCRR module leverages a large volume of such observations and learns the user's revisit preference, given implicit interaction behaviors. It is worth noting that even the NCRR task does not affect the CRR task directly, the optimization of NCRR enables the shared bottom embedding lookup table to better represent revisit preference and benefit the CRR task.
• Embedding lookup table maps large scale sparse inputs into low dimensional representations. It contributes most of the parameters of deep network and the learning requires a huge volume of training samples. The amount of training data in CTR/NCRR tasks is generally larger than that in the CRR task by 1 or 2 magnitudes. This parameter sharing mechanism allows the CRR task to learn from non-click and non-click revisit impressions. Meanwhile, triple task learning co-trains three tasks simultaneously as if they were one task, which can not only reduce storage space for saving duplicate embedding matrixes but also less time-consuming.

3.2 ItemFilter Gate

Moreover, we consider the critical label noise problem in revisit modeling. In some cases whether a user explicitly or implicitly interact with our recommendation, the revisit label might be weekly related to the treatment. In other words, sometimes revisit is only dominated by the characteristics of the user himself. Motivated by this, we propose a gating structure in both CRR and NCRR tasks (as shown in Fig. 3). For user-item pair (u, i), the input feature vector xu,i consists of two parts: user representation xu and item representation xi. We first formulate the gate network as

where σ(·) is the sigmoid activation function, W (g) is the parameter matrix, and gu,i is a scaler which represents the extent to which item characteristics can affect the revisit prediction. Then the input feature vector of CRR or NCRR task is organized asx(gated)

Finally, x(gated) u,i is fed into a multi-layer perceptron to predict the revisit probability in both CRR and NCRR tasks.

Remark. Consider two extreme cases: 1) If the revisit behavior of user is totally irrelevant to the item, which means the revisit label only reflects the user's natural revisit preference, then the output of gate g will be almost 0. Under this case, only user's feature vector will be used to estimate revisit probability; 2) If the revisit behavior is greatly affected by the recommended item, then the output of gate g will be almost 1. Under this case, the characteristics of user and item contribute equally to user's revisit.

3.3 Inverse PropensityWeighting

Addressing the selection bias problem exists in the observed data, we incorporate the IPW-based (Inverse Propensity Weighting) estimators into the proposed triple task learning framework. For CRR estimation, let the marginal probability p (cu,i = 1|xu,i) denote the propensity score pu,i, of observing an entry in R. In practice, the real propensity score pu,i can not be obtained directly. Instead, the IPW-based method will estimate the real propensity as ˆ pu,i and uses ˆ pu,i to inversely weight prediction loss [Hirano et al., 2003; Imbens and Rubin, 2015].

The most commonly used ˆ pu,i estimation method is to employ an independent logistic regression [Austin, 2011]. In UR-IPW, the estimation of ˆ pu,i is well incorporated into the triple task learning framework. The output of CTR prediction is exactly the propensity score for CRR estimation. Unlike the naive estimator εNaive CRR , the IPW estimator is unbiased, which only requires accurately estimated propensity. But such a basic IPW estimator often suffers from high variance [Schnabel et al., 2016]. In other words, we are paying for the unbiased-ness of IPW in terms of variability. Hence, we adopt the Self-Normalized Inverse Propensity Scoring (SNIPS) estimator which introduces a small bias but has lower variance [Swaminathan and Joachims, 2015; Hesterberg, 1995], and the loss function of CRR estimation can be written as follows,

Similarly, we also employ SNIPS estimator for NCRR estimation. The propensity of non-click is

and the loss function of NCRR estimation is written as

Finally, the loss function of UR-IPW is defined as Eq.(11). It consists of CTR loss term and two debiasing estimators.

4 Experiments

To evaluate the effectiveness of the proposed method URIPW, we conducted extensive experiments on offline datasets collected from recommendation scenarios in Alipay. Our proposed UR-IPW is compared with some representative stateof-the-art (SOTA) methods, including DNN, DeepFM [Guo et al., 2017], ESMM [Ma et al., 2018], and MultiIPW [Zhang et al., 2020]. Moreover, we evaluated the effectiveness of our method in a relatively unbiased setting by conducting live A/B testing experiments.

4.1 Experimental Setup

Production Dataset

Our two production datasets are collected from Alipay's recommender system. The two datasets come from different service scenarios and contain traffic logs lasting 7 days. 2 The final datasets are randomly sampled from the original traffic logs according to a fixed ratio. User retention rate is an important business objective for Internet companies. In the following experiments, we focus on improving day 1 retention rate, which is the percentage of users who return on the next day. To be more specific, if the user was exposed to the recommender system and revisit the APP in the next day, the revisit label is 1 otherwise 0. Table 1 summarizes the statistics of the two datasets. The features include user features, item features, and combination features. For each dataset, we split the first 4 days in the time sequence to be training set while the rest to be test set.

2 1) The APP provides various financial services and the data involved is closely related to user privacy. The release of related datasets requires strict approval, we are going through the relevant approval procedure; 2) The data set does not contain any Personal Identifiable Information (PII); 3) The data set is desensitized and encrypted; 4) Adequate data protection was carried out during the experiment to prevent the risk of data copy leakage, and the data set was destroyed after the experiment; 5) The data set is only used for academic research, it does not represent any real business situation.

Baselines

• DNN is a deep neural network baseline model, which has the same structure and hyper-parameters with a single branch in UR-IPW. It is trained with samples on the path "click → revisit" to estimate CRR.
• DeepFM is a factorization-machine based neural network initially designed for CTR prediction, here we adopt the model for CRR estimation.
• ESMM is originally proposed to estimate CVR using a CTR task and a CTCVR task. Here we adopt ESMM to model user sequential path "impression → click → revisit" and is employed for CRR estimation task. For a fair comparison, we use the same backbone structure as the DNN model for ESMM.
• Multi-IPW adopts IPW estimator in a multi-task learning framework to address the selection bias issue in CVR estimation task and provides an unbiased CVR estimator. Here we adopt Multi-IPW for CRR estimation task by eliminating the harmful effects of selection bias. We adopt the the same structure and hyper-parameters with the above deep models, for making a fair comparison.
• UR-IPW (NG) is a lite version of UR-IPWwithout the proposed item filter gate component.

In a nutshell, the basic DNN and DeepFM models learn to predict pCRR directly in the click space without considering the selection bias issue. ESMM and Multi-IPW address the selection bias problem. ESMMmodels pCTR and pCTCRR simultaneously over the entire space. Multi-IPW goes a step further to solve it from a causal perspective. The above methods are originally proposed for the e-commerce scenario, which only models path "impression → click → revisit". UR-IPW (NG) is for an ablation study to illustrate the effectiveness of the designed item filter gate component.

4.2 Implementation Details

All the deep neural network-based models are implemented in TensorFlow v1.13 using Adam optimizer. The learning rate is set as 0.0005 and the mini-batch size is set as 1024. Crossentropy loss function is used for each prediction task in all models. There is 5 layers in the MLP, where the dimension of each layer is set as 512×256×128×32×2.

4.3 Offline Results

In this subsection, we report the CRR AUC and CTCRR AUC of all competitors on the offline test datasets. Scene B is relatively small in scale and is used to verifying the effectiveness and robustness of the proposed method. As shown in Table 2, UR-IPW achieves an absolute AUC gain of 1.5% (scene A) and 2.83% (scene B) on CRR task. For CTCRR task, the AUC gain is 0.69% (scene A) and 2.94% (scene B). This is a significant improvement for industrial application where 0.1% AUC gain is remarkable. It's notable that the unbiased offline testing set is unobtainable in real practice, we cannot force users to randomly click on items to generate unbiased data for CRR estimations. But such offline evaluation still makes sense and is widely accepted by previous researches [Ma et al., 2018; Zhang et al., 2020].

Table 2 presents a summary of the results. It implies that DeepFM achieves slight improvement over the basic DNN model which benefits from the low- and high-order feature interactions learned by the FM layer. However, both DNN and DeepFM are dominated by ESMM and Multi-IPW baselines, which are capable of handling the selection bias problem in CRR estimation. Although ESMM addresses the selection bias problem in CRR estimation, it's still biased [Zhang et al., 2020]. IPW based methods address the selection bias problem from a causal perspective. Table 2 demonstrates that IPW-based methods outperforms ESMM in both production datasets, which implies that causal approach is efficient for eliminating the effect of the selection bias issue.

Our proposed methods provide an elegant solution which not only addresses the selection bias problem but also improves CRR estimation by learning from implicit impressionrevisit samples with triple task modeling. As shown in Table 2, our proposed methods outperform Multi-IPW in both dataset owing to leveraging extra abundant supervised signals from post view non-click user behaviors. UR-IPW (NG) is for an ablation study to illustrate the effectiveness of the proposed item filter gate in solving revisit label noise. Table 2 implies that UR-IPW achieves an absolute AUC gain of 0.1% over on CTCRR task on Scene A and an absolute AUC gain of 1.0% over on CTCRR task on Scene B.

4.4 Live Experiment Results

To address the limitation of offline evaluation and demonstrate the practical value of UR-IPW, we further deployed our proposed UR-IPW in a recommendation scenario that owns millions of active users per day.

We conduct strict A/B testing experiments with 3 × 15% traffic for a week (from 08/03 to 08/09) for two purposes: to cover enough users and to better reveal the long term user retention improvement. The DNN model is the master model already serving in the recommender system. We allocate 15% traffic of DNN as the baseline and allocate 15% for MultiIPW and UR-IPW respectively for comparison. For each user exposed to the APP, we score items by a merge of pCTR and pCRR, which accounts for both reducing churn rate and promoting user activities. The online result is shown in Fig. 4. The x-axis is date and the y-axis represents the most important metric in user retention: amount of click&revisit uv (unique user). It demonstrates that our UR-IPWmethod significantly outperforms the baseline model (red vs green). But the improvement of Multi-IPW compared to the baseline is not stable enough (blue vs green).

Table 3 illustrates the accumulated relative improvements of experimental models compared to the baseline model. We utilize two business objectives as our metrics (Objective 1: the amount of click&revisit uv, Objective 2: the day 1 revisit uv). We demonstrate the confidence intervals below the relative improvements. As shown in Table 3, both Multi-IPW and UR-IPW perform better than the baseline model. By leveraging the implicit impression-revisit signal and utilizing gate component, UR-IPW achieved greater improvement than Multi-IPW. Compared to the model already serving in the recommender system, UR-IPW improved the objective1 by 1.44% and the objective2 by 0.24%. It's worth noting that both of the two improvements are statistical significant (with p-value less than 0.05). This practice-oriented experiment demonstrates the effectiveness of our model in real-word recommendation scenarios.

5 RelatedWork

Existing research in the field of user retention focuses on Customer churn prediction (CCP) [ ´Oskarsd´ottir et al., 2016]. The most important part of CCP approaches is to predict customers who are expected to churn. According to recent researches, machine-learning techniques have been widely used for predicting customers who are expected to churn [Amin et al., 2019]. Such as ensemble-based methods [Idris and Khan, 2014], probabilistic-based methods [Kirui et al., 2013], Neural Networks-based methods [Kasiran et al., 2014], and data certainty based methods [Amin et al., 2019]. However, all the approaches aforementioned are designed only for prediction customer churn and contributes to reducing churn rate. Those approaches only partially improve user retention ratio.

Selection bias is a widely-recognized issue in recommender systems [De Myttenaere et al., 2014; Schnabel et al., 2016]. The work of [Ma et al., 2018] models CVR estimation in the entire exposure space to reduce selection bias, but it's still biased [Zhang et al., 2020]. Causal inference [Imbens and Rubin, 2015] offers a way for handling the MNAR and selection bias problems in recommender systems [Schnabel et al., 2016]. Recent work has combined IPW-based methods with multi-task learning for unbiased CVR estimation in e-commerce [Zhang et al., 2020]. However, when dealing with small scale data, propensity score-based methods may suffer from high variance and high estimation error problems. Various works have been done to overcome those drawbacks [Saito, 2019; Wang et al., 2019].

Given the work above, several questions remain unanswered: 1) The existing research of user retention is mostly designed for reducing churn rate and is not competent for promoting user activity. 2) Existing IPW based methods are designed to handle single task, and thus not be feasible for handling the selection bias within the CRR task and NCRR task.

6 Conclusion

In this paper, we propose a novel approach UR-IPW to model user retention as a CRR estimation problem, which addresses not only reducing churn rate but also promoting user activities. With the help of triple task modeling and inverse propensity weighting based causal approach, UR-IPW elegantly tackles the challenges of implicit impression-revisit effect and the selection bias problem. The proposed framework can be easier generalized to other scenarios by defining different types of explicit/implicit interactions. Both offline and live experiments with millions of users demonstrate that our method significantly outperforms the previous methods. To the best of our knowledge, this is the first work to optimize user retention by modeling user sequential behavior patterns from a causal perspective.