AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System

1 Introduction

Recommender system has become an essential service on online E-commerce business and content platforms to deliver items that best fit users' interests from the substantial number of candidates. Recently, advancements of deep learning (DL) have innovated the recommending strategies in real-world applications [Covington et al., 2016; Zhou et al., 2018] to enable accurate and personalized recommendation.

DL-based recommender systems have to process both sequential and non-sequential input features, and thereby follow a characterized architecture compared to the general DL

∗Equal Contribution

tasks. A canonical DL-based recommendation model usually includes three parts: (1) Behavior modeling [Hidasi and Karatzoglou, 2018; Zhou et al., 2018] probes into capturing user's diverse and dynamic interest from historical behaviors; (2) Interaction exploration [Cheng et al., 2016; Guo et al., 2017] finds useful interactions among different fields to provide memorization and intuitive conjunction evidence for the recommendation. (3) Multi-layer perceptron (MLP) investigation [Covington et al., 2016] aggregates the inputs with the results from the previous parts, and then fuses the features with hidden layers in the MLP.

Industry demands efficiently designing the three parts in the DL-based recommendation model to improve the accuracy and personalization for given recommender tasks in business. Nevertheless, it is hard to find a unified model meeting the requirements in all the scenarios. For the models of behavior modeling, recurrent neural network (RNN) [Hidasi and Karatzoglou, 2018] is hard to preserve long-term behavioral dependencies even though employing gated memory cells. Convolutional neural network (CNN) [Yuan et al., 2019] is capable of learning local feature combinations yet it relies on a wide kernel or deep layers to distinguish longterm interests. The attention mechanism [Zhou et al., 2018] directly aggregates the entire behavior sequence, but it can not capture the evolution of the user's preference [Zhou et al., 2019]. Self-attention [Vaswani et al., 2017] is better for modeling long-term behaviors [Feng et al., 2019; Sun et al., 2019], but it is hard to be deployed in real-time applications that require fast inference with limited resources. Regarding the methods for interaction exploration and MLP investigation, vanilla MLP [Covington et al., 2016] implicitly generalizes high-order information in the network. To help memorize useful features from raw inputs, the existing literature extracts implicit and explicit low-order [Cheng et al., 2016; Guo et al., 2017] and high-order [Lian et al., 2018] interactions and combine them with a pre-defined MLP. However, these methods either require hand-crafted cross features or simply enumerate interactions of bounded degree, and thus introducing noise in the model [Liu et al., 2020]. Moreover, the MLP specified manually or by grid search is usually suboptimal. Hence, adopting and coordinating the task-aware architectures for those three parts are the main challenge for building efficient and accurate recommender systems.

Recently, the neural architecture search (NAS) paradigm

[Zoph and Le, 2017; Real et al., 2019] is proposed to automatically design deep learning models by searching task-specific and data-oriented optimal architecture from the search space, thereby mitigating a lot of human efforts. Some latest attempts [Luo et al., 2019; Liu et al., 2020; Song et al., 2020] incorporate NAS with the recommendation. Nevertheless, they all neglect behavior modeling and only consider interaction exploration and MLP investigation in the restricted search spaces. Different from these methods, we introduce AMEIR, namely Automatic behavior Modeling, interaction Exploration and MLP Investigation in the Recommender system, to automatically find the complete recommendation models for adapting various recommender tasks and mitigating manual efforts. Specifically, AMEIR divides the backbone recommendation models based on the three stages of behavior modeling, interaction exploration, MLP aggregation, and introduces a novel search space containing three tailored subspaces that are designed to cover most of the representative recommender systems and thus allow for searching better architectures. Facing the industrial demands of agile development and architecture iteration, AMEIR realizes the efficient while competitive oneshot random search [Li and Talwalkar, 2019], and proposes a three-step searching pipeline to progressively find the ideal architectures for the corresponding three stages. The search results will be assembled as the final outcome. The experimental results over various recommendation scenarios show that our automatic paradigm consistently achieves state-ofthe-art performance despite the influence of multiple runs, and could outperform both the strongest baselines with elaborate manual design and leading NAS methods with comparable search cost, indicating efficacy, efficiency, and robustness of the proposed method. Further analysis reveals that the overall time cost of AMEIR is similar to the baseline methods without architecture search and the searched models always possess lower model complexity, which demonstrates that AMEIR is efficient in both search and inference phases.

2 AMEIR

In Sec.2, we will introduce the three-stage search space, and the tailored three-step one-shot searching pipeline in AMEIR. Finally, we will discuss the relationship between AMEIR and representative recommender systems.

2.1 Backbone Model in AMEIR's Search Space

The recommendation models in AMEIR's search space share the same three-stage backbone, as illustrated in Fig.1.

Stage 0: Feature Representation and Embedding. In the common recommender systems, the input features are collected in multi-field categorical form, and segregated into sequential features and non-sequential features: (1) The sequential features describe the user's behaviors, which can be represented by a sequence of behavior elements. Each behavior element contains the item profile and the historical context profile. (2) The non-sequential features depict the attribute information of the current recommendation, including user profile, instant context profile, and the optional target profile for click-through rate (CTR) prediction task.

Conforming to [Covington et al., 2016; Zhou et al., 2018], AMEIR transforms the data instances into high-dimensional sparse vectors via one-hot or multi-hot encoding. Each data instance can be formally represented by x = [xa , xb ] = [xa1 , xa2 , ..., xaT , xb1 , xb2 , ..., xbN ]. xa = [xa1 , xa2 , ..., xaT ] denotes the encoded sequential behavior elements of length T with each behavior element xai grouped in Na categorical fields of the same dimension, and xb = [xb1 , xb2 , ..., xbN ] represents N encoded vectors of non-sequential feature fields. AMEIR compresses the sparse features into low dimensional dense vectors via embedding technique. Suppose there are F unique features. AMEIR creates the embedding matrix E ∈ RF×K , where each embedding vector has dimension K. The embedded data instance can then be represented by e = [ea , eb ] = [ea1 , ea2 , ..., eaT , eb1 , eb2 , ..., ebN ].

Stage 1: Behavior Modeling on Sequential Features. To extract the sequential representation from user's behavior, AMEIR introduces a block-wise behavior modeling network to model sequential patterns based on grouped behavioral embedding Ha0 = [ea1 , ea2 , ..., eaT ]. The behavior modeling network is constructed by stacking a fixed number of La residual building blocks, where the l-th block receives the hidden state Hal-1 from the previous block as input and generates a new hidden state Hal of the same dimension. Each block consists of three consecutive operations of normalization, layer, and activation, which are selected from the corresponding operation sets in the search subspace of building blocks. The hidden state of layer l is formulated as:

Hal = Actal (Layeral (Normal (Hal-1 ))) +Hal-1 , (1) where Actal , Layeral and Normal are the selected activation, layer and normalization operations for layer l. As the fully connected networks can only handle fixed-length inputs, AMEIR compresses the final hidden states into fixed dimensions via sequence-level sum pooling: hapool = ∑Ti=1 haLa,i , where haLa,i denotes the i-th element in HaLa . Moreover, to extract the latest interest of the user, the last hidden state haLa,T is also included in the result. Besides, when the target item is involved, AMEIR adopts an attention layer to extract a target-aware representation: haatt = ∑Ti=1 aihaLa,i , with ai computed by the local activation unit [Zhou et al., 2018]. The concatenation ha = [hapool , haLa,T , haatt ] is served as the sequential feature extracted from Stage 1. In practice, this stage remains the procedure to execute looking up embedding only once, which is orthogonal to the industrial large-scale sparsefeatures lookup optimizations.

Stage 2: Interaction Exploration on Non-sequential Features. Learning effective feature interactions is crucial for recommender systems. Different from the widely-used methods that manually extract useful interactions or exhaustively enumerate interactions of bounded degrees [Cheng et al., 2016; Guo et al., 2017; Lian et al., 2018], AMEIR explores a small number of M beneficial interactions pb = [pb1 , pb2 , ..., pbM ] from the search subspace of interactions. The search subspace contains all low-order and high-order interactions among the non-sequential features, where each interaction has the same dimension as the input embeddings. Stage 3: Aggregation MLP Investigation with Shortcut Connection. Given the inputs of sequential feature, non-sequential features, and explored interactions hc0 = [ha , eb , pb ], AMEIR aggregates these features by taking advantage of both generalization from MLP and memorization from a shortcut connection. As MLP affects both recommendation quality and efficiency, AMEIR searches for the MLP settings in Stage 3. Assume the aggregation MLP has Lc layers. AMEIR chooses the dimension and activation function for each layer from the search subspace of MLP. The forward pass of each hidden layer l can be depicted as:

where hcl , Wc l , bcl and Actcl are the output, weight, bias and activation function of the l-th layer. Meanwhile, inspired by the bypaths in [Cheng et al., 2016; Guo et al., 2017; Lian et al., 2018], AMEIR introduces a Squeeze-and-Excitation (SE) [Hu et al., 2018] shortcut connection to introduce attentive recalibration on the feature embeddings and explored interactions Qse = [eb , pb ] = [eb1 , ..., ebN , pb1 , ..., pbM ]. The attention units ase = [ase 1 , ase 2 , ..., aseM+N ] are computed by ase = σ(Wse 2 ReLU(Wse 1 Qsevse )), where vse is applied for linear projection, and Wse 1 ,Wse 2 are weight matrices in the fully-connected layers. The output of SE connection is given by the weighted average hse = ∑M+N i=1 ase i qse i , where qse i is the i-th element in Qse . AMEIR concatenates the results from aggregation MLP and SE connection as the output representation hc = [hcLc , hse ].

Loss Function. For the CTR prediction task, AMEIR minimizes the binary log loss for optimization:

where σ(wThc ) is the predicted CTR score, and S is the training set with x as input and y ∈ {0, 1} as label. For the item retrieval, we adopt the sampled cross entropy loss:

where σ(eTi Whc ) is the relevance score between target item i and final representation; S is the training set of input feature x and target item i; I - i is the item set of negative samples.

2.2 Search Subspaces of Three Stages

For each stage in the backbone, AMEIR introduces a corresponding search subspace to cover the representative recommendation models. We will describe each of the three subspaces below and show a detailed comparison in Sec.2.4.

Search Subspace of Building Blocks. In order to cover the existing behavior modeling methods, AMEIR proposes a search subspace of building blocks in the behavior modeling network, including three operation sets for normalization, layer, and activation. Specifically, AMEIR collects the normalization set of {Layer normalization [Ba et al., 2016], None} and activation set of {ReLU, GeLU [Vaswani et al., 2017], Swish [Ramachandran et al., 2017], Identity} that are commonly used in the previous methods. Regarding the layer operations, AMEIR introduces four categories of candidate layers, namely convolutional layers, recurrent layers, pooling layers, and attention layers to identify sequential patterns in the user history. The convolutional layers are all one-dimension, including standard convolution with kernel size {1, 3} and dilated convolution with kernel size {3, 5, 7}. The pooling layers consist of average and max poolings with kernel size 3. Both convolutional and pooling layers are of stride 1 and SAME padding. Bi-directional GRU (BiGRU) is employed as the recurrent layer as it is faster than Bi-LSTMwithout loss of precision. Two-head and four-head bi-directional self attentions [Vaswani et al., 2017] are also introduced for better behavior modeling. Additionally, when the target item appears in the input, the layer operation set will associate an attention layer that attends to each sequence position from target [Zhou et al., 2018]. Besides, zero operation is also included to implicitly allow a dynamic depth of behavior modeling network. It is worth emphasizing that the operation sets are proposed to cover the existing methods. When a new operation is developed, it can be easily added to the operation set to find better recommendation models.

Search Subspace of Interactions. The existing literature adopts Hadamard product [Lian et al., 2018], inner product [Guo et al., 2017; Qu et al., 2018], bilinear function [Huang et al., 2019] and cross product [Cheng et al., 2016] in the feature interactions. Hadamard product calculates the element-wise product among the feature embeddings, which can be exploited by both MLP and shortcut connection. The inner product can be seen as a simple form of Hadamard product compressed by sum pooling. Bilinear function introduces extra parameters on Hadamard product to learn fine-grained representations, but it can not handle high-order interactions and does not exhibit superiority over Hadamard product when combining with MLP [Huang et al., 2019]. Although the cross product can yield a more flexible form of interactions, it can only be used in the shortcuts [Luo et al., 2019] because the parameters will exponentially explode in the order of interactions and can not be well trained due to the low frequency of occurrence. Therefore, AMEIR chooses Hadamard product as the interaction function due to its expressive power and flexibility. The search subspace of interactions includes all low-order and highorder Hadamard interactions. An interaction of order r can be expressed as ebi1 ⊙ebi2 ⊙...⊙ebir . where ⊙ denotes Hadamard product and ebi1 , ebi2 , ..., ebir are selected from eb .

Search Subspace of MLP. The search subspace of MLP includes dimensions and activation functions of hidden layers. Assume Kc 0 is the dimension of hc0 . The dimensions of hidden layers are chosen from {0.1, 0.2, ..., 1.0} of Kc 0 while ensuring monotonically non-increasing following the practice. The activation functions are selected from {ReLU, Swish, Identity, Dice [Zhou et al., 2018]}.

2.3 Three-step One-shot Searching Pipeline

One-shot NAS in AMEIR. Recent approaches [Pham et al., 2018] introduce efficient one-shot weight-sharing paradigm in NAS to boost search efficiency, where all child models share the weights of common operations in a large one-shot model that subsumes every possible architecture in the search space. Though diverse complex search algorithms have been coupled with one-shot NAS, the latest study [Li and Talwalkar, 2019] reveals that the efficient one-shot random search is surprisingly better than the complex gradient-based methods of DARTS [Liu et al., 2019], SNAS [Xie et al., 2019] and RL-based methods of ENAS [Pham et al., 2018]. To facilitate the industrial demands of agile development and architecture iteration, AMEIR realizes the one-shot random search in the recommendation model: (1) The child models are randomly sampled from the search spaces (actually subspaces) to train the shared weights of the one-shot model and validate its performance on a small subset of validation instances to ensure adequate training while avoiding overfitting. (2) A subset of child models is randomly selected and evaluated on the same validation subset using the inherited weights from the one-shot model. (3) The child models with the best performance will be derived and retrained for the final recommendation outcome. Moreover, to reduce the magnitude of the search space and find the better recommendation models, AMEIR modularizes the search process into three steps matching the three stages in the backbone model by progressively searching for the architectures in the three search subspaces. Meanwhile, AMEIR ensures the fairness of one-shot model training to alleviate the representation shift caused by weight sharing. Details of the three steps are shown below.

Step 1: Behavior Modeling Search. In Step 1, AMEIR searches for the behavior modeling networks from the search subspace of building blocks, as shown in Fig.1. The normalization, layer, and activation operations are randomly selected from the operation sets to build the recommendation model. To decouple behavior modeling from other stages, a pre-defined MLP is exploited to combine ha and eb . The training and evaluation are the same as previously described. Step 2: Interaction Exploration. In Step 2, AMEIR combines one-shot NAS and sequential model-based optimization (SMBO) algorithm [Liu et al., 2018] to explore interactions on non-sequential features. The interaction set pb is initialized with non-sequential features eb , and then progressively updated by several cycles of evolution to increase the interaction order, as illustrated in Fig.1: (a) The candidate interactions in pb intersect with all non-sequential fields from eb via Hadamard product to mutate for higher-order interactions. The expanded interaction set is denoted by pb′ . (b) The intersections in pb′ are trained by one-shot NAS and evaluated on the validation set. (c) Top-k interactions with the highest validation fitness are retained as the new pb , while other interactions are filtered out to avoid exhaustively visiting all possible solutions. After the evolution, AMEIR selects top-M features in pb as the resulting interaction set. In order to train and evaluate interactions in the one-shot model, AMEIR utilizes another pre-defined MLP to aggregate interactions in pb′ with non-sequential embeddings in eb , while the behavior modeling here is left out for efficiency. As the linear projection of feature concatenation is equivalent to feature-wise addition of projected results, i.e., WhT = [W1,W2, ...,WN][h1, h2, ..., hN]T = ∑Ni=1 WihTi , the one-shot NAS in Step 2 is efficiently realized by adding the projections of one uniformly-sampled interaction and all embeddings in eb as the output of the first hidden layer, without accessing to all interactions in pb′ . To further speed up the search process, the weights of the oneshot model are inherited in consecutive evolution cycles.

Step 3: MLP Investigation. In Step 3, AMEIR investigates the dimensions and activations in the aggregation MLP based on the found behavior modeling network and interactions. To find the hidden sizes of MLP, AMEIR pre-allocates a weight matrix with the maximum dimension Wc ∈ RKc 0 ×Kc 0 for each layer. Suppose hin and hout are chosen to be the hidden sizes of previous and current layers. AMEIR slices out sub-matrixWc [: hin, : hout] to assemble the MLP in Eqn. (2). Other details are similar to Steps 1 and 2.

Derivation. After the three-step random search, AMEIR assembles the search results as the final outcome and retrains the derived model for evaluation. It is worth emphasizing that the derived model is initialized with the one-shot model weights because the training protocols of model architectures and datasets are the same for both one-shot model and child models, which is in contrast to the most existing methods that adopt proxy task for one-shot model training [Liu et al., 2019; Xie et al., 2019]. The inherited weights accelerate the retraining process and compensate for the search cost.

2.4 RelatedWork to AMEIR

Behavior Modeling. The existing methods introduce RNN [Hidasi and Karatzoglou, 2018], CNN [Yuan et al.,

2019], or Transformer-based [Sun et al., 2019] behavior modeling networks to capture sequential relations and evolved interests from user behaviors. Then, the sequential features are extracted from the sequence-level hidden states by either taking the last hidden state [Kang and McAuley, 2018] or aggregating the sequence by a pooling layer [Cheng et al., 2016; Covington et al., 2016]. Some recent works [Zhou et al., 2018] further employ an attention mechanism between the sequential representations and target item to retrieve the user's main proposal. These methods can all be found in AMEIR's search space. Specifically, the search subspace of building blocks contains all layer, activation, and normalization operations in those behavior modeling networks. Moreover, the last hidden state, pooling-based aggregation, and attentionbased aggregation are all used for sequence compression.

Interaction Exploration. In parallel with behavior modeling, some works resort to explore feature interactions in the DL-based recommendation models. LR [Cheng et al., 2016], FM [Guo et al., 2017], xDeepFM [Lian et al., 2018] are exploited to import explicit low-order or high-order interactions through shortcut connections, i.e., concatenating interactions to the last hidden layer of MLP, and then apply linear projection with learnable variables or all-one vectors to generate the prediction scores jointly with the hidden layers. Other than the shortcut connection, FibiNet [Huang et al., 2019] take interactions as the input of MLP, while PNN [Qu et al., 2018] further combines interactions with raw embeddings to induce more features in the network. It is not hard to find that all these methods can be derived from AMEIR's search space. Concretely, all low-order and high-order interactions can be extracted from the search subspace of interactions. Meanwhile, raw input embeddings and explored interactions are used by both shortcut connection and MLP, covering the existing interaction-combining strategies. Besides, the attentive SE connection could implicitly characterize attention-based, regression-based, and FM-based shortcut connections.

MLP Investigation. The existing literature [Covington et al., 2016; Zhou et al., 2018] mainly uses sub-optimal handcrafted MLPs to aggregate features, where activation functions and hidden sizes are designed manually or by grid search. Different from these methods, AMEIR tries to find the appropriate MLP through one-shot NAS in a discrete yet flexible search subspace that covers common MLP settings.

Based on the above analysis, it can be inferred that AMEIR's search space covers most of the representative recommendation models. Therefore, AMEIR can perform as good as these methods in various scenarios, and further derive better models when conducting exhaustive search.

3 Experiments

In Sec.3, we will investigate AMEIR on various scenarios, perform additional ablation study and efficiency analysis.

3.1 Industrial Results for AMEIR

To verify all three steps of AMEIR in the industrial scenario, we refer to the Alimama CTR dataset [Feng et al., 2019] that comprises both sequential and non-sequential features for a comprehensive comparison. We use similar preprocessing settings as [Feng et al., 2019], where the embedding size is set to 4 and the maximum sequence length is set to 50. The Adam optimizer with an initial learning rate 1e-5 and batch size 1024 is employed for both AMEIR and baseline methods. For the training of the one-shot model, a single cosine schedule is introduced for learning rate decay. In Steps 1 and 3, AMEIR selects the top-5 models from the randomly sampled 2000 architectures for further evaluation, while in Step 2, AMEIR conducts 4 cycles of evolution, retains top-50 interactions at each cycle, and finally reserves M = 5 interactions as pb . La and Lc are set to 3 and 2 in the backbone model, and the pre-defined MLP is set to [200, 80] during Steps 1 and 2. By conforming to [Li and Talwalkar, 2019; Liu et al., 2019], we run AMEIR for 4 times and report the mean metrics across 4 runs as the final results.

Tab. 1 depicts the results on Alimama dataset. AMEIRA (only uses behavior modeling result from Step 1) presents competitive performance with the promising empirical architectures DIEN and DSIN. Furthermore, a significant improvement (note that an improvement of AUC around 0.00050.001 is already regarded as practically significant in the industrial scenarios [Feng et al., 2019]) can be observed when combining AMEIR-A with searched interactions (AMEIR-B) and MLP (AMEIR-C). An interesting point on the Alimama dataset is that compared to sequential modeling and interaction exploration, the MLP search significantly contributes to the final results, which indicates that all three stages/steps in AMEIR are necessary for high-quality recommendations.

3.2 Applicable for Various Scenarios

To show AMEIR is applicable for various recommendation tasks, we evaluate AMEIR on two additional scenarios, namely sequential scenario, and non-sequential scenario: (1) The sequential scenario only presents sequential user behaviors to examine Step 1 behavior modeling. The training is based on the item retrieval task. The sequential datasets include Amazon Beauty and Steam [Kang and McAuley, 2018]. (2) The non-sequential scenario is introduced to verify the Step 2 interaction exploration and Step 3 MLP investigation, where only non-sequential features are provided for the recommendation. The non-sequential datasets include benchmarking CTR datasets Criteo [Guo et al., 2017] and Avazu [Qu et al., 2018].

Evaluation on Sequential Scenario. Tab. 2 summarizes the results on Beauty and Steam datasets. The hand-crafted methods, including the strongest baselines BERT4Rec and NextItNet, can not dominate both datasets, which empirically justifies that the best architecture is usually task-aware. In contrast, AMEIR consistently achieves better accuracy compared to the baselines with well-designed architectures, indicating the strength of AMEIR and the capability of searching better models from the search subspace of building blocks.

Evaluation on Non-sequential Scenario. Tab. 3 presents the results on non-sequential datasets. It can be observed that AMEIR-I (DeepFM backbone with interactions explored by AMEIR) consistently surpasses competitive hand-crafted baselines xDeepFM and FibiNet as well as AutoML-based baselines AutoCross and AutoFIS on different tasks over multiple runs, validating the efficacy of interaction exploration in Step 2. Moreover, our method gains remarkable improvement over AMEIR-I and exceeds AutoCTR when combining explored interactions with the SE connection (AMEIR-B) and MLP investigation (AMEIR-C). The result verifies the design of both the backbone and search subspaces.

3.3 Ablation Study

Ablation Study on NAS Methods. To study the importance of the proposed search method, we evaluate the performance of different NAS algorithms on the same search space of the Alimama dataset. As shown in Tab. 4, the one-shot random search achieves better performance than both brute-force methods of random, RL, EA, and the algorithmic complex methods of ENAS, DARTS, SNAS with lower search cost. Moreover, AMEIR with the three-stage one-shot searching pipeline brings about a remarkable improvement on the oneshot random search under a similar time budget, demonstrating the effectiveness and efficiency of the proposed method.

Ablation Study on Backbone Model. To further analyze the impact of the backbone model, we isolate different components in Stages 2 and 3, and report the mean metrics on the non-sequential datasets. As shown in Tab. 5, the interactions (searched) applied to MLP and FM bring orthogonal improvement on vanilla DNN. The result reveals that both combining strategies contribute to the result. Additionally, SE interaction (AMEIR-B) and MLP search (AMEIR-C) could further boost the performance of recommendation, suggesting the superiority of the backbone design in AMEIR.

3.4 Efficiency in Search and Inference Phases

Search Efficiency. The search cost of AMEIR on the Beauty, Steam, Avazu, Criteo, Alimama datasets are 0.2, 0.2, 0.25, 0.25, 0.5 GPU-days respectively on a single Tesla V100 GPU, which are all similar to training the individual models of baseline methods. As AMEIR uses the trained one-shot weights to initialize the derived model, the fine-tuning time could generally be ignored. Therefore, the overall time cost of AMEIR is roughly the same as the methods that do not employ NAS. The result proves the efficiency of AMEIR in the search phase, and it is evident that AMEIR can help facilitate

the design of recommender systems in real-world scenarios.

Inference Efficiency. We record the queries/samples-persecond (QPS) of AMEIR and the compared models to examine the inference speed on the Alimama dataset in Tab. 4. Apart from the improvement in accuracy, AMEIR is also competitive in efficiency compared with the commonly-used attention-based models. Besides, the searched MLPs are always much smaller than the hand-crafted ones, such that the numbers of parameters of the searched models are less than half of the baselines. The results indicate that instead of increasing the capacity, AMEIR coordinates the three parts in the recommendation model to find the ideal architecture.

4 Conclusion

In this paper, we introduce AMEIR for automatic behavior modeling, interaction exploration, and MLP investigation in the recommender systems. Owing to the design of novel search space and search pipeline, AMEIR outperforms the competitive baselines in various scenarios with lower model complexity and comparable time cost, demonstrating efficiency, effectiveness, and robustness of the proposed method.

Acknowledgements

This work is partially supported by National Key Research and Development Program No. 2017YFB0803302, Beijing Academy of Artificial Intelligence (BAAI), NSFC 62032003 and 61632017.