{"type":"ScholarlyArticle","title":"MLP4Rec: A Pure MLP Architecture for Sequential Recommendations","authors":[{"name":"Muyang Li","slug":"muyang-li","given":"Muyang","family":"Li","affiliation":"University of Sydney"},{"name":"Xiangyu Zhao","slug":"xiangyu-zhao","given":"Xiangyu","family":"Zhao","affiliation":"City University of Hong Kong"},{"name":"Chuan Lyu","slug":"chuan-lyu","given":"Chuan","family":"Lyu","affiliation":"Zhejiang University"},{"name":"Minghao Zhao","slug":"minghao-zhao","given":"Minghao","family":"Zhao","affiliation":"Fuxi AI Lab , Netease"},{"name":"Runze Wu","slug":"runze-wu","given":"Runze","family":"Wu","affiliation":"Fuxi AI Lab , Netease"},{"name":"Ruocheng Guo","slug":"ruocheng-guo","given":"Ruocheng","family":"Guo","affiliation":"Bytedance AI Lab"}],"abstract":"Self-attention models have achieved state-of-the-art performance in sequential recommender systems by capturing the sequential dependencies among user-item interactions. However, they rely on positional embeddings to retain the sequential information, which may break the semantics of item embeddings. In addition, most existing works assume that such sequential dependencies exist solely in the item embeddings, but neglect their existence among the item features. In this work, we propose a novel sequential recommender system (MLP4Rec) based on the recent advances of MLP-based architectures, which is naturally sensitive to the order of items in a sequence. To be specific, we develop a tri-directional fusion scheme to coherently capture sequential, cross-channel and cross-feature correlations. Extensive experiments demonstrate the effectiveness of MLP4Rec over various representative baselines upon two benchmark datasets. The simple architecture of MLP4Rec also leads to the linear computational complexity as well as much fewer model parameters than existing self-attention methods.","doi":"10.24963/ijcai.2022/297","url":"https://ijcai.org/proceedings/2022/297","pdf":"https://ijcai.org/proceedings/2022/0297.pdf","markdown":"https://ijcai.org/proceedings/2022/297.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/297","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":297,"pages":{"first":2138,"last":2144},"keywords":["Data Mining: Recommender Systems"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987056","length_sec":81,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985199","length_sec":627,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Ba et al., 2016] Jimmy Lei Ba, Jamie Ryan Kiros, and Ge- offrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.","title":"Layer normalization","authors":"Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E Hinton","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cheng et al., 2016] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recommender systems. In Proc. of DLRS, 2016.","title":"Wide & deep learning for recommender systems","authors":"Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir","year":"2016","venue":"Proc. of DLRS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proc. of CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hidasi et al., 2015] Balázs Hidasi, Alexandros Karat- zoglou, Linas Baltrunas, and Domonkos Tikk. Session- based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939, 2015.","title":"Sessionbased recommendations with recurrent neural networks","authors":"Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, Domonkos Tikk","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hidasi et al., 2016] Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou, and Domonkos Tikk. Paral- lel recurrent neural network architectures for feature-rich session-based recommendations. In Proc. of RecSys, 2016.","title":"Parallel recurrent neural network architectures for feature-rich session-based recommendations","authors":"Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou, Domonkos Tikk","year":"2016","venue":"Proc. of RecSys","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kang and McAuley, 2018] Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In Proc. of ICDM, 2018.","title":"Self-attentive sequential recommendation","authors":"Wang-Cheng Kang, Julian Mcauley","year":"2018","venue":"Proc. of ICDM","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lee et al., 2021] Hojoon Lee, Dongyoon Hwang, Sungh- wan Hong, Changyeon Kim, Seungryong Kim, and Jaegul Choo. Moi-mixer: Improving mlp-mixer with multi order interactions in sequential recommendation. arXiv preprint arXiv:2108.07505, 2021.","title":"Moi-mixer: Improving mlp-mixer with multi order interactions in sequential recommendation","authors":"Lee","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2017] Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. Neural attentive session-based recommendation. In Proc. of CIKM, 2017.","title":"Neural attentive session-based recommendation","authors":"Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, Jun Ma","year":"2017","venue":"Proc. of CIKM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2018] Zhi Li, Hongke Zhao, Qi Liu, Zhenya Huang, Tao Mei, and Enhong Chen. Learning from his- tory and present: Next-item recommendation via discrimi- natively exploiting user behaviors. In Proc. of KDD, 2018.","title":"Learning from history and present: Next-item recommendation via discriminatively exploiting user behaviors","authors":"Zhi Li, Hongke Zhao, Qi Liu, Zhenya Huang, Tao Mei, Enhong Chen","year":"2018","venue":"Proc. of KDD","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2021] Hanxiao Liu, Zihang Dai, David R So, and Quoc V Le. Pay attention to mlps. arXiv preprint arXiv:2105.08050, 2021.","title":"Pay attention to mlps","authors":"Liu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Quadrana et al., 2017] Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. Personalizing session-based recommendations with hier- archical recurrent neural networks. In Proc. of RecSys, 2017.","title":"Personalizing session-based recommendations with hierarchical recurrent neural networks","authors":"Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi","year":"2017","venue":"Proc. of RecSys","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Rendle et al., 2009] Steffen Rendle, Christoph Freuden- thaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In Proc. of UAI, 2009.","title":"Bpr: Bayesian personalized ranking from implicit feedback","authors":"Rendle","year":"2009","venue":"Proc. of UAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rendle et al., 2010] Steffen Rendle, Christoph Freuden- thaler, and Lars Schmidt-Thieme. Factorizing personal- ized markov chains for next-basket recommendation. In Proc. of WWW, 2010.","title":"Factorizing personalized markov chains for next-basket recommendation","authors":"Steffen Rendle, Christoph Freudenthaler, Lars Schmidt-Thieme","year":"2010","venue":"Proc. of WWW","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sun et al., 2019] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequen- tial recommendation with bidirectional encoder represen- tations from transformer. In Proc. of CIKM, 2019.","title":"Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer","authors":"Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, Peng Jiang","year":"2019","venue":"Proc. of CIKM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tolstikhin et al., 2021] Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. In Proc. of NeurIPS, 2021.","title":"Mlp-mixer: An all-mlp architecture for vision","authors":"Neil Ilya O Tolstikhin, Alexander Houlsby, Lucas Kolesnikov, Xiaohua Beyer, Thomas Zhai, Jessica Unterthiner, Andreas Yung, Daniel Steiner, Jakob Keysers, Uszkoreit","year":"2021","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Touvron et al., 2021] Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Gautier Izacard, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek, et al. Resmlp: Feedforward net- works for image classification with data-efficient training. arXiv preprint arXiv:2105.03404, 2021.","title":"Resmlp: Feedforward networks for image classification with data-efficient training","authors":"Touvron","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proc. of NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yu et al., 2016] Feng Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. A dynamic recurrent model for next basket recommendation. In Proc. of SIGIR, 2016.","title":"A dynamic recurrent model for next basket recommendation","authors":"Feng Yu, Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan","year":"2016","venue":"Proc. of SIGIR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang et al., 2019] Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Deqing Wang, Guanfeng Liu, and Xiaofang Zhou. Feature-level deeper self-attention network for sequential recommendation. In Proc. of IJCAI, 2019.","title":"Feature-level deeper self-attention network for sequential recommendation","authors":"Tingting Zhang, Pengpeng Zhao, Yanchi Liu, S Victor, Jiajie Sheng, Deqing Xu, Guanfeng Wang, Xiaofang Liu, Zhou","year":"2019","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/600"},{"position":21,"raw":"[Zhao et al., 2018a] Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. Deep rein- forcement learning for page-wise recommendations. In Proc. of RecSys, 2018.","title":"Deep reinforcement learning for page-wise recommendations","authors":"Zhao","year":"2018","venue":"Proc. of RecSys","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhao et al., 2018b] Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, and Dawei Yin. Recom- mendations with negative feedback via pairwise deep re- inforcement learning. In Proc. of KDD, 2018.","title":"Recommendations with negative feedback via pairwise deep reinforcement learning","authors":"Zhao","year":"2018","venue":"Proc. of KDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhao et al., 2019] Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin. Deep reinforcement learning for search, recommendation, and online advertising: a survey. ACM SIGWEB Newsletter, 2019.","title":"Deep reinforcement learning for search, recommendation, and online advertising: a survey","authors":"Xiangyu Zhao, Long Xia, Jiliang Tang, Dawei Yin","year":"2019","venue":"ACM SIGWEB Newsletter","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2021] Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Yushuo Chen, Xingyu Pan, Kaiyuan Li, Yujie Lu, Hui Wang, Changxin Tian, et al. Recbole: To- wards a unified, comprehensive and efficient framework for recommendation algorithms. In Proc. of CIKM, 2021.","title":"Recbole: Towards a unified, comprehensive and efficient framework for recommendation algorithms","authors":"Shanlei Wayne Xin Zhao, Yupeng Mu, Zihan Hou, Yushuo Lin, Xingyu Chen, Kaiyuan Pan, Yujie Li, Hui Lu, Changxin Wang, Tian","year":"2021","venue":"Proc. of CIKM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zheng et al., 2021] Jianqiao Zheng, Sameera Ramasinghe, and Simon Lucey. Rethinking positional encoding. arXiv preprint arXiv:2107.02561, 2021.","title":"Rethinking positional encoding","authors":"Zheng","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}