{"type":"ScholarlyArticle","title":"Feature-level Deeper Self-Attention Network for Sequential Recommendation","authors":[{"name":"Tingting Zhang","slug":"tingting-zhang","given":"Tingting","family":"Zhang","affiliation":"Institute of AI , School of Computer Science and Technology , Soochow University , China; Zhejiang Lab , China"},{"name":"Pengpeng Zhao","slug":"pengpeng-zhao","given":"Pengpeng","family":"Zhao","affiliation":"Institute of AI , School of Computer Science and Technology , Soochow University , China; Zhejiang Lab , China"},{"name":"Yanchi Liu","slug":"yanchi-liu","given":"Yanchi","family":"Liu","affiliation":"Rutgers University , New Jersey , USA"},{"name":"Victor S. Sheng","slug":"victor-s-sheng","given":"Victor S.","family":"Sheng","affiliation":"University of Central Arkansas , Conway , USA"},{"name":"Jiajie Xu","slug":"jiajie-xu","given":"Jiajie","family":"Xu","affiliation":"Institute of AI , School of Computer Science and Technology , Soochow University , China"},{"name":"Deqing Wang","slug":"deqing-wang","given":"Deqing","family":"Wang","affiliation":"School of Computer , Beihang University , Beijing , China"},{"name":"Guanfeng Liu","slug":"guanfeng-liu","given":"Guanfeng","family":"Liu","affiliation":"Department of Computing , Macquarie University , Sydney , Australia"},{"name":"Xiaofang Zhou","slug":"xiaofang-zhou","given":"Xiaofang","family":"Zhou","affiliation":"The University of Queensland , Brisbane , Australia; Zhejiang Lab , China"}],"abstract":"Sequential recommendation, which aims to recommend next item that the user will likely interact in a near future, has become essential in various Internet applications. Existing methods usually consider the transition patterns between items, but ignore the transition patterns between features of items. We argue that only the item-level sequences cannot reveal the full sequential patterns, while explicit and implicit feature-level sequences can help extract the full sequential patterns. In this paper, we propose a novel method named Feature-level Deeper Self-Attention Network (FDSA) for sequential recommendation. Specifically, FDSA first integrates various heterogeneous features of items into feature sequences with different weights through a vanilla mechanism. After that, FDSA applies separated self-attention blocks on item-level sequences and feature-level sequences, respectively, to model item transition patterns and feature transition patterns. Then, we integrate the outputs of these two blocks to a fully-connected layer for next item recommendation. Finally, comprehensive experimental results demonstrate that considering the transition relationships between features can significantly improve the performance of sequential recommendation.","doi":"10.24963/ijcai.2019/600","url":"https://ijcai.org/proceedings/2019/600","pdf":"https://ijcai.org/proceedings/2019/0600.pdf","markdown":"https://ijcai.org/proceedings/2019/600.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/600","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":600,"pages":{"first":4320,"last":4326},"keywords":["Machine Learning: Recommender Systems","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Al-Rfou et al., 2019] Rami Al-Rfou, Dokook Choe, Noah Constant, Mandy Guo, and Llion Jones. Character-level language modeling with deeper self-attention. In AAAI, 2019.","title":"Character-level language modeling with deeper self-attention","authors":"Rami Al-Rfou, Dokook Choe, Noah Constant, Mandy Guo, Llion Jones","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2017] Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, and Tat-Seng Chua. Sca-cnn: Spatial and channel-wise attention in convo- lutional networks for image captioning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6298-6306. IEEE, 2017.","title":"Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning","authors":"Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, Tat-Seng Chua","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao. Syntax-directed atten- tion for neural machine translation. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.","title":"Syntax-directed attention for neural machine translation","authors":"Chen","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chung et al., 2014] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evalua- tion of gated recurrent neural networks on sequence mod- eling. CoRR, abs/1412.3555, 2014.","title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","authors":"Chung","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gehring et al., 2017] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. Convolu- tional sequence to sequence learning. arXiv preprint arX- iv:1705.03122, 2017.","title":"Convolutional sequence to sequence learning","authors":"Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N Dauphin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2017] Ruining He, Wang-Cheng Kang, and Ju- lian McAuley. Translation-based recommendation. In Proceedings of the Eleventh ACM Conference on Recom- mender Systems, pages 161-169. ACM, 2017.","title":"Translation-based recommendation","authors":"He","year":"2017","venue":"Proceedings of the Eleventh ACM Conference on Recommender Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2018] Xiangnan He, Zhankui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. Nais: Neural attentive item similarity model for recommenda- tion. IEEE Transactions on Knowledge and Data Engi- neering, 30(12):2354-2366, 2018.","title":"Nais: Neural attentive item similarity model for recommendation","authors":"He","year":"2018","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hidasi et al., 2016] Balázs Hidasi, Alexandros Karat- zoglou, Linas Baltrunas, and Domonkos Tikk. Session- based recommendations with recurrent neural networks. In ICLR, 2016.","title":"Sessionbased recommendations with recurrent neural networks","authors":"Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, Domonkos Tikk","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2018] Xiaowen Huang, Shengsheng Qian, Quan Fang, Jitao Sang, and Changsheng Xu. Csan: Con- textual self-attention network for user sequential recom- mendation. In 2018 ACM Multimedia Conference on Mul- timedia Conference, pages 447-455. ACM, 2018.","title":"Csan: Contextual self-attention network for user sequential recommendation","authors":"Xiaowen Huang, Shengsheng Qian, Quan Fang, Jitao Sang, Changsheng Xu","year":"2018","venue":"ACM Multimedia Conference on Multimedia Conference","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kang and McAuley, 2018] Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM), pages 197-206. IEEE, 2018.","title":"Self-attentive sequential recommendation","authors":"Wang-Cheng Kang, Julian Mcauley","year":"2018","venue":"2018 IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2019] Xiangpeng Li, Jingkuan Song, Lianli Gao, Xianglong Liu, Wenbing Huang, Xiangnan He, and Chuang Gan. Beyond rnns: Positional self-attention with co-attention for video question answering. In Thirty-Third AAAI Conference on Artificial Intelligence, 2019.","title":"Beyond rnns: Positional self-attention with co-attention for video question answering","authors":"Li","year":"2019","venue":"Thirty-Third AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Rendle et al., 2009] Steffen Rendle, Christoph Freuden- thaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the twenty-fifth conference on uncertain- ty in artificial intelligence, pages 452-461. AUAI Press, 2009.","title":"Bpr: Bayesian personalized ranking from implicit feedback","authors":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme","year":"2009","venue":"Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Rendle et al., 2010] Steffen Rendle, Christoph Freuden- thaler, and Lars Schmidt-Thieme. Factorizing personal- ized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web, pages 811-820. ACM, 2010.","title":"Factorizing personalized markov chains for next-basket recommendation","authors":"Steffen Rendle, Christoph Freudenthaler, Lars Schmidt-Thieme","year":"2010","venue":"Proceedings of the 19th international conference on World wide web","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Shen et al., 2018] Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. Disan: Di- rectional self-attention network for rnn/cnn-free language understanding. In Thirty-Second AAAI Conference on Ar- tificial Intelligence, 2018.","title":"Disan: Directional self-attention network for rnn/cnn-free language understanding","authors":"Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, Chengqi Zhang","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tang and Wang, 2018] Jiaxi Tang and Ke Wang. Person- alized top-n sequential recommendation via convolution- al sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 565-573. ACM, 2018.","title":"Personalized top-n sequential recommendation via convolutional sequence embedding","authors":"Wang ; ] Jiaxi Tang, Ke Tang, Wang","year":"2018","venue":"Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Yuan et al., 2019] Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. A simple convolutional generative network for next item rec- ommendation. In Proceedings of the Twelfth ACM Interna- tional Conference on Web Search and Data Mining, pages 582-590. ACM, 2019.","title":"A simple convolutional generative network for next item recommendation","authors":"Yuan","year":"2019","venue":"Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhao et al., 2019] Pengpeng Zhao, Haifeng Zhu, Yanchi Li- u, Zhixu Li, Jiajie Xu, and Victor S Sheng. Where to go next: A spatio-temporal lstm model for next poi recom- mendation. In AAAI, 2019.","title":"Where to go next: A spatio-temporal lstm model for next poi recommendation","authors":"Zhao","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhou et al., 2018] Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, and Jun Gao. Atrank: An attention-based user behavior modeling frame- work for recommendation. In Thirty-Second AAAI Confer- ence on Artificial Intelligence, 2018.","title":"Atrank: An attention-based user behavior modeling framework for recommendation","authors":"Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, Jun Gao","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}