{"type":"ScholarlyArticle","title":"Internal and Contextual Attention Network for Cold-start Multi-channel Matching in Recommendation","authors":[{"name":"Ruobing Xie","slug":"ruobing-xie","given":"Ruobing","family":"Xie","affiliation":"WeChat Search Application Department , Tencent , China"},{"name":"Zhijie Qiu","slug":"zhijie-qiu","given":"Zhijie","family":"Qiu","affiliation":"WeChat Search Application Department , Tencent , China"},{"name":"Jun Rao","slug":"jun-rao","given":"Jun","family":"Rao","affiliation":"WeChat Search Application Department , Tencent , China"},{"name":"Yi Liu","slug":"yi-liu","given":"Yi","family":"Liu","affiliation":"WeChat Search Application Department , Tencent , China"},{"name":"Bo Zhang","slug":"bo-zhang","given":"Bo","family":"Zhang","affiliation":"WeChat Search Application Department , Tencent , China"},{"name":"Leyu Lin","slug":"leyu-lin","given":"Leyu","family":"Lin","affiliation":"WeChat Search Application Department , Tencent , China"}],"abstract":"Real-world integrated personalized recommendation systems usually deal with millions of heterogeneous items. It is extremely challenging to conduct full corpus retrieval with complicated models due to the tremendous computation costs. Hence, most large-scale recommendation systems consist of two modules: a multi-channel matching module to efficiently retrieve a small subset of candidates, and a ranking module for precise personalized recommendation. However, multi-channel matching usually suffers from cold-start problems when adding new channels or new data sources. To solve this issue, we propose a novel Internal and contextual attention network (ICAN), which highlights channel-specific contextual information and feature field interactions between multiple channels. In experiments, we conduct both offline and online evaluations with case studies on a real-world integrated recommendation system. The significant improvements confirm the effectiveness and robustness of ICAN, especially for cold-start channels. Currently, ICAN has been deployed on WeChat Top Stories used by millions of users. The source code can be obtained from https://github.com/zhijieqiu/ICAN.","doi":"10.24963/ijcai.2020/379","url":"https://ijcai.org/proceedings/2020/379","pdf":"https://ijcai.org/proceedings/2020/0379.pdf","markdown":"https://ijcai.org/proceedings/2020/379.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/379","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":379,"pages":{"first":2732,"last":2738},"keywords":["Machine Learning: Recommender Systems"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24089/file_24089.m3u8","length_sec":299,"provider":"hls"}],"references":[{"position":1,"raw":"[Breese et al., 1998] John S Breese, David Heckerman, and Carl Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of UAI, 1998.","title":"Empirical analysis of predictive algorithms for collaborative filtering","authors":"David John S Breese, Carl Heckerman, Kadie","year":"1998","venue":"Proceedings of UAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cheng et al., 2016] Heng-Tze Cheng, Levent Koc, Jeremi- ah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems, 2016.","title":"Wide & deep learning for recommender systems","authors":"Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir","year":"2016","venue":"Proceedings of the 1st workshop on deep learning for recommender systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Covington et al., 2016] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recom- mendations. In Proceedings of RecSys, 2016.","title":"Deep neural networks for youtube recommendations","authors":"Paul Covington, Jay Adams, Emre Sargin","year":"2016","venue":"Proceedings of RecSys","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deldjoo et al., 2019] Yashar Deldjoo, Maurizio Ferrar- i Dacrema, Mihai Gabriel Constantin, Hamid Eghbal- Zadeh, Stefano Cereda, Markus Schedl, Bogdan Ionescu, and Paolo Cremonesi. Movie genome: alleviating new item cold start in movie recommendation. User Modeling and User-Adapted Interaction, 2019.","title":"Movie genome: alleviating new item cold start in movie recommendation","authors":"Yashar Deldjoo, Maurizio Ferrari Dacrema, Mihai Gabriel Constantin, Hamid Eghbal-Zadeh, Stefano Cereda, Markus Schedl, Bogdan Ionescu, Paolo Cremonesi","year":"2019","venue":"User Modeling and User-Adapted Interaction","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guo et al., 2017] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: a factorization-machine based neural network for ctr predic- tion. In Proceedings of IJCAI, 2017.","title":"Deepfm: a factorization-machine based neural network for ctr prediction","authors":"Guo","year":"2017","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/239"},{"position":6,"raw":"[He and Chua, 2017] Xiangnan He and Tat-Seng Chua. Neu- ral factorization machines for sparse predictive analytics. In Proceedings of SIGIR, 2017.","title":"Xiangnan He and Tat-Seng Chua. Neural factorization machines for sparse predictive analytics","authors":"Chua He","year":"2017","venue":"Proceedings of SIGIR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2018] Xiaowen Huang, Shengsheng Qian, Quan Fang, Jitao Sang, and Changsheng Xu. Csan: Con- textual self-attention network for user sequential recom- mendation. In Proceedings of ACMMM, 2018.","title":"Csan: Contextual self-attention network for user sequential recommendation","authors":"Xiaowen Huang, Shengsheng Qian, Quan Fang, Jitao Sang, Changsheng Xu","year":"2018","venue":"Proceedings of ACMMM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Johnson et al., 2019] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 2019.","title":"Billion-scale similarity search with gpus","authors":"Jeff Johnson, Matthijs Douze, Hervé Jégou","year":"2019","venue":"IEEE Transactions on Big Data","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Juan et al., 2016] Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. Field-aware factorization ma- chines for ctr prediction. In Proceedings of RecSys, 2016.","title":"Field-aware factorization machines for ctr prediction","authors":"Juan","year":"2016","venue":"Proceedings of RecSys","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khribi et al., 2008] Mohamed Koutheaïr Khribi, Mohamed Jemni, and Olfa Nasraoui. Automatic recommendations for e-learning personalization based on web usage mining techniques and information retrieval. In 2008 Eighth IEEE International Conference on Advanced Learning Tech- nologies, 2008.","title":"Automatic recommendations for e-learning personalization based on web usage mining techniques and information retrieval","authors":"Mohamed Koutheaïr Khribi, Mohamed Jemni, Olfa Nasraoui","year":"2008","venue":"Eighth IEEE International Conference on Advanced Learning Technologies","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Koren et al., 2009] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recom- mender systems. Computer, 2009.","title":"Matrix factorization techniques for recommender systems","authors":"Koren","year":"2009","venue":"Computer","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lian et al., 2018] Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. xdeepfm: Combining explicit and implicit feature interac- tions for recommender systems. In Proceedings of KDD, 2018.","title":"xdeepfm: Combining explicit and implicit feature interactions for recommender systems","authors":"Lian","year":"2018","venue":"Proceedings of KDD","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Okura et al., 2017] Shumpei Okura, Yukihiro Tagami, Shin- go Ono, and Akira Tajima. Embedding-based news recom- mendation for millions of users. In Proceedings of KDD, 2017. [Perozzi et al., 2014]","title":"Embedding-based news recommendation for millions of users","authors":"Okura","year":"2014","venue":"Proceedings of KDD","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In Proceedings of KDD, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"Proceedings of KDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rendle, 2010] Steffen Rendle. Factorization machines. In Proceedings of ICDM, 2010.","title":"Factorization machines","authors":"Steffen Rendle","year":"2010","venue":"Proceedings of ICDM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sarwar et al., 2001] Badrul Munir Sarwar, George Karypis, Joseph A Konstan, John Riedl, et al. Item-based collabora- tive filtering recommendation algorithms. In Proceedings of WWW, 2001.","title":"Item-based collaborative filtering recommendation algorithms","authors":"Munir Badrul, George Sarwar, Joseph A Karypis, John Konstan, Riedl","year":"2001","venue":"Proceedings of WWW","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Schein et al., 2002] Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. Methods and met- rics for cold-start recommendations. In Proceedings of SI- GIR, 2002.","title":"Methods and metrics for cold-start recommendations","authors":"Alexandrin Andrew I Schein, Lyle H Popescul, David M Ungar, Pennock","year":"2002","venue":"Proceedings of SI-GIR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Song et al., 2019] Weiping Song, Chence Shi, Zhiping Xi- ao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. Autoint: Automatic feature interaction learning via self- attentive neural networks. In Proceedings of CIKM, 2019.","title":"Autoint: Automatic feature interaction learning via selfattentive neural networks","authors":"Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, Jian Tang","year":"2019","venue":"Proceedings of CIKM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of NIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2017] Ruoxi Wang, Bin Fu, Gang Fu, and Min- gliang Wang. Deep & cross network for ad click predic- tions. In Proceedings of ADKDD, 2017.","title":"Deep & cross network for ad click predictions","authors":"Wang","year":"2017","venue":"Proceedings of ADKDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2018] Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. Billion- scale commodity embedding for e-commerce recommen- dation in alibaba. In Proceedings of KDD, 2018.","title":"Billionscale commodity embedding for e-commerce recommendation in alibaba","authors":"Wang","year":"2018","venue":"Proceedings of KDD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xiao et al., 2017] Jun Xiao, Hao Ye, Xiangnan He, Han- wang Zhang, Fei Wu, and Tat-Seng Chua. Attentional factorization machines: Learning the weight of feature in- teractions via attention networks. In Proceedings of IJCAI, 2017.","title":"Attentional factorization machines: Learning the weight of feature interactions via attention networks","authors":"Jun Xiao, Xiangnan Hao Ye, Hanwang He, Fei Zhang, Tat-Seng Wu, Chua","year":"2017","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/435"},{"position":23,"raw":"[Zhou et al., 2018a] Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, and Jun Gao. Atrank: An attention-based user behavior modeling frame- work for recommendation. In Proceedings of AAAI, 2018.","title":"Atrank: An attention-based user behavior modeling framework for recommendation","authors":"Zhou","year":"2018","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2018b] Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click- through rate prediction. In Proceedings of KDD, 2018.","title":"Deep interest network for clickthrough rate prediction","authors":"Zhou","year":"2018","venue":"Proceedings of KDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2019] Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. Deep interest evolution network for click-through rate pre- diction. In Proceedings of AAAI, 2019.","title":"Deep interest evolution network for click-through rate prediction","authors":"Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, Kun Gai","year":"2019","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2018] Han Zhu, Xiang Li, Pengye Zhang, Guozheng Li, Jie He, Han Li, and Kun Gai. Learning tree- based deep model for recommender systems. In Proceed- ings of KDD, 2018.","title":"Learning treebased deep model for recommender systems","authors":"Han Zhu, Xiang Li, Pengye Zhang, Guozheng Li, Jie He, Han Li, Kun Gai","year":"2018","venue":"Proceedings of KDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhu et al., 2019] Han Zhu, Daqing Chang, Ziru Xu, Pengye Zhang, Xiang Li, Jie He, Han Li, Jian Xu, and Kun Gai. Joint optimization of tree-based index and deep model for recommender systems. In Proceedings of NIPS, 2019.","title":"Joint optimization of tree-based index and deep model for recommender systems","authors":"Han Zhu, Daqing Chang, Ziru Xu, Pengye Zhang, Xiang Li, Jie He, Han Li, Jian Xu, Kun Gai","year":"2019","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}