{"type":"ScholarlyArticle","title":"Representation Learning-Assisted Click-Through Rate Prediction","authors":[{"name":"Wentao Ouyang","slug":"wentao-ouyang","given":"Wentao","family":"Ouyang","affiliation":""},{"name":"Xiuwu Zhang","slug":"xiuwu-zhang","given":"Xiuwu","family":"Zhang","affiliation":""},{"name":"Shukui Ren","slug":"shukui-ren","given":"Shukui","family":"Ren","affiliation":""},{"name":"Chao Qi","slug":"chao-qi","given":"Chao","family":"Qi","affiliation":""},{"name":"Zhaojie Liu","slug":"zhaojie-liu","given":"Zhaojie","family":"Liu","affiliation":""},{"name":"Yanlong Du","slug":"yanlong-du","given":"Yanlong","family":"Du","affiliation":""}],"abstract":"Click-through rate (CTR) prediction is a critical task in online advertising systems. Most existing methods mainly model the feature-CTR relationship and suffer from the data sparsity issue. In this paper, we propose DeepMCP, which models other types of relationships in order to learn more informative and statistically reliable feature representations, and in consequence to improve the performance of CTR prediction. In particular, DeepMCP contains three parts: a matching subnet, a correlation subnet and a prediction subnet. These subnets model the user-ad, ad-ad and feature-CTR relationship respectively. When these subnets are jointly optimized under the supervision of the target labels, the learned feature representations have both good prediction powers and good representation abilities. Experiments on two large-scale datasets demonstrate that DeepMCP outperforms several state-of-the-art models for CTR prediction.","doi":"10.24963/ijcai.2019/634","url":"https://ijcai.org/proceedings/2019/634","pdf":"https://ijcai.org/proceedings/2019/0634.pdf","markdown":"https://ijcai.org/proceedings/2019/634.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/634","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning Applications","track":"Main track","number":634,"pages":{"first":4561,"last":4567},"keywords":["Machine Learning Applications: Other Applications","Machine Learning Applications: Applications of Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Chapelle et al., 2015] Olivier Chapelle, Eren Manavoglu, and Romer Rosales. Simple and scalable response pre- diction for display advertising. ACM Transactions on In- telligent Systems and Technology, 5(4):61, 2015.","title":"Simple and scalable response prediction for display advertising","authors":"Chapelle","year":"2015","venue":"ACM Transactions on Intelligent Systems and Technology","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cheng et al., 2016] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recommender systems. In DLRS, pages 7-10. ACM, 2016.","title":"Wide & deep learning for recommender systems","authors":"Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir","year":"2016","venue":"DLRS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Covington et al., 2016] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recom- mendations. In RecSys, pages 191-198. ACM, 2016.","title":"Deep neural networks for youtube recommendations","authors":"Paul Covington, Jay Adams, Emre Sargin","year":"2016","venue":"RecSys","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Duchi et al., 2011] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(Jul):2121-2159, 2011.","title":"Adaptive subgradient methods for online learning and stochastic optimization","authors":"John Duchi, Elad Hazan, Yoram Singer","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gao et al., 2018] Li Gao, Hong Yang, Jia Wu, Chuan Zhou, Weixue Lu, and Yue Hu. Recommendation with multi- source heterogeneous information. In IJCAI, 2018.","title":"Recommendation with multisource heterogeneous information","authors":"Li Gao, Hong Yang, Jia Wu, Chuan Zhou, Weixue Lu, Yue Hu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gong et al., 2019] Yu Gong, Xusheng Luo, Yu Zhu, Wenwu Ou, Zhao Li, Muhua Zhu, Kenny Q Zhu, Lu Duan, and Xi Chen. Deep cascade multi-task learning for slot filling in online shopping assistant. In AAAI, 2019.","title":"Deep cascade multi-task learning for slot filling in online shopping assistant","authors":"Yu Gong, Xusheng Luo, Yu Zhu, Wenwu Ou, Zhao Li, Muhua Zhu, Lu Kenny Q Zhu, Xi Duan, Chen","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guo et al., 2017] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: a factorization-machine based neural network for ctr predic- tion. In IJCAI, pages 1725-1731, 2017.","title":"Deepfm: a factorization-machine based neural network for ctr prediction","authors":"Guo","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/239"},{"position":8,"raw":"[He and Chua, 2017] Xiangnan He and Tat-Seng Chua. Neu- ral factorization machines for sparse predictive analytics. In SIGIR, pages 355-364. ACM, 2017.","title":"Xiangnan He and Tat-Seng Chua. Neural factorization machines for sparse predictive analytics","authors":"Chua He","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2014] Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et al. Practical lessons from pre- dicting clicks on ads at facebook. In ADKDD, pages 1-9. ACM, 2014.","title":"Practical lessons from predicting clicks on ads at facebook","authors":"Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers","year":"2014","venue":"ADKDD","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2013] Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using click- through data. In CIKM, pages 2333-2338. ACM, 2013.","title":"Learning deep structured semantic models for web search using clickthrough data","authors":"Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, Larry Heck","year":"2013","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2018] Jizhou Huang, Wei Zhang, Yaming Sun, Haifeng Wang, and Ting Liu. Improving entity rec- ommendation with search log and multi-task learning. In IJCAI, pages 4107-4114, 2018.","title":"Improving entity recommendation with search log and multi-task learning","authors":"Jizhou Huang, Wei Zhang, Yaming Sun, Haifeng Wang, Ting Liu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/571"},{"position":12,"raw":"[Juan et al., 2016] Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. Field-aware factorization ma- chines for ctr prediction. In RecSys, pages 43-50. ACM, 2016.","title":"Field-aware factorization machines for ctr prediction","authors":"Juan","year":"2016","venue":"RecSys","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Koren et al., 2009] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recom- mender systems. Computer, (8):30-37, 2009.","title":"Matrix factorization techniques for recommender systems","authors":"Koren","year":"2009","venue":"Computer","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- offrey Hinton. Deep learning. nature, 521(7553):436, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu and others, 2009] Tie-Yan Liu et al. Learning to rank for information retrieval. Foundations and Trends R in In- formation Retrieval, 3(3):225-331, 2009.","title":"Learning to rank for information retrieval","authors":"Others ; Tie-Yan Liu, Liu","year":"2009","venue":"Foundations and Trends R in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[McMahan et al., 2013] H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. Ad click prediction: a view from the trenches. In KDD, pages 1222-1230. ACM, 2013.","title":"Ad click prediction: a view from the trenches","authors":"Gary H Brendan Mcmahan, David Holt, Michael Sculley, Dietmar Young, Julian Ebner, Lan Grady, Todd Nie, Eugene Phillips, Davydov, Daniel Golovin","year":"2013","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed rep- resentations of words and phrases and their composition- ality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nair and Hinton, 2010] Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann ma- chines. In ICML, pages 807-814, 2010.","title":"Rectified linear units improve restricted boltzmann machines","authors":"Hinton Nair, Vinod Nair, Geoffrey E Hinton","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pan et al., 2018] Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. Field-weighted factorization machines for click-through rate prediction in display advertising. In WWW, pages 1349-1357. International World Wide Web Conferences Steering Committee, 2018.","title":"Field-weighted factorization machines for click-through rate prediction in display advertising","authors":"Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, Quan Lu","year":"2018","venue":"International World Wide Web Conferences Steering Committee","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Qu et al., 2016] Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. Product-based neural networks for user response prediction. In ICDM, pages 1149-1154. IEEE, 2016.","title":"Product-based neural networks for user response prediction","authors":"Qu","year":"2016","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rendle, 2010] Steffen Rendle. Factorization machines. In ICDM, pages 995-1000. IEEE, 2010.","title":"Factorization machines","authors":"Steffen Rendle","year":"2010","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rendle, 2012] Steffen Rendle. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Tech- nology, 3(3):57, 2012.","title":"Factorization machines with libfm","authors":"Steffen Rendle","year":"2012","venue":"ACM Transactions on Intelligent Systems and Technology","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Richardson et al., 2007] Matthew Richardson, Ewa Domi- nowska, and Robert Ragno. Predicting clicks: estimating the click-through rate for new ads. In WWW, pages 521- 530. ACM, 2007.","title":"Predicting clicks: estimating the click-through rate for new ads","authors":"Matthew Richardson, Ewa Dominowska, Robert Ragno","year":"2007","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shan et al., 2016] Ying Shan, T Ryan Hoens, Jian Jiao, Hai- jing Wang, Dong Yu, and JC Mao. Deep crossing: Web- scale modeling without manually crafted combinatorial features. In KDD, pages 255-262. ACM, 2016.","title":"Deep crossing: Webscale modeling without manually crafted combinatorial features","authors":"Ying Shan, Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, J C Mao","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Re- search, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Van den Oord et al., 2013] Aaron Van den Oord, Sander Dieleman, and Benjamin Schrauwen. Deep content-based music recommendation. In NIPS, pages 2643-2651, 2013.","title":"Deep content-based music recommendation","authors":"Oord Van Den","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2017] Ruoxi Wang, Bin Fu, Gang Fu, and Min- gliang Wang. Deep & cross network for ad click predic- tions. In ADKDD, page 12. ACM, 2017.","title":"Deep & cross network for ad click predictions","authors":"Wang","year":"2017","venue":"ADKDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2016a] Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. Collaborative knowledge base embedding for recommender systems. In KDD, pages 353-362. ACM, 2016.","title":"Collaborative knowledge base embedding for recommender systems","authors":"Zhang","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2016b] Weinan Zhang, Tianming Du, and Jun Wang. Deep learning over multi-field categorical data. In ECIR, pages 45-57. Springer, 2016.","title":"Deep learning over multi-field categorical data","authors":"Zhang","year":"2016","venue":"ECIR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhao et al., 2018] Kui Zhao, Yuechuan Li, Zhaoqian Shuai, and Cheng Yang. Learning and transferring ids represen- tation in e-commerce. In KDD, pages 1031-1039. ACM, 2018.","title":"Learning and transferring ids representation in e-commerce","authors":"Kui Zhao, Yuechuan Li, Zhaoqian Shuai, Cheng Yang","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhou et al., 2018] Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click- through rate prediction. In KDD, pages 1059-1068. ACM, 2018.","title":"Deep interest network for clickthrough rate prediction","authors":"Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, Kun Gai","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}