{"type":"ScholarlyArticle","title":"Convolutional Neural Networks based Click-Through Rate Prediction with Multiple Feature Sequences","authors":[{"name":"Patrick P. K. Chan","slug":"patrick-p-k-chan","given":"Patrick P. K.","family":"Chan","affiliation":"School of Computer Science and Engineering , South China University of Technology , China"},{"name":"Xian Hu","slug":"xian-hu","given":"Xian","family":"Hu","affiliation":"School of Computer Science and Engineering , South China University of Technology , China; Tencent , China"},{"name":"Lili Zhao","slug":"lili-zhao","given":"Lili","family":"Zhao","affiliation":"Tencent , China"},{"name":"Daniel S. Yeung","slug":"daniel-s-yeung","given":"Daniel S.","family":"Yeung","affiliation":"School of Computer Science and Engineering , South China University of Technology , China"},{"name":"Dapeng Liu","slug":"dapeng-liu","given":"Dapeng","family":"Liu","affiliation":"Tencent , China"},{"name":"Lei Xiao","slug":"lei-xiao","given":"Lei","family":"Xiao","affiliation":"Tencent , China"}],"abstract":"Convolutional Neural Network (CNN) achieved satisfying performance in click-through rate (CTR) prediction in recent studies. Since features used in CTR prediction have no meaningful sequence in nature, the features can be arranged in any order. As CNN learns the local information of a sample, the feature sequence may influence its performance significantly. However, this problem has not been fully investigated. This paper firstly investigates whether and how the feature sequence affects the performance of the CNN-based CTR prediction method. As the data distribution of CTR prediction changes with time, the best current sequence may not be suitable for future data. Two multi-sequence models are proposed to learn the information provided by different sequences. The first model learns all sequences using a single feature learning module, while each sequence is learnt individually by a feature learning module in the second one. Moreover, a method of generating a set of embedding sequences which aims to consider the combined influence of all feature pairs on feature learning is also introduced. The experiments are conducted to demonstrate the effectiveness and stability of our proposed models in the offline and online environment on both the benchmark Avazu dataset and a real commercial dataset.","doi":"10.24963/ijcai.2018/277","url":"https://ijcai.org/proceedings/2018/277","pdf":"https://ijcai.org/proceedings/2018/0277.pdf","markdown":"https://ijcai.org/proceedings/2018/277.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/277","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":277,"pages":{"first":2007,"last":2013},"keywords":["Machine Learning: Neural Networks","Machine Learning: Deep Learning","Machine Learning: Classification","Machine Learning: Learning Preferences or Rankings"],"videos":[],"references":[{"position":1,"raw":"[Calders and Jaroszewicz, 2007] Toon Calders and Szymon Jaroszewicz. Efficient auc optimization for classification. In European Conference on Principles and Practice of Knowledge Discovery in Databases, pages 42-53, 2007.","title":"Efficient auc optimization for classification","authors":"Jaroszewicz Calders, Toon Calders, Szymon Jaroszewicz","year":"2007","venue":"European Conference on Principles and Practice of Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2016] Junxuan Chen, Baigui Sun, Hao Li, Hongtao Lu, and Xian-Sheng Hua. Deep ctr prediction in display advertising. In Proceedings of the 2016 ACM on Multimedia Conference, pages 811-820. ACM, 2016.","title":"Deep ctr prediction in display advertising","authors":"Chen","year":"2016","venue":"Proceedings of the 2016 ACM on Multimedia Conference","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2017] Yue Deng, Yilin Shen, Hongxia Jin, Yue Deng, Yilin Shen, Hongxia Jin, Yue Deng, Yilin Shen, and Hongxia Jin. Disguise adversarial networks for click- through rate prediction. In Twenty-Sixth International Joint Conference on Artificial Intelligence, pages 1589- 1595, 2017.","title":"Disguise adversarial networks for clickthrough rate prediction","authors":"Yue Deng, Yilin Shen, Hongxia Jin, Yue Deng, Yilin Shen, Hongxia Jin, Yue Deng, Yilin Shen, Hongxia Jin","year":"2017","venue":"Twenty-Sixth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Edizel et al., 2017] Bora Edizel, Amin Mantrach, and Xiao Bai. Deep character-level click-through rate prediction for sponsored search. arXiv preprint arXiv:1707.02158, 2017.","title":"Deep character-level click-through rate prediction for sponsored search","authors":"Edizel","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guo et al., 2017] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: A factorization-machine based neural network for ctr predic- tion. arXiv preprint arXiv:1703.04247, 2017.","title":"Deepfm: A factorization-machine based neural network for ctr prediction","authors":"Guo","year":"2017","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/239"},{"position":6,"raw":"[He et al., 2014] Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et al. Practical lessons from pre- dicting clicks on ads at facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising, pages 1-9. ACM, 2014.","title":"Practical lessons from predicting clicks on ads at facebook","authors":"Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers","year":"2014","venue":"Proceedings of the Eighth International Workshop on Data Mining for Online Advertising","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Juan et al., 2016] Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. Field-aware factorization ma- chines for ctr prediction. In Proceedings of the 10th ACM Conference on Recommender Systems, pages 43-50. ACM, 2016.","title":"Field-aware factorization machines for ctr prediction","authors":"Juan","year":"2016","venue":"Proceedings of the 10th ACM Conference on Recommender Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Computer Science, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2015] Siqin Li, Lei Lin, and Chengjie Sun. Click- through rate prediction for search advertising based on convolution neural network. Intelligent Computer and Ap- plications, 5:007, 2015.","title":"Clickthrough rate prediction for search advertising based on convolution neural network","authors":"Li","year":"2015","venue":"Intelligent Computer and Applications","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2015] Qiang Liu, Feng Yu, Shu Wu, and Liang Wang. A convolutional click prediction model. In Pro- ceedings of the 24th ACM International on Conference on Information and Knowledge Management, pages 1743- 1746. ACM, 2015.","title":"A convolutional click prediction model","authors":"Qiang Liu, Feng Yu, Shu Wu, Liang Wang","year":"2015","venue":"Proceedings of the 24th ACM International on Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2017] Xun Liu, Wei Xue, Lei Xiao, and Bo Zhang. Pbodl: Parallel bayesian online deep learn- ing for click-through rate prediction in tencent advertising system. arXiv preprint arXiv:1707.00802, 2017.","title":"Pbodl: Parallel bayesian online deep learning for click-through rate prediction in tencent advertising system","authors":"Xun Liu, Wei Xue, Lei Xiao, Bo Zhang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[McMahan et al., 2013] H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. Ad click prediction: a view from the trench- es. In Proceedings of the 19th ACM SIGKDD internation- al conference on Knowledge discovery and data mining, pages 1222-1230. ACM, 2013.","title":"Ad click prediction: a view from the trenches","authors":"Gary H Brendan Mcmahan, David Holt, Michael Sculley, Dietmar Young, Julian Ebner, Lan Grady, Todd Nie, Eugene Phillips, Davydov, Daniel Golovin","year":"2013","venue":"Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rendle, 2010] Steffen Rendle. Factorization machines. In Data Mining (ICDM), 2010 IEEE 10th International Con- ference on, pages 995-1000. IEEE, 2010.","title":"Factorization machines","authors":"Steffen Rendle","year":"2010","venue":"IEEE 10th International Conference on","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shan et al., 2016] Ying Shan, T Ryan Hoens, Jian Jiao, Hai- jing Wang, Dong Yu, and JC Mao. Deep crossing: Web- scale modeling without manually crafted combinatorial features. In Proceedings of the 22nd ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, pages 255-262. ACM, 2016.","title":"Deep crossing: Webscale modeling without manually crafted combinatorial features","authors":"Ying Shan, Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, J C Mao","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2016] Peng Wang, Bo Xu, Jiaming Xu, Guan- hua Tian, Cheng Lin Liu, and Hongwei Hao. Semantic expansion using word embedding clustering and convolu- tional neural network for improving short text classifica- tion. Neurocomputing, 174(PB):806-814, 2016.","title":"Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification","authors":"Peng Wang, Bo Xu, Jiaming Xu, Guanhua Tian, Cheng, Lin Liu, Hongwei Hao","year":"2016","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2017a] Jin Wang, Zhongyuan Wang, Dawei Zhang, and Jun Yan. Combining knowledge with deep convolutional neural networks for short text classification. In Twenty-Sixth International Joint Conference on Artifi- cial Intelligence, pages 2915-2921, 2017.","title":"Combining knowledge with deep convolutional neural networks for short text classification","authors":"Wang","year":"2017","venue":"Twenty-Sixth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/406"},{"position":18,"raw":"[Wang et al., 2017b] Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click pre- dictions. arXiv preprint arXiv:1708.05123, 2017.","title":"Deep & cross network for ad click predictions","authors":"Wang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yan et al., 2014] Ling Yan, Wujun Li, Guirong Xue, and D- ingyi Han. Coupled group lasso for web-scale ctr predic- tion in display advertising. In International Conference on Machine Learning, pages 802-810, 2014.","title":"Coupled group lasso for web-scale ctr prediction in display advertising","authors":"Ling Yan, Wujun Li, Guirong Xue, Dingyi Han","year":"2014","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yang et al., 2017] Jufeng Yang, Dongyu She, and Ming Sun. Joint image emotion classification and distribution learning via deep convolutional neural network. In Twenty- Sixth International Joint Conference on Artificial Intelli- gence, pages 3266-3272, 2017.","title":"Joint image emotion classification and distribution learning via deep convolutional neural network","authors":"Yang","year":"2017","venue":"Twenty-Sixth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/456"},{"position":21,"raw":"[Zhang et al., 2015] Xiang Zhang, Junbo Zhao, and Yann Lecun. Character-level convolutional networks for text classification. pages 649-657, 2015.","title":"Character-level convolutional networks for text classification","authors":"Xiang Zhang, Junbo Zhao, Yann Lecun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhou et al., 2017] Guorui Zhou, Chengru Song, Xiaoqiang Zhu, Xiao Ma, Yanghui Yan, Xingya Dai, Han Zhu, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click- through rate prediction. 2017.","title":"Deep interest network for clickthrough rate prediction","authors":"Zhou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}