{"type":"ScholarlyArticle","title":"InteractionNN: A Neural Network for Learning Hidden Features in Sparse Prediction","authors":[{"name":"Xiaowang Zhang","slug":"xiaowang-zhang","given":"Xiaowang","family":"Zhang","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin 300350 , China"},{"name":"Qiang Gao","slug":"qiang-gao","given":"Qiang","family":"Gao","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin 300350 , China; Tianjin Key Laboratory of Cognitive Computing and Application , Tianjin 300350 , China"},{"name":"Zhiyong Feng","slug":"zhiyong-feng","given":"Zhiyong","family":"Feng","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin 300350 , China"}],"abstract":"In this paper, we present a neural network (InteractionNN) for sparse predictive analysis where hidden features of sparse data can be learned by multilevel feature interaction. To characterize multilevel interaction of features, InteractionNN consists of three modules, namely, nonlinear interaction pooling, layer-lossing, and embedding. Nonlinear interaction pooling (NI pooling) is a hierarchical structure and, by shortcut connection, constructs low-level feature interactions from basic dense features to elementary features. Layer-lossing is a feed-forward neural network where high-level feature interactions can be learned from low-level feature interactions via correlation of all layers with target. Moreover, embedding is to extract basic dense features from sparse features of data which can help in reducing our proposed model computational complex. Finally, our experiment evaluates on the two benchmark datasets and the experimental results show that InteractionNN performs better than most of state-of-the-art models in sparse regression.","doi":"10.24963/ijcai.2019/602","url":"https://ijcai.org/proceedings/2019/602","pdf":"https://ijcai.org/proceedings/2019/0602.pdf","markdown":"https://ijcai.org/proceedings/2019/602.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/602","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":602,"pages":{"first":4334,"last":4340},"keywords":["Machine Learning: Deep Learning","Machine Learning: Feature Selection ; Learning Sparse Models"],"videos":[],"references":[{"position":1,"raw":"[Bayer et al., 2017] Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle (2017). A generic coordi- nate descent framework for learning from implicit feed- back. In Proc. of WWW'17, pp.1341-1350.","title":"A generic coordinate descent framework for learning from implicit feedback","authors":"Bayer","year":"2017","venue":"Proc. of WWW'17","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2016] Junxuan Chen, Baigui Sun, Hao Li, Hongtao Lu, and Xian-Sheng Hua (2016). Deep ctr prediction in display advertising. In Proc. of MM'16, pp.811-820.","title":"Deep ctr prediction in display advertising","authors":"Chen","year":"2016","venue":"Proc. of MM'16","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng et al., 2010] Haibin Cheng and Erick Cantú-Paz (2010). Personalized click prediction in sponsored search. In Proc. of WSDM'10, pp.351-360.","title":"Personalized click prediction in sponsored search","authors":"Haibin Cheng, Erick Cantú-Paz","year":"2010","venue":"Proc. of WSDM'10","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cheng et al., 2016] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah (2016). Wide & deep learning for recommender systems. In Proc. of RecSys'16, pp.7-10.","title":"Wide & deep learning for recommender systems","authors":"Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah","year":"2016","venue":"Proc. of RecSys'16","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guo et al., 2017] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He (2017). DeepFM: A Factorization-machine based neural network for CTR prediction. In Proc. of IJCAI'17, pp:1725-1731.","title":"DeepFM: A Factorization-machine based neural network for CTR prediction","authors":"Guo","year":"2017","venue":"Proc. of IJCAI'17","doi":null,"url":null,"internal_url":"/proceedings/2017/239"},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun (2016). Deep residual learning for image recognition. In Proc. of CVPR'16, pp.770-778.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proc. of CVPR'16","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua (2016). Fast matrix factoriza- tion for online recommendation with implicit feedback. In Proc. of SIGIR'16, pp.549-558.","title":"Fast matrix factorization for online recommendation with implicit feedback","authors":"He","year":"2016","venue":"Proc. of SIGIR'16","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2017] Xiangnan He and Tat-Seng Chua (2017). Neural factorization machines for sparse predictive an- alytics. In Proc. of SIGIR'17, pp.355-364.","title":"Xiangnan He and Tat-Seng Chua (2017). Neural factorization machines for sparse predictive analytics","authors":"He","year":"2017","venue":"Proc. of SIGIR'17","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2017] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger (2017). Densely Connected Convolutional Networks. In Proc. of CVPR'17, pp.2261-2269.","title":"Densely Connected Convolutional Networks","authors":"Gao Huang, Zhuang Liu, Laurens Van Der Maaten, Kilian Q Weinberger","year":"2017","venue":"Proc. of CVPR'17","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ioffe et al., 2015] Sergey Ioffe and Christian Szegedy (2015). Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proc. of ICML'15, pp.448-456.","title":"Batch normalization: accelerating deep network training by reducing internal covariate shift","authors":"Sergey Ioffe, Christian Szegedy","year":"2015","venue":"Proc. of ICML'15","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Juan et al., 2016] Yu-Chin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin (2016). Field-aware factoriza- tion machines for ctr prediction. In Proc. of RecSys'16, pp.43-50.","title":"Field-aware factorization machines for ctr prediction","authors":"Yu-Chin Juan, Yong Zhuang, Wei-Sheng Chin, Chih-Jen Lin","year":"2016","venue":"Proc. of RecSys'16","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2015] Qiang Liu, Feng Yu, Shu Wu, and Liang Wang (2015). A convolutional click prediction model. In Proc. of CIKM'15, pp.1743-1746.","title":"A convolutional click prediction model","authors":"Qiang Liu, Feng Yu, Shu Wu, Liang Wang","year":"2015","venue":"Proc. of CIKM'15","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2018] Han Liu, Xiangnan He, Fuli Feng, Liqiang Nie, Rui Liu, and Hanwang Zhang (2018). Discrete fac- torization machines for fast feature-based recommenda- tion. In Proc. of IJCAI'18, pp.3449-3455.","title":"Discrete factorization machines for fast feature-based recommendation","authors":"Han Liu, Xiangnan He, Fuli Feng, Liqiang Nie, Rui Liu, Hanwang Zhang","year":"2018","venue":"Proc. of IJCAI'18","doi":null,"url":null,"internal_url":"/proceedings/2018/479"},{"position":14,"raw":"[Liu et al., 2018] Yang Liu, Linfeng Li, and Jun Liu (2018). Bilateral neural embedding for collaborative filtering- based multimedia recommendation. Multimedia Tools Appl., 77(10):12533-12544.","title":"Bilateral neural embedding for collaborative filteringbased multimedia recommendation","authors":"Yang Liu, Linfeng Li, Jun Liu","year":"2018","venue":"Multimedia Tools Appl","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mcmahan et al., 2013] H. Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, Sharat Chikkerur, Dan Liu, Martin Watten- berg, Arnar Mar Hrafnkelsson, Tom Boulos, and Jeremy Kubica (2013). Ad click prediction: A view from the trenches. In Proc. of KDD'13, pp.1222-1230.","title":"Ad click prediction: A view from the trenches","authors":"H, Brendan Mcmahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, Sharat Chikkerur, Dan Liu, Martin Wattenberg, Arnar Mar Hrafnkelsson, Tom Boulos, Jeremy Kubica","year":"2013","venue":"Proc. of KDD'13","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pan et al., 2018] Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu (2018). Field-weighted factorization machines for click- through rate prediction in display advertising. In Proc. of WWW'18, pp.1349-1357.","title":"Field-weighted factorization machines for clickthrough rate prediction in display advertising","authors":"Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, Quan Lu","year":"2018","venue":"Proc. of WWW'18","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pearl, 2018] Judea Pearl (2018) Theoretical impediments to machine learning with seven sparks from the causal revolution. In Proc. of WSDM'18, Article 3.","title":"Judea Pearl (2018) Theoretical impediments to machine learning with seven sparks from the causal revolution","authors":"Pearl","year":"2018","venue":"Proc. of WSDM'18","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qu et al., 2016] Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang (2016). Product-based neural networks for user response predic- tion. In Proc. of ICDM'16, pp.1149-1154.","title":"Product-based neural networks for user response prediction","authors":"Qu","year":"2016","venue":"Proc. of ICDM'16","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rendle, 2011] Steffen Rendle (2011). Factorization ma- chines. In Proc. of SIGIR'11, pp.635-644.","title":"Factorization machines","authors":"Steffen Rendle","year":"2011","venue":"Proc. of SIGIR'11","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rendle, 2012] Steffen Rendle (2012). Factorization ma- chines with libFM. ACM TIST, 3(3):57:1-57:22.","title":"Factorization machines with libFM","authors":"Steffen Rendle","year":"2012","venue":"ACM TIST","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E. Hin- ton, Alex Krizhevsky, Ilya Sutskever and Ruslan Salakhutdinov (2014). Dropout: A simple way to pre- vent neural networks from overfitting. J. Mach. Learn. Res, 15(1):1929-1958.","title":"Dropout: A simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shi et al., 2014] Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, Joaquin Quiñonero Candela (2014). Practical lessons from predicting clicks on ads at facebook. In Proc. of ADKDD'14, pp.5:1-5:9.","title":"Practical lessons from predicting clicks on ads at facebook","authors":"Shi","year":"2014","venue":"Proc. of ADKDD'14","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2017] Ruoxi Wang, Bin Fu, Gang Fu and Min- gliang Wang (2017). Deep & cross network for AD click predictions. In Proc. of ADKDD'17, pp.12:1-12:7.","title":"Deep & cross network for AD click predictions","authors":"Wang","year":"2017","venue":"Proc. of ADKDD'17","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xiao et al., 2017] Jun Xiao, Hao Ye, Xiangnan He, Han- wang Zhang, Fei Wu, and Tat-Seng Chua (2017). At- tentional factorization machines: Learning the weight of feature interactions via attention networks. In Proc. of IJCAI'17, pp.3119-3125.","title":"Attentional factorization machines: Learning the weight of feature interactions via attention networks","authors":"Jun Xiao, Xiangnan Hao Ye, Hanwang He, Fei Zhang, Tat-Seng Wu, Chua","year":"2017","venue":"Proc. of IJCAI'17","doi":null,"url":null,"internal_url":"/proceedings/2017/435"},{"position":25,"raw":"[Ying et al., 2016] Ying Shan, T. Ryan Hoens, Jian Jiao, Hai- jing Wang, Dong Yu, and J. C. Mao (2016). Deep cross- ing: web-scale modeling without manually crafted com- binatorial features. In Proc. of SIGKDD'16, pp.255- 262.","title":"Deep crossing: web-scale modeling without manually crafted combinatorial features","authors":"Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, J C Mao","year":"2016","venue":"Proc. of SIGKDD'16","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2016] Weinan Zhang, Tianming Du, and Jun Wang (2016). Deep learning over multi-field categorical data: A case study on user response prediction. In Proc. of ECIR'16, pp.45-57.","title":"Deep learning over multi-field categorical data: A case study on user response prediction","authors":"Weinan Zhang, Tianming Du, Jun Wang","year":"2016","venue":"Proc. of ECIR'16","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}