{"type":"ScholarlyArticle","title":"An Input-aware Factorization Machine for Sparse Prediction","authors":[{"name":"Yantao Yu","slug":"yantao-yu","given":"Yantao","family":"Yu","affiliation":"Graduate School at Shenzhen , Tsinghua University"},{"name":"Zhen Wang","slug":"zhen-wang","given":"Zhen","family":"Wang","affiliation":"Graduate School at Shenzhen , Tsinghua University"},{"name":"Bo Yuan","slug":"bo-yuan","given":"Bo","family":"Yuan","affiliation":"Graduate School at Shenzhen , Tsinghua University"}],"abstract":"Factorization machines (FMs) are a class of general predictors working effectively with sparse data, which represents features using factorized parameters and weights. However, the accuracy of FMs can be adversely affected by the fixed representation trained for each feature, as the same feature is usually not equally predictive and useful in different instances. In fact, the inaccurate representation of features may even introduce noise and degrade the overall performance. In this work, we improve FMs by explicitly considering the impact of individual input upon the representation of features. We propose a novel model named \\textit{Input-aware Factorization Machine} (IFM), which learns a unique input-aware factor for the same feature in different instances via a neural network. Comprehensive experiments on three real-world recommendation datasets are used to demonstrate the effectiveness and mechanism of IFM. Empirical results indicate that IFM is significantly better than the standard FM model and consistently outperforms four state-of-the-art deep learning based methods.","doi":"10.24963/ijcai.2019/203","url":"https://ijcai.org/proceedings/2019/203","pdf":"https://ijcai.org/proceedings/2019/0203.pdf","markdown":"https://ijcai.org/proceedings/2019/203.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/203","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Humans and AI","track":"Main track","number":203,"pages":{"first":1466,"last":1472},"keywords":["Multidisciplinary Topics and Applications: Information Retrieval","Humans and AI: Personalization and User Modeling","Machine Learning: Data Mining","Machine Learning: Recommender Systems"],"videos":[],"references":[{"position":1,"raw":"[Baltrunas et al., 2015] Linas Baltrunas, Karen Church, Alexandros Karatzoglou, and Nuria Oliver. Frappe: Un- derstanding the usage and perception of mobile app rec- ommendations in-the-wild. CoRR, abs/1505.03014, 2015.","title":"Frappe: Un-derstanding the usage and perception of mobile app recommendations in-the-wild","authors":"Linas Baltrunas, Karen Church, Alexandros Karatzoglou, Nuria Oliver","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cheng et al., 2014] Chen Cheng, Fen Xia, Tong Zhang, Ir- win King, and Michael R Lyu. Gradient boosting factor- ization machines. In Proceedings of the 8th ACM Con- ference on Recommender systems, pages 265-272. ACM, 2014.","title":"Gradient boosting factorization machines","authors":"Chen Cheng, Fen Xia, Tong Zhang, Irwin King, Michael R Lyu","year":"2014","venue":"Proceedings of the 8th ACM Conference on Recommender systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng et al., 2016] Heng Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, and Hemal Shah. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Sys- tems, pages 7-10. ACM, 2016.","title":"Wide & deep learning for recommender systems","authors":"Levent Heng Tze Cheng, Jeremiah Koc, Tal Harmsen, Hemal Shaked, Shah","year":"2016","venue":"Proceedings of the 1st Workshop on Deep Learning for Recommender Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Guo et al., 2017] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: a factorization-machine based neural network for ctr predic- tion. In Proceedings of the 26th International Joint Con- ference on Artificial Intelligence, pages 1725-1731, 2017.","title":"Deepfm: a factorization-machine based neural network for ctr prediction","authors":"Guo","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/239"},{"position":5,"raw":"[Harper and Konstan, 2016] F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):19, 2016.","title":"The movielens datasets: History and context","authors":"Maxwell Harper, Joseph A Konstan","year":"2016","venue":"Acm transactions on interactive intelligent systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He and Chua, 2017] Xiangnan He and Tat-Seng Chua. Neu- ral factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR con- ference on Research and Development in Information Re- trieval, pages 355-364. ACM, 2017.","title":"Xiangnan He and Tat-Seng Chua. Neural factorization machines for sparse predictive analytics","authors":"Chua He","year":"2017","venue":"Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Juan et al., 2016] Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. Field-aware factorization ma- chines for ctr prediction. In Proceedings of the 10th ACM Conference on Recommender Systems, pages 43-50. ACM, 2016.","title":"Field-aware factorization machines for ctr prediction","authors":"Juan","year":"2016","venue":"Proceedings of the 10th ACM Conference on Recommender Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Koren et al., 2009] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recom- mender systems. Computer, (8):30-37, 2009.","title":"Matrix factorization techniques for recommender systems","authors":"Koren","year":"2009","venue":"Computer","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lian et al., 2018] Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. xdeepfm: Combining explicit and implicit feature inter- actions for recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, pages 1754-1763. ACM, 2018.","title":"xdeepfm: Combining explicit and implicit feature interactions for recommender systems","authors":"Lian","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[McMahan et al., 2013] H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. Ad click prediction: a view from the trenches. In Proceedings of the 19th ACM SIGKDD in- ternational conference on Knowledge discovery and data mining, pages 1222-1230. ACM, 2013.","title":"Ad click prediction: a view from the trenches","authors":"Gary H Brendan Mcmahan, David Holt, Michael Sculley, Dietmar Young, Julian Ebner, Lan Grady, Todd Nie, Eugene Phillips, Davydov, Daniel Golovin","year":"2013","venue":"Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Pan et al., 2018] Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. Field-weighted factorization machines for click-through rate prediction in display advertising. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, pages 1349-1357. International World Wide Web Confer- ences Steering Committee, 2018.","title":"Field-weighted factorization machines for click-through rate prediction in display advertising","authors":"Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, Quan Lu","year":"2018","venue":"Proceedings of the 2018 World Wide Web Conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Petroni et al., 2015] Fabio Petroni, Luciano Del Corro, and Rainer Gemulla. Core: Context-aware open relation ex- traction with factorization machines. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1763-1773, 2015.","title":"Core: Context-aware open relation extraction with factorization machines","authors":"Fabio Petroni, Luciano Del Corro, Rainer Gemulla","year":"2015","venue":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Qu et al., 2016] Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. Product-based neural networks for user response prediction. In Data Min- ing (ICDM), 2016 IEEE 16th International Conference on, pages 1149-1154. IEEE, 2016.","title":"Product-based neural networks for user response prediction","authors":"Qu","year":"2016","venue":"IEEE 16th International Conference on","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rendle et al., 2011] Steffen Rendle, Zeno Gantner, Christoph Freudenthaler, and Lars Schmidt-Thieme. Fast context-aware recommendations with factorization machines. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, pages 635-644. ACM, 2011.","title":"Fast context-aware recommendations with factorization machines","authors":"Steffen Rendle, Zeno Gantner, Christoph Freudenthaler, Lars Schmidt-Thieme","year":"2011","venue":"Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rendle, 2010] Steffen Rendle. Factorization machines. In Data Mining (ICDM), 2010 IEEE 10th International Con- ference on, pages 995-1000. IEEE, 2010.","title":"Factorization machines","authors":"Steffen Rendle","year":"2010","venue":"IEEE 10th International Conference on","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rendle, 2012] Steffen Rendle. Factorization machines with libFM. ACM Trans. Intell. Syst. Technol., 3(3):57:1-57:22, May 2012.","title":"Factorization machines with libFM","authors":"Steffen Rendle","year":"2012","venue":"ACM Trans. Intell. Syst. Technol","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shan et al., 2016] Ying Shan, T Ryan Hoens, Jian Jiao, Hai- jing Wang, Dong Yu, and JC Mao. Deep crossing: Web- scale modeling without manually crafted combinatorial features. In Proceedings of the 22nd ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, pages 255-262. ACM, 2016.","title":"Deep crossing: Webscale modeling without manually crafted combinatorial features","authors":"Ying Shan, Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, J C Mao","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Re- search, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sukhbaatar et al., 2015] Sainbayar Sukhbaatar, Jason We- ston, Rob Fergus, et al. End-to-end memory networks. In Advances in neural information processing systems, pages 2440-2448, 2015.","title":"End-to-end memory networks","authors":"Sukhbaatar","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Weston et al., 2014] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. arXiv preprint arXiv:1410.3916, 2014.","title":"Memory networks","authors":"Jason Weston, Sumit Chopra","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xiao et al., 2017] Jun Xiao, Hao Ye, Xiangnan He, Han- wang Zhang, Fei Wu, and Tat-Seng Chua. Attentional fac- torization machines: learning the weight of feature inter- actions via attention networks. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pages 3119-3125, 2017.","title":"Attentional factorization machines: learning the weight of feature interactions via attention networks","authors":"Jun Xiao, Xiangnan Hao Ye, Hanwang He, Fei Zhang, Tat-Seng Wu, Chua","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/435"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}