{"type":"ScholarlyArticle","title":"Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks","authors":[{"name":"Jun Xiao","slug":"jun-xiao","given":"Jun","family":"Xiao","affiliation":"College of Computer Science , Zhejiang University"},{"name":"Hao Ye","slug":"hao-ye","given":"Hao","family":"Ye","affiliation":"College of Computer Science , Zhejiang University"},{"name":"Xiangnan He","slug":"xiangnan-he","given":"Xiangnan","family":"He","affiliation":"School of Computing , National University of Singapore"},{"name":"Hanwang Zhang","slug":"hanwang-zhang","given":"Hanwang","family":"Zhang","affiliation":"School of Computing , National University of Singapore"},{"name":"Fei Wu","slug":"fei-wu","given":"Fei","family":"Wu","affiliation":"College of Computer Science , Zhejiang University"},{"name":"Tat-Seng Chua","slug":"tat-seng-chua","given":"Tat-Seng","family":"Chua","affiliation":"School of Computing , National University of Singapore"}],"abstract":"Factorization Machines (FMs) are a supervised learning approach that enhances the linear regression model by incorporating the second-order feature interactions. Despite effectiveness, FM can be hindered by its modelling of all feature interactions with the same weight, as not all feature interactions are equally useful and predictive. For example, the interactions with useless features may even introduce noises and adversely degrade the performance. In this work, we improve FM by discriminating the importance of different feature interactions. We propose a novel model named Attentional Factorization Machine (AFM), which learns the importance of each feature interaction from data via a neural attention network. Extensive experiments on two real-world datasets demonstrate the effectiveness of AFM. Empirically, it is shown on regression task AFM betters FM with a 8.6% relative improvement, and consistently outperforms the state-of-the-art deep learning methods Wide&Deep [Cheng et al., 2016] and DeepCross [Shan et al., 2016] with a much simpler structure and fewer model parameters. Our implementation of AFM is publicly available at: https://github.com/hexiangnan/attentional_factorization_machine","doi":"10.24963/ijcai.2017/435","url":"https://ijcai.org/proceedings/2017/435","pdf":"https://ijcai.org/proceedings/2017/0435.pdf","markdown":"https://ijcai.org/proceedings/2017/435.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/435","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":435,"pages":{"first":3119,"last":3125},"keywords":["Multidisciplinary Topics and Applications: Personalization and User Modeling","Machine Learning: Data Mining","Machine Learning: Machine Learning","Machine Learning: Neural Networks"],"videos":[],"references":[{"position":1,"raw":"[Baltrunas et al., 2015] Linas Baltrunas, Karen Church, Alexan- dros Karatzoglou, and Nuria Oliver. Frappe: Understanding the usage and perception of mobile app recommendations in-the- wild. CoRR, abs/1505.03014, 2015.","title":"Frappe: Understanding the usage and perception of mobile app recommendations in-thewild","authors":"Baltrunas","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bayer et al., 2017] Immanuel Bayer, Xiangnan He, Bhargav Kana- gal, and Steffen Rendle. A generic coordinate descent framework for learning from implicit feedback. In WWW, 2017.","title":"A generic coordinate descent framework for learning from implicit feedback","authors":"Bayer","year":"2017","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blondel et al., 2016] Mathieu Blondel, Akinori Fujino, Naonori Ueda, and Masakazu Ishihata. Higher-order factorization ma- chines. In NIPS, 2016.","title":"Higher-order factorization machines","authors":"Mathieu Blondel, Akinori Fujino, Naonori Ueda, Masakazu Ishihata","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2016] Tao Chen, Xiangnan He, and Min-Yen Kan. Context-aware image tweet modelling and recommendation. In MM, 2016.","title":"Context-aware image tweet modelling and recommendation","authors":"Tao Chen, Xiangnan He, Min-Yen Kan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2017a] Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. Attentive collab- orative filtering: Multimedia recommendation with feature-and item-level attention. In SIGIR, 2017.","title":"Attentive collaborative filtering: Multimedia recommendation with feature-and item-level attention","authors":"Chen","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2017b] Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, and Tat-Seng Chua. SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning. In CVPR, 2017.","title":"SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning","authors":"Chen","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2014] Chen Cheng, Fen Xia, Tong Zhang, Irwin King, and Michael R Lyu. Gradient boosting factorization ma- chines. In RecSys, 2014.","title":"Gradient boosting factorization machines","authors":"Chen Cheng, Fen Xia, Tong Zhang, Irwin King, Michael R Lyu","year":"2014","venue":"RecSys","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cheng et al., 2016] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, et al. Wide & deep learning for recommender systems. In DLRS, 2016.","title":"Wide & deep learning for recommender systems","authors":"Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen","year":"2016","venue":"DLRS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Harper and Konstan, 2015] F. Maxwell Harper and Joseph A. Kon- stan. The movielens datasets: History and context. ACM TIIS, 2015.","title":"The movielens datasets: History and context","authors":"F, Maxwell Harper, Joseph A Konstan","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016a] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"He","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016b] Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. Fast matrix factorization for online recom- mendation with implicit feedback. In SIGIR, 2016.","title":"Fast matrix factorization for online recommendation with implicit feedback","authors":"He","year":"2016","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2017a] Xiangnan He, Ming Gao, Min-Yen Kan, and Dingxian Wang. BiRank: Towards ranking on bipartite graphs. IEEE TKDE, 2017.","title":"BiRank: Towards ranking on bipartite graphs","authors":"He","year":"2017","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2017b] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filering. In WWW, 2017.","title":"Neural collaborative filering","authors":"He","year":"2017","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Juan et al., 2016] Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. Field-aware factorization machines for ctr pre- diction. In RecSys, 2016.","title":"Field-aware factorization machines for ctr prediction","authors":"Juan","year":"2016","venue":"RecSys","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Koren, 2008] Yehuda Koren. Factorization meets the neighbor- hood: A multifaceted collaborative filtering model. In KDD, 2008.","title":"Factorization meets the neighborhood: A multifaceted collaborative filtering model","authors":"Koren ;, Yehuda Koren","year":"2008","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Petroni et al., 2015] Fabio Petroni, Luciano Del Corro, and Rainer Gemulla. Core: Context-aware open relation extraction with fac- torization machines. In EMNLP, 2015.","title":"Core: Context-aware open relation extraction with factorization machines","authors":"Fabio Petroni, Luciano Del Corro, Rainer Gemulla","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rendle et al., 2011] Steffen Rendle, Zeno Gantner, Christoph Freudenthaler, and Lars Schmidt-Thieme. Fast context-aware recommendations with factorization machines. In SIGIR, 2011.","title":"Fast context-aware recommendations with factorization machines","authors":"Steffen Rendle, Zeno Gantner, Christoph Freudenthaler, Lars Schmidt-Thieme","year":"2011","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rendle, 2010] Steffen Rendle. Factorization machines. In ICDM, 2010.","title":"Factorization machines","authors":"Steffen Rendle","year":"2010","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rendle, 2012] Steffen Rendle. Factorization machines with libfm. ACM TIST, 2012.","title":"Factorization machines with libfm","authors":"Steffen Rendle, Rendle","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[He and Chua, 2017] Xiangnan He and Tat-Seng Chua. Neural fac- torization machines for sparse predictive analytics. In SIGIR, 2017.","title":"Xiangnan He and Tat-Seng Chua. Neural factorization machines for sparse predictive analytics","authors":"Chua He","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[He et al., 2014] Xiangnan He, Min-Yen Kan, Peichu Xie, and Xiao Chen. Comment-based multi-view clustering of web 2.0 items. In WWW, 2014.","title":"Comment-based multi-view clustering of web 2.0 items","authors":"He","year":"2014","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shan et al., 2016] Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. Deep crossing: Web-scale mod- eling without manually crafted combinatorial features. In KDD, 2016.","title":"Deep crossing: Web-scale modeling without manually crafted combinatorial features","authors":"Ying Shan, Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, J C Mao","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shen et al., 2015] Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR, 2015.","title":"Supervised discrete hashing","authors":"Fumin Shen, Chunhua Shen, Wei Liu, Heng Tao Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. JMLR, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2015] Meng Wang, Xueliang Liu, and Xindong Wu. Visual classification by l1-hypergraph modeling. IEEE TKDE, 2015.","title":"Visual classification by l1-hypergraph modeling","authors":"Meng Wang, Xueliang Liu, Xindong Wu","year":"2015","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2016] Meng Wang, Weijie Fu, Shijie Hao, Dacheng Tao, and Xindong Wu. Scalable semi-supervised learning by ef- ficient anchor graph regularization. IEEE TKDE, 2016.","title":"Scalable semi-supervised learning by efficient anchor graph regularization","authors":"Meng Wang, Weijie Fu, Shijie Hao, Dacheng Tao, Xindong Wu","year":"2016","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2017a] Xiang Wang, Xiangnan He, Liqiang Nie and Tat-Seng Chua Item Silk Road: Recommending Items from In- formation Domains to Social Users SIGIR, 2017.","title":"Recommending Items from Information Domains to Social Users SIGIR","authors":"Wang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2017b] Meng Wang, Weijie Fu, Shijie Hao, Hengchang Liu, and Xindong Wu. Learning on big graph: Label inference and regularization with anchor hierarchy. IEEE TKDE, 2017.","title":"Learning on big graph: Label inference and regularization with anchor hierarchy","authors":"Wang","year":"2017","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xiong et al., 2017] Chenyan Xiong, Jimie Callan, and Tie-Yen Liu. Learning to attend and to rank with word-entity duets. In SIGIR, 2017.","title":"Learning to attend and to rank with word-entity duets","authors":"Xiong","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yang et al., 2014] Yang Yang, Zheng-Jun Zha, Yue Gao, Xiaofeng Zhu, and Tat-Seng Chua. Exploiting web images for seman- tic video indexing via robust sample-specific loss. IEEE TMM, 2014.","title":"Exploiting web images for semantic video indexing via robust sample-specific loss","authors":"Yang Yang, Zheng-Jun Zha, Yue Gao, Xiaofeng Zhu, Tat-Seng Chua","year":"2014","venue":"IEEE TMM","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2015] Yang Yang, Zhigang Ma, Yi Yang, Feiping Nie, and Heng Tao Shen. Multitask spectral clustering by exploring intertask correlation. IEEE TCYB, 2015.","title":"Multitask spectral clustering by exploring intertask correlation","authors":"Yang Yang, Zhigang Ma, Yi Yang, Feiping Nie, Heng Tao Shen","year":"2015","venue":"IEEE TCYB","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2016a] Hanwang Zhang, Xindi Shang, Huanbo Luan, Meng Wang, and Tat-Seng Chua. Learning from collective intel- ligence: Feature learning using social images and tags. TMM, 2016.","title":"Learning from collective intelligence: Feature learning using social images and tags","authors":"Zhang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2016b] Hanwang Zhang, Fumin Shen, Wei Liu, Xi- angnan He, Huanbo Luan, and Tat-Seng Chua. Discrete collabo- rative filtering. In SIGIR, 2016.","title":"Discrete collaborative filtering","authors":"Zhang","year":"2016","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2017] Hanwang Zhang, Zawlin Kyaw, Shih-Fu Chang, and Tat-Seng Chua. Visual translation embedding net- work for visual relation detection. In CVPR, 2017.","title":"Visual translation embedding network for visual relation detection","authors":"Hanwang Zhang, Zawlin Kyaw, Shih-Fu Chang, Tat-Seng Chua","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhao et al., 2015] Zhou Zhao, Lijun Zhang, Xiaofei He, and Wil- fred Ng. Expert finding for question answering via graph regu- larized matrix completion. TKDE, 2015.","title":"Expert finding for question answering via graph regularized matrix completion","authors":"Zhou Zhao, Lijun Zhang, Xiaofei He, Wilfred Ng","year":"2015","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhao et al., 2016] Zhou Zhao, Hanqing Lu, Deng Cai, Xiaofei He, and Yueting Zhuang. User Preference Learning for Online Social Recommendation. TKDE, 2016.","title":"User Preference Learning for Online Social Recommendation","authors":"Zhou Zhao, Hanqing Lu, Deng Cai, Xiaofei He, Yueting Zhuang","year":"2016","venue":"TKDE","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}