{"type":"ScholarlyArticle","title":"Locally Linear Factorization Machines","authors":[{"name":"Chenghao Liu","slug":"chenghao-liu","given":"Chenghao","family":"Liu","affiliation":"School of Computer Science and Technology , Zhejiang University , China"},{"name":"Teng Zhang","slug":"teng-zhang","given":"Teng","family":"Zhang","affiliation":"School of Computer Science and Technology , Zhejiang University , China"},{"name":"Peilin Zhao","slug":"peilin-zhao","given":"Peilin","family":"Zhao","affiliation":"Artificial Intelligence Department , Ant Financial Services Group , China"},{"name":"Jun Zhou","slug":"jun-zhou","given":"Jun","family":"Zhou","affiliation":"Artificial Intelligence Department , Ant Financial Services Group , China"},{"name":"Jianling Sun","slug":"jianling-sun","given":"Jianling","family":"Sun","affiliation":"School of Computer Science and Technology , Zhejiang University , China"}],"abstract":"Factorization Machines (FMs) are a widely used method for efficiently using high-order feature interactions in classification and regression tasks. Unfortunately, despite increasing interests in FMs, existing work only considers high order information of the input features which limits their capacities in non-linear problems and fails to capture the underlying structures of more complex data. In this work, we present a novel Locally Linear Factorization Machines (LLFM) which overcomes this limitation by exploring local coding technique. Unlike existing local coding classifiers that involve a phase of unsupervised anchor point learning and predefined local coding scheme which is suboptimal as the class label information is not exploited in discovering the encoding and thus can result in a suboptimal encoding for prediction, we formulate a joint optimization over the anchor points, local coding coordinates and FMs variables to minimize classification or regression risk. Empirically, we demonstrate that our approach achieves much better predictive accuracy than other competitive methods which employ LLFM with unsupervised anchor point learning and predefined local coding scheme.","doi":"10.24963/ijcai.2017/319","url":"https://ijcai.org/proceedings/2017/319","pdf":"https://ijcai.org/proceedings/2017/0319.pdf","markdown":"https://ijcai.org/proceedings/2017/319.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/319","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":319,"pages":{"first":2294,"last":2300},"keywords":["Machine Learning: Classification","Machine Learning: Data Mining","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Anava and Levy, 2016] Oren Anava and Kfir Levy. k*- nearest neighbors: From global to local. In Advances in Neural Information Processing Systems, pages 4916- 4924, 2016.","title":"k*nearest neighbors: From global to local","authors":"Levy ; Anava, Oren Anava, Kfir Levy","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blondel et al., 2015] Mathieu Blondel, Akinori Fujino, and Naonori Ueda. Convex factorization machines. In Joint European Conference on Machine Learning and Know- ledge Discovery in Databases, pages 19-35. Springer, 2015.","title":"Convex factorization machines","authors":"Mathieu Blondel, Akinori Fujino, Naonori Ueda","year":"2015","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blondel et al., 2016a] Mathieu Blondel, Akinori Fujino, Naonori Ueda, and Masakazu Ishihata. Higher-order fac- torization machines. In Advances in Neural Information Processing Systems, pages 3351-3359, 2016.","title":"Higher-order factorization machines","authors":"Blondel","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blondel et al., 2016b] Mathieu Blondel, Masakazu Ishihata, Akinori Fujino, and Naonori Ueda. Polynomial networks and factorization machines: New insights and efficient trai- ning algorithms. In Proceedings of The 33rd International Conference on Machine Learning, pages 850-858, 2016.","title":"Polynomial networks and factorization machines: New insights and efficient training algorithms","authors":"Blondel","year":"2016","venue":"Proceedings of The 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bordes et al., 2009] Antoine Bordes, Léon Bottou, and Pa- trick Gallinari. Sgd-qn: Careful quasi-newton stochastic gradient descent. Journal of Machine Learning Research, 10(Jul):1737-1754, 2009.","title":"Sgd-qn: Careful quasi-newton stochastic gradient descent","authors":"Antoine Bordes, Léon Bottou, Patrick Gallinari","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cao et al., 2007] Zhe Cao, Tao Qin, Tie-Yan Liu, Ming- Feng Tsai, and Hang Li. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th international conference on Machine learning, pages 129- 136. ACM, 2007.","title":"Learning to rank: from pairwise approach to listwise approach","authors":"Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, Hang Li","year":"2007","venue":"Proceedings of the 24th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"CHIN et al., ] WEI-SHENG CHIN, BO-WEN YUAN, MENG-YUAN YANG, and CHIH-JEN LIN. An efficient alternating newton method for learning factorization ma- chines. [Friedman et al., 2001] Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning, vo- lume 1. Springer series in statistics Springer, Berlin, 2001.","title":"An efficient alternating newton method for learning factorization machines","authors":"Chin","year":"2001","venue":"The elements of statistical learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gu and Han, 2013] Quanquan Gu and Jiawei Han. Cluste- red support vector machines. In AISTATS, pages 307-315, 2013.","title":"Clustered support vector machines","authors":"Han ; Gu, Quanquan Gu, Jiawei Han","year":"2013","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ladicky and Torr, 2011] Lubor Ladicky and Philip Torr. Locally linear support vector machines. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 985-992, 2011.","title":"Lubor Ladicky and Philip Torr. Locally linear support vector machines","authors":"Torr Ladicky","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning (ICML-11)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin and Ye, 2016] Ming Lin and Jieping Ye. A non-convex one-pass framework for generalized factorization machine and rank-one matrix sensing. In Advances in Neural Infor- mation Processing Systems, pages 1633-1641, 2016.","title":"A non-convex one-pass framework for generalized factorization machine and rank-one matrix sensing","authors":"Lin, Ye, Ming Lin, Jieping Ye","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2011] Lingqiao Liu, Lei Wang, and Xinwang Liu. In defense of soft-assignment coding. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 2486-2493. IEEE, 2011.","title":"In defense of soft-assignment coding","authors":"Liu","year":"2011","venue":"Computer Vision (ICCV), 2011 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mao et al., 2015] Xue Mao, Zhouyu Fu, Ou Wu, and Wei- ming Hu. Optimizing locally linear classifiers with super- vised anchor point learning. In IJCAI, pages 3699-3706, 2015.","title":"Optimizing locally linear classifiers with supervised anchor point learning","authors":"Xue Mao, Zhouyu Fu, Ou Wu, Weiming Hu","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/520"},{"position":13,"raw":"[Rendle, 2010] Steffen Rendle. Factorization machines. In 2010 IEEE International Conference on Data Mining, pa- ges 995-1000. IEEE, 2010.","title":"Factorization machines","authors":"Steffen Rendle","year":"2010","venue":"2010 IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rendle, 2012] Steffen Rendle. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST), 3(3):57, 2012.","title":"Factorization machines with libfm","authors":"Steffen Rendle","year":"2012","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shalev-Shwartz et al., 2007] Shai Shalev-Shwartz, Yoram Singer, and Nathan Srebro. Pegasos: Primal estimated sub- gradient solver for svm. In Proceedings of the 24th inter- national conference on Machine learning, pages 807-814. ACM, 2007.","title":"Pegasos: Primal estimated subgradient solver for svm","authors":"Shai Shalev-Shwartz, Yoram Singer, Nathan Srebro","year":"2007","venue":"Proceedings of the 24th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Van Gemert et al., 2008] Jan C Van Gemert, Jan-Mark Geu- sebroek, Cor J Veenman, and Arnold WM Smeulders. Kernel codebooks for scene categorization. In European conference on computer vision, pages 696-709. Springer, 2008.","title":"Kernel codebooks for scene categorization","authors":"Van Gemert","year":"2008","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vapnik, 2013] Vladimir Vapnik. The nature of statistical le- arning theory. Springer Science & Business Media, 2013.","title":"The nature of statistical learning theory","authors":"Vladimir Vapnik","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2010] Zhuang Wang, Koby Crammer, and Slo- bodan Vucetic. Multi-class pegasos on a budget. In Pro- ceedings of the 27th International Conference on Machine Learning (ICML-10), pages 1143-1150, 2010.","title":"Multi-class pegasos on a budget","authors":"Zhuang Wang, Koby Crammer, Slobodan Vucetic","year":"2010","venue":"Proceedings of the 27th International Conference on Machine Learning (ICML-10)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yu et al., 2009] Kai Yu, Tong Zhang, and Yihong Gong. Nonlinear learning using local coordinate coding. In Ad- vances in neural information processing systems, pages 2223-2231, 2009.","title":"Nonlinear learning using local coordinate coding","authors":"Kai Yu, Tong Zhang, Yihong Gong","year":"2009","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhao et al., 2011] Peilin Zhao, Rong Jin, Tianbao Yang, and Steven C Hoi. Online auc maximization. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 233-240, 2011.","title":"Online auc maximization","authors":"Peilin Zhao, Rong Jin, Tianbao Yang, Steven C Hoi","year":"2011","venue":"Proceedings of the 28th international conference on machine learning (ICML-11)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}