{"type":"ScholarlyArticle","title":"Multiplicative Sparse Feature Decomposition for Efficient Multi-View Multi-Task Learning","authors":[{"name":"Lu Sun","slug":"lu-sun","given":"Lu","family":"Sun","affiliation":"Bioinformatics Center , Institute for Chemical Research , Kyoto University , Japan"},{"name":"Canh Hao Nguyen","slug":"canh-hao-nguyen","given":"Canh Hao","family":"Nguyen","affiliation":"Bioinformatics Center , Institute for Chemical Research , Kyoto University , Japan"},{"name":"Hiroshi Mamitsuka","slug":"hiroshi-mamitsuka","given":"Hiroshi","family":"Mamitsuka","affiliation":"Bioinformatics Center , Institute for Chemical Research , Kyoto University , Japan; Department of Computer Science , Aalto University , Finland"}],"abstract":"Multi-view multi-task learning refers to dealing with dual-heterogeneous data,where each sample has multi-view features,and multiple tasks are correlated via common views.Existing methods do not sufficiently address three key challenges:(a) saving task correlation efficiently, (b) building a sparse model and (c) learning view-wise weights.In this paper, we propose a new method to directly handle these challenges based on multiplicative sparse feature decomposition.For (a), the weight matrix is decomposed into two components via low-rank constraint matrix factorization, which saves task correlation by learning a reduced number of model parameters.For (b) and (c), the first component is further decomposed into two sub-components,to select topic-specific features and learn view-wise importance, respectively. Theoretical analysis reveals its equivalence with a general form of joint regularization,and motivates us to develop a fast optimization algorithm in a linear complexity w.r.t. the data size.Extensive experiments on both simulated and real-world datasets validate its efficiency.","doi":"10.24963/ijcai.2019/486","url":"https://ijcai.org/proceedings/2019/486","pdf":"https://ijcai.org/proceedings/2019/0486.pdf","markdown":"https://ijcai.org/proceedings/2019/486.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/486","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":486,"pages":{"first":3506,"last":3512},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Feature Selection ; Learning Sparse Models","Machine Learning: Multi-instance;Multi-label;Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Blum and Mitchell, 1998] Avrim Blum and Tom Mitchell. Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Compu- tational learning theory, pages 92-100. ACM, 1998.","title":"Combining labeled and unlabeled data with co-training","authors":"Avrim Blum, Tom Mitchell","year":"1998","venue":"Proceedings of the eleventh annual conference on Computational learning theory","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2011] Jianhui Chen, Jiayu Zhou, and Jieping Ye. Integrating low-rank and group-sparse structures for robust multi-task learning. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 42-50. ACM, 2011.","title":"Integrating low-rank and group-sparse structures for robust multi-task learning","authors":"Jianhui Chen, Jiayu Zhou, Jieping Ye","year":"2011","venue":"Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gong et al., 2012a] Pinghua Gong, Jieping Ye, and Chang- shui Zhang. Multi-stage multi-task feature learning. In Advances in neural information processing systems, pages 1988-1996, 2012.","title":"Multi-stage multi-task feature learning","authors":"Gong","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gong et al., 2012b] Pinghua Gong, Jieping Ye, and Chang- shui Zhang. Robust multi-task feature learning. In Pro- ceedings of the 18th ACM SIGKDD international con- ference on Knowledge discovery and data mining, pages 895-903. ACM, 2012.","title":"Robust multi-task feature learning","authors":"Gong","year":"2012","venue":"Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Han and Zhang, 2015] Lei Han and Yu Zhang. Learning multi-level task groups in multi-task learning. In AAAI, volume 15, pages 2638-2644, 2015.","title":"Learning multi-level task groups in multi-task learning","authors":"Zhang ; ] Lei Han, Yu Han, Zhang","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hardoon et al., 2004] David R Hardoon, Sandor Szedmak, and John Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. Neural computation, 16(12):2639-2664, 2004.","title":"Canonical correlation analysis: An overview with application to learning methods","authors":"Sandor David R Hardoon, John Szedmak, Shawe-Taylor","year":"2004","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He and Lawrence, 2011] Jingrui He and Rick Lawrence. A graph-based framework for multi-task multi-view learn- ing. Proceedings of the 28th International Conference on Machine Learning, ICML 2011, pages 25-32, 2011.","title":"Jingrui He and Rick Lawrence. A graph-based framework for multi-task multi-view learning","authors":"Lawrence He","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning, ICML 2011","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hoerl and Kennard, 1970] Arthur E Hoerl and Robert W Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55-67, 1970.","title":"Ridge regression: Biased estimation for nonorthogonal problems","authors":"E Arthur, Robert W Hoerl, Kennard","year":"1970","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jalali et al., 2010] Ali Jalali, Sujay Sanghavi, Chao Ruan, and Pradeep K Ravikumar. A dirty model for multi-task learning. In Advances in neural information processing systems, pages 964-972, 2010.","title":"A dirty model for multi-task learning","authors":"Ali Jalali, Sujay Sanghavi, Pradeep K Chao Ruan, Ravikumar","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jin et al., 2013] Xin Jin, Fuzhen Zhuang, Shuhui Wang, Qing He, and Zhongzhi Shi. Shared Structure Learning for Multiple Tasks with Multiple Views. In Machine Learning and Knowledge Discovery in Databases, pages 353-368, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg.","title":"Shared Structure Learning for Multiple Tasks with Multiple Views","authors":"Xin Jin, Fuzhen Zhuang, Shuhui Wang, Qing He, Zhongzhi Shi","year":"2013","venue":"Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kan et al., 2016] Meina Kan, Shiguang Shan, Haihong Zhang, Shihong Lao, and Xilin Chen. Multi-view discrim- inant analysis. IEEE transactions on pattern analysis and machine intelligence, 38(1):188-194, 2016.","title":"Multi-view discriminant analysis","authors":"Meina Kan, Shiguang Shan, Haihong Zhang, Shihong Lao, Xilin Chen","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Huan, 2018] Xiaoli Li and Jun Huan. Interactions modeling in multi-task multi-view learning with consis- tent task diversity. In Proceedings of the 27th ACM Inter- national Conference on Information and Knowledge Man- agement, CIKM '18, pages 853-861. ACM, 2018.","title":"Interactions modeling in multi-task multi-view learning with consistent task diversity","authors":"Huan ; ] Xiaoli Li, Jun Li, Huan","year":"2018","venue":"Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM '18","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2009a] Han Liu, Mark Palatucci, and Jian Zhang. Blockwise coordinate descent procedures for the multi- task lasso, with applications to neural semantic basis dis- covery. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 649-656. ACM, 2009.","title":"Blockwise coordinate descent procedures for the multi-task lasso, with applications to neural semantic basis discovery","authors":"Liu","year":"2009","venue":"Proceedings of the 26th Annual International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2009b] Jun Liu, Shuiwang Ji, and Jieping Ye. Multi-task feature learning via efficient l2, 1-norm mini- mization. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI '09, pages 339-348. AUAI Press, 2009.","title":"Multi-task feature learning via efficient l2, 1-norm minimization","authors":"Liu","year":"2009","venue":"Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI '09","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lozano and Swirszcz, 2012] Aurelie C Lozano and Grze- gorz Swirszcz. Multi-level lasso for sparse multi-task re- gression. In Proceedings of the 29th International Cofer- ence on International Conference on Machine Learning, pages 595-602. Omnipress, 2012.","title":"Multi-level lasso for sparse multi-task regression","authors":"Aurelie C Lozano, Grzegorz Swirszcz","year":"2012","venue":"Proceedings of the 29th International Coference on International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lu et al., 2017] Chun-Ta Lu, Lifang He, Weixiang Shao, Bokai Cao, and Philip S. Yu. Multilinear Factorization Machines for Multi-Task Multi-View Learning. In Pro- ceedings of the Tenth ACM International Conference on Web Search and Data Mining -WSDM '17, pages 701- 709, 2017.","title":"Multilinear Factorization Machines for Multi-Task Multi-View Learning","authors":"Chun-Ta Lu, Lifang He, Weixiang Shao, Bokai Cao, Philip S Yu","year":"2017","venue":"Proceedings of the Tenth ACM International Conference on Web Search and Data Mining -WSDM '17","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Muslea et al., 2006] Ion Muslea, Steven Minton, and Craig A Knoblock. Active learning with multiple views. Journal of Artificial Intelligence Research, 27:203-233, 2006.","title":"Ion Muslea, Steven Minton, and Craig A Knoblock","authors":"Muslea","year":"2006","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nesterov, 2013] Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013.","title":"Introductory lectures on convex optimization: A basic course","authors":"Yurii Nesterov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sindhwani et al., 2005] Vikas Sindhwani, Partha Niyogi, and Mikhail Belkin. A co-regularization approach to semi- supervised learning with multiple views. In Proceedings of ICML workshop on learning with multiple views, volume 2005, pages 74-79. Citeseer, 2005.","title":"A co-regularization approach to semisupervised learning with multiple views","authors":"Vikas Sindhwani, Partha Niyogi, Mikhail Belkin","year":"2005","venue":"Proceedings of ICML workshop on learning with multiple views","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sun and Jin, 2011] Shiliang Sun and Feng Jin. Robust co- training. International Journal of Pattern Recognition and Artificial Intelligence, 25(07):1113-1126, 2011.","title":"Robust cotraining","authors":"Jin ; Sun, Shiliang Sun, Feng Jin","year":"2011","venue":"International Journal of Pattern Recognition and Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tibshirani, 1996] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267-288, 1996.","title":"Regression shrinkage and selection via the lasso","authors":"Robert Tibshirani","year":"1996","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2016] Xin Wang, Jinbo Bi, Shipeng Yu, Jiang- wen Sun, and Minghu Song. Multiplicative multitask fea- ture learning. Journal of Machine Learning Research, 17(80):1-33, 2016.","title":"Multiplicative multitask feature learning","authors":"Xin Wang, Jinbo Bi, Shipeng Yu, Jiangwen Sun, Minghu Song","year":"2016","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xie and Sun, 2015] Xijiong Xie and Shiliang Sun. Multi- view twin support vector machines. Intelligent Data Anal- ysis, 19(4):701-712, 2015.","title":"Xijiong Xie and Shiliang Sun. Multiview twin support vector machines","authors":"Sun Xie","year":"2015","venue":"Intelligent Data Analysis","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang and Huan, 2012] Jintao Zhang and Jun Huan. Induc- tive multi-task learning with multiple view data. In Pro- ceedings of the 18th ACM SIGKDD international confer- ence on Knowledge discovery and data mining -KDD '12, page 543, 2012.","title":"Inductive multi-task learning with multiple view data","authors":"Huan Zhang, Jintao Zhang, Jun Huan","year":"2012","venue":"Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining -KDD '12","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2018] D. Zhou, J. Wang, B. Jiang, H. Guo, and Y. Li. Multi-task multi-view learning based on coopera- tive multi-objective optimization. IEEE Access, 6:19465- 19477, 2018.","title":"Multi-task multi-view learning based on cooperative multi-objective optimization","authors":"D Zhou, J Wang, B Jiang, H Guo, Y Li","year":"2018","venue":"IEEE Access","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}