{"type":"ScholarlyArticle","title":"Fast and Robust Multi-View Multi-Task Learning via Group Sparsity","authors":[{"name":"Lu Sun","slug":"lu-sun","given":"Lu","family":"Sun","affiliation":"Bioinformatics Center , Institute for Chemical Research , Kyoto University , Japan"},{"name":"Canh Hao Nguyen","slug":"canh-hao-nguyen","given":"Canh Hao","family":"Nguyen","affiliation":"Bioinformatics Center , Institute for Chemical Research , Kyoto University , Japan"},{"name":"Hiroshi Mamitsuka","slug":"hiroshi-mamitsuka","given":"Hiroshi","family":"Mamitsuka","affiliation":"Bioinformatics Center , Institute for Chemical Research , Kyoto University , Japan; Department of Computer Science , Aalto University , Finland"}],"abstract":"Multi-view multi-task learning has recently attracted more and more attention due to its dual-heterogeneity, i.e.,each task has heterogeneous features from multiple views, and probably correlates with other tasks via common views.Existing methods usually suffer from three problems: 1) lack the ability to eliminate noisy features, 2) hold a strict assumption on view consistency and 3) ignore the possible existence of task-view outliers.To overcome these limitations, we propose a robust method with joint group-sparsity by decomposing feature parameters into a sum of two components,in which one saves relevant features (for Problem 1) and flexible view consistency (for Problem 2),while the other detects task-view outliers (for Problem 3).With a global convergence property, we develop a fast algorithm to solve the optimization problem in a linear time complexity w.r.t. the number of features and labeled samples.Extensive experiments on various synthetic and real-world datasets demonstrate its effectiveness.","doi":"10.24963/ijcai.2019/485","url":"https://ijcai.org/proceedings/2019/485","pdf":"https://ijcai.org/proceedings/2019/0485.pdf","markdown":"https://ijcai.org/proceedings/2019/485.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/485","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":485,"pages":{"first":3499,"last":3505},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Feature Selection ; Learning Sparse Models","Machine Learning: Multi-instance;Multi-label;Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Belkin et al., 2006] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled exam- ples. Journal of machine learning research, 7(Nov):2399- 2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"Mikhail Belkin, Partha Niyogi, Vikas Sindhwani","year":"2006","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blum and Mitchell, 1998] Avrim Blum and Tom Mitchell. Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Compu- tational learning theory, pages 92-100. ACM, 1998.","title":"Combining labeled and unlabeled data with co-training","authors":"Avrim Blum, Tom Mitchell","year":"1998","venue":"Proceedings of the eleventh annual conference on Computational learning theory","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Boyd et al., 2011] S Boyd, N Parikh, B Peleato E Chu, and J Eckstein. Distributed Optimization and Statistical Learn- ing via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning, 3(1):1-122, 2011.","title":"Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers","authors":"N Boyd, B Parikh, Peleato E Chu, Eckstein","year":"2011","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2010] Jian-Feng Cai, Emmanuel J. Candès, and Zuowei Shen. A Singular Value Thresholding Algorithm for Matrix Completion. SIAM Journal on Optimization, 20(4):1956-1982, 2010.","title":"A Singular Value Thresholding Algorithm for Matrix Completion","authors":"Jian-Feng Cai, Emmanuel J Candès, Zuowei Shen","year":"2010","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yan-Tao Zheng. Nus- wide: A real-world web image database from national uni- versity of singapore. In Proc. of ACM Conf. on Image and Video Retrieval (CIVR'09), Santorini, Greece., 2009.","title":"Nuswide: A real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yan-Tao Zheng","year":"2009","venue":"Proc. of ACM Conf. on Image and Video Retrieval (CIVR'09)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goldstein et al., 2014] Tom Goldstein, Brendan O'Donoghue, Simon Setzer, and Richard Baraniuk. Fast alternating direction optimization methods. SIAM Journal on Imaging Sciences, 7(3):1588-1623, 2014.","title":"Fast alternating direction optimization methods","authors":"Tom Goldstein, O' Brendan, Simon Donoghue, Richard Setzer, Baraniuk","year":"2014","venue":"SIAM Journal on Imaging Sciences","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gong et al., 2012] Pinghua Gong, Jieping Ye, and Chang- shui Zhang. Robust multi-task feature learning. In Pro- ceedings of the 18th ACM SIGKDD international con- ference on Knowledge discovery and data mining, pages 895-903. ACM, 2012.","title":"Robust multi-task feature learning","authors":"Gong","year":"2012","venue":"Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Grave et al., 2011] Edouard Grave, Guillaume R Obozin- ski, and Francis R Bach. Trace lasso: a trace norm reg- ularization for correlated designs. In Advances in Neural Information Processing Systems, pages 2187-2195, 2011.","title":"Trace lasso: a trace norm regularization for correlated designs","authors":"Grave","year":"2011","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He and Lawrence, 2011] Jingrui He and Rick Lawrence. A graph-based framework for multi-task multi-view learn- ing. Proceedings of the 28th International Conference on Machine Learning, ICML 2011, pages 25-32, 2011.","title":"Jingrui He and Rick Lawrence. A graph-based framework for multi-task multi-view learning","authors":"Lawrence He","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning, ICML 2011","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hoerl and Kennard, 1970] Arthur E Hoerl and Robert W Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55-67, 1970.","title":"Ridge regression: Biased estimation for nonorthogonal problems","authors":"E Arthur, Robert W Hoerl, Kennard","year":"1970","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huiskes and Lew, 2008] Mark J. Huiskes and Michael S. Lew. The mir flickr retrieval evaluation. In MIR '08: Pro- ceedings of the 2008 ACM International Conference on Multimedia Information Retrieval, New York, NY, USA, 2008. ACM.","title":"The mir flickr retrieval evaluation","authors":"Mark J Huiskes, Michael S Lew","year":"2008","venue":"MIR '08: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jin et al., 2013] Xin Jin, Fuzhen Zhuang, Shuhui Wang, Qing He, and Zhongzhi Shi. Shared Structure Learn- ing for Multiple Tasks with Multiple Views. In Hendrik Blockeel, Kristian Kersting, Siegfried Nijssen, and Filip Železný, editors, Machine Learning and Knowledge Dis- covery in Databases, pages 353-368, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg.","title":"Shared Structure Learning for Multiple Tasks with Multiple Views","authors":"Xin Jin, Fuzhen Zhuang, Shuhui Wang, Qing He, Zhongzhi Shi","year":"2013","venue":"Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jin et al., 2014] Xin Jin, Fuzhen Zhuang, Hui Xiong, Changying Du, Ping Luo, and Qing He. Multi-task Multi- view Learning for Heterogeneous Tasks. Proc.CIKM, 3:441-450, 2014.","title":"Multi-task Multiview Learning for Heterogeneous Tasks","authors":"Xin Jin, Fuzhen Zhuang, Hui Xiong, Changying Du, Ping Luo, Qing He","year":"2014","venue":"Proc.CIKM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2009] Jun Liu, Shuiwang Ji, and Jieping Ye. Multi-task feature learning via efficient l2, 1-norm mini- mization. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI '09, pages 339-348, Arlington, Virginia, United States, 2009. AUAI Press.","title":"Multi-task feature learning via efficient l2, 1-norm minimization","authors":"Liu","year":"2009","venue":"Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI '09","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lu et al., 2017] Chun-Ta Lu, Lifang He, Weixiang Shao, Bokai Cao, and Philip S. Yu. Multilinear Factorization Machines for Multi-Task Multi-View Learning. In Pro- ceedings of the Tenth ACM International Conference on Web Search and Data Mining -WSDM '17, pages 701- 709, 2017.","title":"Multilinear Factorization Machines for Multi-Task Multi-View Learning","authors":"Chun-Ta Lu, Lifang He, Weixiang Shao, Bokai Cao, Philip S Yu","year":"2017","venue":"Proceedings of the Tenth ACM International Conference on Web Search and Data Mining -WSDM '17","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nesterov, 2013] Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013.","title":"Introductory lectures on convex optimization: A basic course","authors":"Yurii Nesterov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Qian and Zhai, 2014] Mingjie Qian and Chengxiang Zhai. Unsupervised feature selection for multi-view clustering on text-image web news data. In Proceedings of the 23rd ACM international conference on conference on in- formation and knowledge management, pages 1963-1966. ACM, 2014.","title":"Unsupervised feature selection for multi-view clustering on text-image web news data","authors":"Mingjie Qian, Chengxiang Zhai","year":"2014","venue":"Proceedings of the 23rd ACM international conference on conference on information and knowledge management","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Recht et al., 2010] Benjamin Recht, Maryam Fazel, and Pablo A Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM review, 52(3):471-501, 2010.","title":"Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization","authors":"Benjamin Recht, Maryam Fazel, Pablo A Parrilo","year":"2010","venue":"SIAM review","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sindhwani et al., 2005] Vikas Sindhwani, Partha Niyogi, and Mikhail Belkin. A co-regularization approach to semi- supervised learning with multiple views. In Proceedings of ICML workshop on learning with multiple views, volume 2005, pages 74-79. Citeseer, 2005.","title":"A co-regularization approach to semisupervised learning with multiple views","authors":"Vikas Sindhwani, Partha Niyogi, Mikhail Belkin","year":"2005","venue":"Proceedings of ICML workshop on learning with multiple views","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tibshirani, 1996] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267-288, 1996.","title":"Regression shrinkage and selection via the lasso","authors":"Robert Tibshirani","year":"1996","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang and Huan, 2012] Jintao Zhang and Jun Huan. Induc- tive multi-task learning with multiple view data. In Pro- ceedings of the 18th ACM SIGKDD international confer- ence on Knowledge discovery and data mining -KDD '12, page 543, 2012.","title":"Inductive multi-task learning with multiple view data","authors":"Huan Zhang, Jintao Zhang, Jun Huan","year":"2012","venue":"Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining -KDD '12","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhou et al., 2018] D. Zhou, J. Wang, B. Jiang, H. Guo, and Y. Li. Multi-task multi-view learning based on coopera- tive multi-objective optimization. IEEE Access, 6:19465- 19477, 2018.","title":"Multi-task multi-view learning based on cooperative multi-objective optimization","authors":"D Zhou, J Wang, B Jiang, H Guo, Y Li","year":"2018","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zou and Hastie, 2005] Hui Zou and Trevor Hastie. Regular- ization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Method- ology), 67(2):301-320, 2005.","title":"Regularization and variable selection via the elastic net","authors":"Hastie Zou, Hui Zou, Trevor Hastie","year":"2005","venue":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}