{"type":"ScholarlyArticle","title":"Group LASSO with Asymmetric Structure Estimation for Multi-Task Learning","authors":[{"name":"Saullo H. G. Oliveira","slug":"saullo-h-g-oliveira","given":"Saullo H. G.","family":"Oliveira","affiliation":"School of Electrical and Computer Engineering -FEEC , University of Campinas -Unicamp , Brazil"},{"name":"André R. Gonçalves","slug":"andre-r-goncalves","given":"André  R.","family":"Gonçalves","affiliation":"Lawrence Livermore National Laboratory , USA"},{"name":"Fernando J. Von Zuben","slug":"fernando-j-von-zuben","given":"Fernando J.","family":"Von Zuben","affiliation":"School of Electrical and Computer Engineering -FEEC , University of Campinas -Unicamp , Brazil"}],"abstract":"Group LASSO is a widely used regularization that imposes sparsity considering groups of covariates. When used in Multi-Task Learning (MTL) formulations, it makes an underlying assumption that if one group of covariates is not relevant for one or a few tasks, it is also not relevant for all tasks, thus implicitly assuming that all tasks are related.\r\nThis implication can easily lead to negative transfer if this assumption does not hold for all tasks.\r\nSince for most practical applications we hardly know a priori how the tasks are related, several approaches have been conceived in the literature to (i) properly capture the transference structure, (ii) improve interpretability of the tasks interplay, and (iii) penalize potential negative transfer.\r\nRecently, the automatic estimation of asymmetric structures inside the learning process was capable of effectively avoiding negative transfer.\r\nOur proposal is the first attempt in the literature to conceive a Group LASSO with asymmetric transference formulation, looking for the best of both worlds in a  framework that admits the overlap of groups.\r\nThe resulting optimization problem is solved by an alternating procedure with fast methods.\r\nWe performed experiments using synthetic and real datasets to compare our proposal with state-of-the-art approaches, evidencing the promising predictive performance and distinguished interpretability of our proposal.\r\nThe real case study involves the prediction of cognitive scores for  Alzheimer's disease progression assessment.\r\nThe source codes are available at GitHub.","doi":"10.24963/ijcai.2019/444","url":"https://ijcai.org/proceedings/2019/444","pdf":"https://ijcai.org/proceedings/2019/0444.pdf","markdown":"https://ijcai.org/proceedings/2019/444.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/444","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":444,"pages":{"first":3202,"last":3208},"keywords":["Machine Learning: Feature Selection ; Learning Sparse Models","Machine Learning Applications: Other Applications","Machine Learning: Multi-instance;Multi-label;Multi-view learning","Machine Learning: Transfer, Adaptation, Multi-task Learning"],"videos":[],"references":[{"position":1,"raw":"[Argyriou et al., 2008] Andreas Argyriou, Theodoros Evge- niou, and Massimiliano Pontil. Convex multi-task feature learning. Machine Learning, 73(3):243-272, 2008.","title":"Convex multi-task feature learning","authors":"Andreas Argyriou, Theodoros Evgeniou, Massimiliano Pontil","year":"2008","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baxter, 1997] Jonathan Baxter. A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling. Machine Learning, 28(1):7-39, 1997.","title":"A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling","authors":"Jonathan Baxter","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beck and Teboulle, 2009] Amir Beck and Marc Teboulle. A Fast Iterative Shrinkage-Thresholding Algorithm for Lin- ear Inverse Problems. SIAM Journal on Imaging Sciences, 2(1):183-202, 2009.","title":"A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems","authors":"Amir Beck, Marc Teboulle","year":"2009","venue":"SIAM Journal on Imaging Sciences","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boyd et al., 2011] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed Opti- mization and Statistical Learning via the Alternating Di- rection Method of Multipliers. Found. Trends Mach. Learn., 3(1), 2011.","title":"Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers","authors":"Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein","year":"2011","venue":"Found. Trends Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Caruana, 1997] Rich Caruana. Multitask learning. Machine Learning, 28(1), July 1997.","title":"Multitask learning","authors":"Rich Caruana","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gonc ¸alves et al., 2016] André R. Gonc ¸alves, Fernando J. Von Zuben, and Arindam Banerjee. Multi-task Sparse Structure Learning with Gaussian Copula Models. Journal of Machine Learning Research, 17(33):1-30, 2016.","title":"Multi-task Sparse Structure Learning with Gaussian Copula Models","authors":"R André, Fernando J Gonc, Arindam Von Zuben, Banerjee","year":"2016","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jacob et al., 2009] Laurent Jacob, Guillaume Obozinski, and Jean-Philippe Vert. Group Lasso with Overlap and Graph Lasso. In International Conference on Machine Learning, pages 433-440, 2009.","title":"Group Lasso with Overlap and Graph Lasso","authors":"Laurent Jacob, Guillaume Obozinski, Jean-Philippe Vert","year":"2009","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kang et al., 2011] Zhuoliang Kang, Kristen Grauman, and Fei Sha. Learning with whom to share in multi-task feature learning. In International Conference on Machine Learn- ing, pages 521-528, 2011.","title":"Learning with whom to share in multi-task feature learning","authors":"Kang","year":"2011","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kumar and Daumé, 2012] Abhishek Kumar and Hal Daumé. Learning task grouping and overlap in multi- task learning. In International Conference on Machine Learning, 2012.","title":"Learning task grouping and overlap in multitask learning","authors":"Daumé Kumar, Abhishek Kumar, Hal Daumé","year":"2012","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lee et al., 2016] Giwoong Lee, Eunho Yang, and Sung Ju Hwang. Asymmetric Multi-task Learning Based on Task Relatedness and Loss. In International Conference on Ma- chine Learning, pages 230-238, 2016.","title":"Asymmetric Multi-task Learning Based on Task Relatedness and Loss","authors":"Lee","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Leung et al., 2013] Kelvin K. Leung, Jonathan W. Bartlett, Josephine Barnes, Emily N. Manning, Sebastien Ourselin, and Nick C. and Fox. Cerebral atrophy in mild cog- nitive impairment and Alzheimer disease. Neurology, 80(7):648-654, 2013.","title":"Cerebral atrophy in mild cognitive impairment and Alzheimer disease","authors":"Kelvin K Leung, Jonathan W Bartlett, Josephine Barnes, Emily N Manning, Sebastien Ourselin, Nick C Fox","year":"2013","venue":"Neurology","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2009] Jun Liu, Shuiwang Ji, and Jieping Ye. Multi-task feature learning via efficient l2,1-norm mini- mization. In Conference on Uncertainty in Artificial Intel- ligence, pages 339-348, 2009.","title":"Multi-task feature learning via efficient l2,1-norm minimization","authors":"Liu","year":"2009","venue":"Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2018] Xiaoli Liu, André R. Goncalves, Peng Cao, Dazhe Zhao, and Arindam Banerjee. Modeling Alzheimer's Disease Cognitive Scores Using Multi-Task Sparse Group Lasso. Computerized Medical Imaging and Graphics, 66:100 -114, 2018.","title":"Modeling Alzheimer's Disease Cognitive Scores Using Multi-Task Sparse Group Lasso","authors":"Xiaoli Liu, André R Goncalves, Peng Cao, Dazhe Zhao, Arindam Banerjee","year":"2018","venue":"Computerized Medical Imaging and Graphics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Teipel et al., 2002] Stefan J. Teipel, Wolfram Bayer, Gene E. Alexander, York Zebuhr, Diane Teichberg, Luka Kulic, Marc B. Schapiro, Hans-Jürgen Möller, Stanley I. Rapoport, and Harald Hampel. Progression of Corpus Callosum Atrophy in Alzheimer Disease. Archives of Neurology, 59(2):243-248, 02 2002.","title":"Progression of Corpus Callosum Atrophy in Alzheimer Disease","authors":"Stefan J Teipel, Wolfram Bayer, Gene E Alexander, York Zebuhr, Diane Teichberg, Luka Kulic, Marc B Schapiro, Hans-Jürgen Möller, Stanley I Rapoport, Harald Hampel","year":"2002","venue":"Archives of Neurology","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Thrun and O'Sullivan, 1996] Sebastian Thrun and Joseph O'Sullivan. Discovering Structure in Multiple Learning Tasks: The TC Algorithm. In International Conference on Machine Learning, pages 489-497, 1996.","title":"Discovering Structure in Multiple Learning Tasks: The TC Algorithm","authors":"O'sullivan ; Thrun, Sebastian Thrun, Joseph O' Sullivan","year":"1996","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tibshirani, 1996] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1):267-288, 1996.","title":"Regression shrinkage and selection via the lasso","authors":"Robert Tibshirani","year":"1996","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2012] Hua Wang, Feiping Nie, Heng Huang, Sungeun Kim, Kwangsik Nho, Shannon L. Risacher, An- drew J. Saykin, Li Shen, and For the Alzheimer's Dis- ease Neuroimaging Initiative. Identifying quantitative trait loci via group-sparse multitask regression and feature se- lection: an imaging genetics study of the ADNI cohort. Bioinformatics, 28(2):229-237, 2012.","title":"For the Alzheimer's Disease Neuroimaging Initiative. Identifying quantitative trait loci via group-sparse multitask regression and feature selection: an imaging genetics study of the ADNI cohort","authors":"Hua Wang, Feiping Nie, Heng Huang, Sungeun Kim, Kwangsik Nho, Shannon L Risacher, Andrew J Saykin, Li Shen","year":"2012","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yuan and Lin, 2006] Ming Yuan and Yi Lin. Model selec- tion and estimation in regression with grouped variables. Journal of the Royal Statistical Society. Series B (Statisti- cal Methodology), 68:49-67, 2006.","title":"Model selection and estimation in regression with grouped variables","authors":"Lin ; Yuan, Ming Yuan, Yi Lin","year":"2006","venue":"Journal of the Royal Statistical Society. Series B (Statistical Methodology)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhang and Yeung, 2010] Yu Zhang and Dit-Yan Yeung. A convex formulation for learning task relationships in multi-task learning. In Conference on Uncertainty in Arti- ficial Intelligence, pages 733-742, 2010.","title":"A convex formulation for learning task relationships in multi-task learning","authors":"Yeung ; ] Yu Zhang, Dit-Yan Zhang, Yeung","year":"2010","venue":"Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang, 2013] Yu Zhang. Heterogeneous-neighborhood- based multi-task local learning algorithms. In Advances in Neural Information Processing Systems, pages 1896- 1904, 2013.","title":"Heterogeneous-neighborhoodbased multi-task local learning algorithms","authors":"Yu Zhang","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zou, 2006] Hui Zou. The adaptive lasso and its oracle prop- erties. Journal of the American Statistical Association, 101:1418-1429, 2006.","title":"The adaptive lasso and its oracle properties","authors":"Hui Zou","year":"2006","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}