{"type":"ScholarlyArticle","title":"Exploiting High-Order Information in Heterogeneous Multi-Task Feature Learning","authors":[{"name":"Yong Luo","slug":"yong-luo","given":"Yong","family":"Luo","affiliation":"† School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Dacheng Tao","slug":"dacheng-tao","given":"Dacheng","family":"Tao","affiliation":"‡ UBTech Sydney AI Institute and SIT , FEIT , The University of Sydney , Australia"},{"name":"Yonggang Wen","slug":"yonggang-wen","given":"Yonggang","family":"Wen","affiliation":"† School of Computer Science and Engineering , Nanyang Technological University , Singapore"}],"abstract":"Multi-task feature learning (MTFL) aims to improve the generalization performance of multiple related learning tasks by sharing features between them. It has been successfully applied to many pattern recognition and biometric prediction problems. Most of current MTFL methods assume that different tasks exploit the same feature representation, and thus are not applicable to the scenarios where data are drawn from heterogeneous domains. Existing heterogeneous transfer learning (including multi-task learning) approaches handle multiple heterogeneous domains by usually learning feature transformations across different domains, but they ignore the high-order statistics (correlation information) which can only be discovered by simultaneously exploring all domains. We therefore develop a tensor based heterogeneous MTFL (THMTFL) framework to exploit such high-order information. Specifically, feature transformations of all domains are learned together, and finally used to derive new representations. A connection between all domains is built by using the transformations to project the pre-learned predictive structures of different domains into a common subspace, and minimizing their divergence in the subspace. By exploring the high-order information, the proposed THMTFL can obtain more reliable feature transformations compared with existing heterogeneous transfer learning approaches. Extensive experiments on both text categorization and social image annotation demonstrate superiority of the proposed method.","doi":"10.24963/ijcai.2017/340","url":"https://ijcai.org/proceedings/2017/340","pdf":"https://ijcai.org/proceedings/2017/0340.pdf","markdown":"https://ijcai.org/proceedings/2017/340.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/340","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":340,"pages":{"first":2443,"last":2449},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Amini et al., 2009] Massih Amini, Nicolas Usunier, and Cyril Goutte. Learning from multiple partially observed views-an application to multilingual text categorization. In NIPS, pages 28-36, 2009.","title":"Learning from multiple partially observed views-an application to multilingual text categorization","authors":"Massih Amini, Nicolas Usunier, Cyril Goutte","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ammar et al., 2015] Haitham Bou Ammar, Eric Eaton, José Marcio Luna, and Paul Ruvolo. Autonomous cross- domain knowledge transfer in lifelong policy gradient re- inforcement learning. In IJCAI, pages 3345-3351, 2015.","title":"Autonomous crossdomain knowledge transfer in lifelong policy gradient reinforcement learning","authors":"Haitham Bou Ammar, Eric Eaton, José, Marcio Luna, Paul Ruvolo","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ando and Zhang, 2005] Rie Kubota Ando and Tong Zhang. A framework for learning predictive structures from mul- tiple tasks and unlabeled data. JMLR, 6:1817-1853, 2005.","title":"A framework for learning predictive structures from multiple tasks and unlabeled data","authors":"Rie Kubota, Ando, Tong Zhang","year":"2005","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Argyriou et al., 2008] A. Argyriou, T. Evgeniou, and M. Pontil. Convex multi-task feature learning. Machine Learning, 73(3):243-272, 2008.","title":"Convex multi-task feature learning","authors":"A Argyriou, T Evgeniou, M Pontil","year":"2008","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Caruana, 1997] R. Caruana. Multitask learning. Machine Learning, 28(1):41-75, 1997.","title":"Multitask learning","authors":"R Caruana","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. NUS- WIDE: a real-world web image database from national u- niversity of singapore. In CIVR, 2009.","title":"NUS-WIDE: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"CIVR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[De Lathauwer et al., 2000] Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. On the best rank-1 and rank-(r1, r2, ..., rn) approximation of higher-order tensors. SIAM JMAA, 21(4):1324-1342, 2000.","title":"On the best rank-1 and rank-(r1, r2, ..., rn) approximation of higher-order tensors","authors":"Lieven De Lathauwer, Bart De Moor, Joos Vandewalle","year":"2000","venue":"SIAM JMAA","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dietterich and Bakiri, 1995] Thomas G. Dietterich and Ghulum Bakiri. Solving multiclass learning problems via error-correcting output codes. JAIR, 2:263-286, 1995.","title":"Solving multiclass learning problems via error-correcting output codes","authors":"Thomas G Dietterich, Ghulum Bakiri","year":"1995","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gonc ¸alves et al., 2017] André R Gonc ¸alves, Arindam Banerjee, and Fernando J Von Zuben. Spatial projection of multiple climate variables using hierarchical multitask learning. In AAAI, 2017.","title":"Spatial projection of multiple climate variables using hierarchical multitask learning","authors":"Arindam André R Gonc ¸alves, Fernando J Von Banerjee, Zuben","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gong et al., 2013] Pinghua Gong, Jieping Ye, and Chang- shui Zhang. Multi-stage multi-task feature learning. JML- R, 14(1):2979-3010, 2013.","title":"Pinghua Gong, Jieping Ye, and Changshui Zhang. Multi-stage multi-task feature learning","authors":"Gong","year":"2013","venue":"JML-R","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jin et al., 2015] Xin Jin, Fuzhen Zhuang, Sinno Jialin Pan, Changying Du, Ping Luo, and Qing He. Heterogeneous multi-task semantic feature learning for classification. In CIKM, pages 1847-1850, 2015.","title":"Heterogeneous multi-task semantic feature learning for classification","authors":"Xin Jin, Fuzhen Zhuang, Sinno Jialin Pan, Changying Du, Ping Luo, Qing He","year":"2015","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin, 2007] Chih-Jen Lin. Projected gradient methods for nonnegative matrix factorization. Neural Computation, 19(10):2756-2779, 2007.","title":"Projected gradient methods for nonnegative matrix factorization","authors":"Chih-Jen Lin","year":"2007","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu and Tao, 2016] Tongliang Liu and Dacheng Tao. Clas- sification with noisy labels by importance reweighting. IEEE TPAMI, 38(3):447-461, 2016.","title":"Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting","authors":"Tao Liu","year":"2016","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2009] Jun Liu, Shuiwang Ji, and Jieping Ye. Multi-task feature learning via efficient l 2, 1-norm min- imization. In UAI, pages 339-348, 2009.","title":"Multi-task feature learning via efficient l 2, 1-norm minimization","authors":"Liu","year":"2009","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2017] Tongliang Liu, Dacheng Tao, Mingli Song, and Stephen J Maybank. Algorithm-dependent gener- alization bounds for multi-task learning. IEEE TPAMI, 39(2):227-241, 2017.","title":"Algorithm-dependent generalization bounds for multi-task learning","authors":"Liu","year":"2017","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lowe, 2004] David G Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91-110, 2004.","title":"Distinctive image features from scale-invariant keypoints","authors":"David G Lowe","year":"2004","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Luo et al., 2013] Yong Luo, Dacheng Tao, Bo Geng, Chao Xu, and Stephen J Maybank. Manifold regularized multi- task learning for semi-supervised multilabel image classi- fication. IEEE TIP, 22(2):523-536, 2013.","title":"Manifold regularized multitask learning for semi-supervised multilabel image classification","authors":"Yong Luo, Dacheng Tao, Bo Geng, Chao Xu, Stephen J Maybank","year":"2013","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Luo et al., 2015] Yong Luo, Dacheng Tao, Kotagiri Ra- mamohanarao, Chao Xu, and Yonggang Wen. Tensor canonical correlation analysis for multi-view dimension reduction. IEEE TKDE, 27(11):3111-3124, 2015.","title":"Tensor canonical correlation analysis for multi-view dimension reduction","authors":"Yong Luo, Dacheng Tao, Kotagiri Ramamohanarao, Chao Xu, Yonggang Wen","year":"2015","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Luo et al., 2016] Yong Luo, Yonggang Wen, Dacheng Tao, Jie Gui, and Chao Xu. Large margin multi-modal multi- task feature extraction for image classification. IEEE TIP, 25(1):414-427, 2016.","title":"Large margin multi-modal multitask feature extraction for image classification","authors":"Yong Luo, Yonggang Wen, Dacheng Tao, Jie Gui, Chao Xu","year":"2016","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nesterov, 2005] Yu Nesterov. Smooth minimization of non-smooth functions. Mathematical Programming, 103(1):127-152, 2005.","title":"Smooth minimization of non-smooth functions","authors":"Yu Nesterov","year":"2005","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Quattoni et al., 2007] Ariadna Quattoni, Michael Collins, and Trevor Darrell. Learning visual representations using images with captions. In IEEE CVPR, pages 1-8, 2007.","title":"Learning visual representations using images with captions","authors":"Ariadna Quattoni, Michael Collins, Trevor Darrell","year":"2007","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sokolova and Lapalme, 2009] Marina Sokolova and Guy Lapalme. A systematic analysis of performance measures for classification tasks. Information Processing and Man- agement, 45(4):427-437, 2009.","title":"A systematic analysis of performance measures for classification tasks","authors":"Lapalme Sokolova, Marina Sokolova, Guy Lapalme","year":"2009","venue":"Information Processing and Management","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tao et al., 2007a] Dacheng Tao, Xuelong Li, Xindong Wu, Weiming Hu, and Stephen J Maybank. Supervised tensor learning. KIS, 13(1):1-42, 2007.","title":"Supervised tensor learning","authors":"Tao","year":"2007","venue":"KIS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tao et al., 2007b] Dacheng Tao, Xuelong Li, Xindong Wu, and Stephen J Maybank. General tensor discriminant anal- ysis and gabor features for gait recognition. IEEE TPAMI, 29(10):1700-1715, 2007.","title":"General tensor discriminant analysis and gabor features for gait recognition","authors":"Tao","year":"2007","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang and Mahadevan, 2011] Chang Wang and Sridhar Ma- hadevan. Heterogeneous domain adaptation using mani- fold alignment. In IJCAI, pages 1541-1546, 2011.","title":"Heterogeneous domain adaptation using manifold alignment","authors":"Chang Wang, Sridhar Mahadevan","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/259"},{"position":26,"raw":"[Wang et al., 2007] Huan Wang, Shuicheng Yan, Dong Xu, Xiaoou Tang, and Thomas Huang. Trace ratio vs. ratio trace for dimensionality reduction. In IEEE CVPR, pages 1-8, 2007.","title":"Trace ratio vs. ratio trace for dimensionality reduction","authors":"Huan Wang, Shuicheng Yan, Dong Xu, Xiaoou Tang, Thomas Huang","year":"2007","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2016] Xuezhi Wang, Junier B Oliva, Jef- f Schneider, and Barnabás Póczos. Nonparametric risk and stability analysis for multi-task learning problems. In IJ- CAI, pages 2146-2152, 2016.","title":"Nonparametric risk and stability analysis for multi-task learning problems","authors":"Wang","year":"2016","venue":"IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2016/306"},{"position":28,"raw":"[Xu et al., 2014] Chang Xu, Dacheng Tao, and Chao Xu. Large-margin multi-view information bottleneck. IEEE T- PAMI, 36(8):1559-1572, 2014.","title":"Large-margin multi-view information bottleneck","authors":"Chang Xu, Dacheng Tao, Chao Xu","year":"2014","venue":"IEEE T-PAMI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xu et al., 2015] Chang Xu, Dacheng Tao, and Chao X- u. Multi-view intact space learning. IEEE TPAMI, 37(12):2531-2544, 2015.","title":"Multi-view intact space learning","authors":"Chang Xu, Dacheng Tao, Chao Xu","year":"2015","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang and Yeung, 2011] Yu Zhang and Dit-Yan Yeung. Multi-task learning in heterogeneous feature spaces. In AAAI, pages 574-579, 2011.","title":"Multi-task learning in heterogeneous feature spaces","authors":"Yeung Zhang, Yu Zhang, Dit-Yan Yeung","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhou et al., 2010] Tianyi Zhou, Dacheng Tao, and Xindong Wu. Nesvm: A fast gradient method for support vector machines. In IEEE ICDM, pages 679-688, 2010.","title":"Nesvm: A fast gradient method for support vector machines","authors":"Zhou","year":"2010","venue":"IEEE ICDM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}