{"type":"ScholarlyArticle","title":"Adaptively Unified Semi-supervised Learning for Cross-Modal Retrieval","authors":[{"name":"Liang Zhang","slug":"liang-zhang","given":"Liang","family":"Zhang","affiliation":"University of Chinese Academy of Sciences , Beijing , 100049 , China; Key Laboratory of Big Data Mining and Knowledge Management , CAS , Beijing , 100190 , China"},{"name":"Bingpeng Ma","slug":"bingpeng-ma","given":"Bingpeng","family":"Ma","affiliation":"University of Chinese Academy of Sciences , Beijing , 100049 , China; Key Lab of Intell. Info. Process. , Inst. of Comput. Tech. , CAS , Beijing , 100190 , China; Key Laboratory of Big Data Mining and Knowledge Management , CAS , Beijing , 100190 , China"},{"name":"Jianfeng He","slug":"jianfeng-he","given":"Jianfeng","family":"He","affiliation":"University of Chinese Academy of Sciences , Beijing , 100049 , China; Key Laboratory of Big Data Mining and Knowledge Management , CAS , Beijing , 100190 , China"},{"name":"Guorong Li","slug":"guorong-li","given":"Guorong","family":"Li","affiliation":"University of Chinese Academy of Sciences , Beijing , 100049 , China; Key Laboratory of Big Data Mining and Knowledge Management , CAS , Beijing , 100190 , China"},{"name":"Qingming Huang","slug":"qingming-huang","given":"Qingming","family":"Huang","affiliation":"University of Chinese Academy of Sciences , Beijing , 100049 , China; Key Lab of Intell. Info. Process. , Inst. of Comput. Tech. , CAS , Beijing , 100190 , China; Key Laboratory of Big Data Mining and Knowledge Management , CAS , Beijing , 100190 , China"},{"name":"Qi Tian","slug":"qi-tian","given":"Qi","family":"Tian","affiliation":"Department of Computer Science , University of Texas at San Antonio , TX , 78249 , USA"}],"abstract":"Motivated by the fact that both relevancy of class labels and unlabeled data can help to strengthen multi-modal correlation, this paper proposes a novel method for cross-modal retrieval. To make each sample moving to the direction of its relevant label while far away from that of its irrelevant ones, a novel dragging technique is fused into a unified linear regression model. By this way, not only the relation between embedded features and relevant class labels but also the relation between embedded features and irrelevant class labels can be exploited. Moreover, considering that some unlabeled data contain specific semantic information, a weighted regression model is designed to adaptively enlarge their contribution while weaken that of the unlabeled data with non-specific semantic information. Hence, unlabeled data can supply semantic information to enhance discriminant ability of classifier. Finally, we integrate the constraints into a joint minimization formulation and develop an efficient optimization algorithm to learn a discriminative common subspace for different modalities. Experimental results on Wiki, Pascal and NUS-WIDE datasets show that the proposed method outperforms the state-of-the-art methods even when we set 20% samples without class labels.","doi":"10.24963/ijcai.2017/476","url":"https://ijcai.org/proceedings/2017/476","pdf":"https://ijcai.org/proceedings/2017/0476.pdf","markdown":"https://ijcai.org/proceedings/2017/476.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/476","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":476,"pages":{"first":3406,"last":3412},"keywords":["Machine Learning: Multi-instance/Multi-label/Multi-view learning","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff A. Bilmes, and Karen Livescu. Deep canonical correlation analysis. In Proceedings of the 30th International Confer- ence on Machine Learning, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Jeff A Bilmes, Karen Livescu","year":"2013","venue":"Proceedings of the 30th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. NUS- WIDE: a real-world web image database from national u- niversity of singapore. In Proceedings of the 8th ACM International Conference on Image and Video Retrieval, 2009.","title":"NUS-WIDE: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"Proceedings of the 8th ACM International Conference on Image and Video Retrieval","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Everingham et al., 2010] Mark Everingham, Luc J. Van Gool, Christopher K. I. Williams, John M. Winn, and An- drew Zisserman. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2):303-338, 2010.","title":"Andrew Zisserman. The pascal visual object classes (VOC) challenge","authors":"Mark Everingham, J Luc, Van Gool, K I Christopher, John M Williams, Winn","year":"2010","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gong et al., 2016] Chen Gong, Dacheng Tao, Stephen J. Maybank, Wei Liu, Guoliang Kang, and Jie Yang. Multi-modal curriculum learning for semi-supervised im- age classification. IEEE Trans. Image Processing, 25(7):3249-3260, 2016.","title":"Multi-modal curriculum learning for semi-supervised image classification","authors":"Chen Gong, Dacheng Tao, Stephen J Maybank, Wei Liu, Guoliang Kang, Jie Yang","year":"2016","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hardoon et al., 2004] David R. Hardoon, Sándor Szedmák, and John Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. Neural Computation, 16(12):2639-2664, 2004.","title":"Canonical correlation analysis: An overview with application to learning methods","authors":"David R Hardoon, Sándor Szedmák, John Shawe-Taylor","year":"2004","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jia et al., 2014] Yangqing Jia, Evan Shelhamer, Jeff Don- ahue, Sergey Karayev, Jonathan Long, Ross B. Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolu- tional architecture for fast feature embedding. In Proceed- ings of the ACM International Conference on Multimedia, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross B Girshick, Sergio Guadarrama, Trevor Darrell","year":"2014","venue":"Proceedings of the ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kan et al., 2016] Meina Kan, Shiguang Shan, Haihong Zhang, Shihong Lao, and Xilin Chen. Multi-view discrim- inant analysis. IEEE Trans. Pattern Anal. Mach. Intell., 38(1):188-194, 2016.","title":"Multi-view discriminant analysis","authors":"Meina Kan, Shiguang Shan, Haihong Zhang, Shihong Lao, Xilin Chen","year":"2016","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kang et al., 2015] Cuicui Kang, Shiming Xiang, Shengcai Liao, Changsheng Xu, and Chunhong Pan. Learning con- sistent feature representation for cross-modal multimedia retrieval. IEEE Trans. Multimedia, 17(3):370-381, 2015.","title":"Learning consistent feature representation for cross-modal multimedia retrieval","authors":"Kang","year":"2015","venue":"IEEE Trans. Multimedia","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li and Tang, 2017] Zechao Li and Jinhui Tang. Weakly su- pervised deep matrix factorization for social image under- standing. IEEE Trans. Image Processing, 26(1):276-288, 2017.","title":"Weakly supervised deep matrix factorization for social image understanding","authors":"Li, Tang, Zechao Li, Jinhui Tang","year":"2017","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2015] Zechao Li, Jing Liu, Jinhui Tang, and Han- qing Lu. Robust structured subspace learning for data representation. IEEE Trans. Pattern Anal. Mach. Intell., 37(10):2085-2098, 2015.","title":"Robust structured subspace learning for data representation","authors":"Zechao Li, Jing Liu, Jinhui Tang, Hanqing Lu","year":"2015","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(11):2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their composi- tionality. In Advances in Neural Information Processing Systems, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ngiam et al., 2011] Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y. Ng. Mul- timodal deep learning. In Proceedings of the 28th Interna- tional Conference on Machine Learning, 2011.","title":"Multimodal deep learning","authors":"Ngiam","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Peng et al., 2016] Yuxin Peng, Xiaohua Zhai, Yunzhen Zhao, and Xin Huang. Semi-supervised cross-media fea- ture learning with unified patch graph regularization. IEEE Trans. Circuits Syst. Video Techn., 26(3):583-596, 2016.","title":"Semi-supervised cross-media feature learning with unified patch graph regularization","authors":"Yuxin Peng, Xiaohua Zhai, Yunzhen Zhao, Xin Huang","year":"2016","venue":"IEEE Trans. Circuits Syst. Video Techn","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ranjan et al., 2015] Viresh Ranjan, Nikhil Rasiwasia, and C. V. Jawahar. Multi-label cross-modal retrieval. In IEEE International Conference on Computer Vision, 2015.","title":"Multi-label cross-modal retrieval","authors":"Nikhil Viresh Ranjan, C V Rasiwasia, Jawahar","year":"2015","venue":"IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rasiwasia et al., 2010] Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert R. G. Lanckriet, Roger Levy, and Nuno Vasconcelos. A new approach to cross-modal multimedia retrieval. In Proceed- ings of the 18th International Conference on Multimedia 2010, 2010.","title":"A new approach to cross-modal multimedia retrieval","authors":"Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert R G Lanckriet, Roger Levy, Nuno Vasconcelos","year":"2010","venue":"Proceedings of the 18th International Conference on Multimedia 2010","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rasiwasia et al., 2014] Nikhil Rasiwasia, Dhruv Mahajan, Vijay Mahadevan, and Gaurav Aggarwal. Cluster canoni- cal correlation analysis. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and S- tatistics, 2014.","title":"Cluster canonical correlation analysis","authors":"Nikhil Rasiwasia, Dhruv Mahajan, Vijay Mahadevan, Gaurav Aggarwal","year":"2014","venue":"Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sharma et al., 2012] Abhishek Sharma, Abhishek Kumar, Hal Daumé III, and David W. Jacobs. Generalized mul- tiview analysis: A discriminative latent space. In IEEE Conference on Computer Vision and Pattern Recognition, 2012.","title":"Generalized multiview analysis: A discriminative latent space","authors":"Abhishek Sharma, Abhishek Kumar, Hal Daumé, Iii, David W Jacobs","year":"2012","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Srivastava and Salakhutdinov, 2012] Nitish Srivastava and Ruslan Salakhutdinov. Multimodal learning with deep boltzmann machines. In Advances in Neural Information Processing Systems, 2012.","title":"Multimodal learning with deep boltzmann machines","authors":"Nitish Srivastava, Ruslan Salakhutdinov","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2013] Kaiye Wang, Ran He, Wei Wang, Liang Wang, and Tieniu Tan. Learning coupled feature spaces for cross-modal matching. In IEEE International Conference on Computer Vision, 2013.","title":"Learning coupled feature spaces for cross-modal matching","authors":"Wang","year":"2013","venue":"IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2014] De Wang, Feiping Nie, and Heng Huang. Large-scale adaptive semi-supervised learning via unified inductive and transductive model. In The 20th ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining, 2014.","title":"Large-scale adaptive semi-supervised learning via unified inductive and transductive model","authors":"De Wang, Feiping Nie, Heng Huang","year":"2014","venue":"The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xiang et al., 2012] Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, and Changshui Zhang. Discrimi- native least squares regression for multiclass classification and feature selection. IEEE Trans. Neural Netw. Learning Syst., 23(11):1738-1754, 2012.","title":"Discriminative least squares regression for multiclass classification and feature selection","authors":"Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, Changshui Zhang","year":"2012","venue":"IEEE Trans. Neural Netw. Learning Syst","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2015] Xing Xu, Yang Yang, Atsushi Shimada, Rin-ichiro Taniguchi, and Li He. Semi-supervised cou- pled dictionary learning for cross-modal retrieval in inter- net images and texts. In Proceedings of the 23rd Annual ACM Conference on Multimedia Conference, 2015.","title":"Semi-supervised coupled dictionary learning for cross-modal retrieval in internet images and texts","authors":"Xing Xu, Yang Yang, Atsushi Shimada, Rin-Ichiro Taniguchi, Li He","year":"2015","venue":"Proceedings of the 23rd Annual ACM Conference on Multimedia Conference","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhai et al., 2014] Xiaohua Zhai, Yuxin Peng, and Jianguo Xiao. Learning cross-media joint representation with s- parse and semisupervised regularization. IEEE Trans. Cir- cuits Syst. Video Techn., 24(6):965-978, 2014.","title":"Learning cross-media joint representation with sparse and semisupervised regularization","authors":"Xiaohua Zhai, Yuxin Peng, Jianguo Xiao","year":"2014","venue":"IEEE Trans. Circuits Syst. Video Techn","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}