{"type":"ScholarlyArticle","title":"Adaptive Semi-Supervised Learning with Discriminative Least Squares Regression","authors":[{"name":"Minnan Luo","slug":"minnan-luo","given":"Minnan","family":"Luo","affiliation":"SPKLSTN Lab , Department of Computer Science , Xi'an Jiaotong University , Shaanxi , China."},{"name":"Lingling Zhang","slug":"lingling-zhang","given":"Lingling","family":"Zhang","affiliation":"SPKLSTN Lab , Department of Computer Science , Xi'an Jiaotong University , Shaanxi , China."},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"Center for OPTical Imagery Analysis and Learning , Northwestern Polytechnical University , China."},{"name":"Xiaojun Chang","slug":"xiaojun-chang","given":"Xiaojun","family":"Chang","affiliation":"School of Computer Science , Carnegie Mellon University , PA , USA."},{"name":"Buyue Qian","slug":"buyue-qian","given":"Buyue","family":"Qian","affiliation":"SPKLSTN Lab , Department of Computer Science , Xi'an Jiaotong University , Shaanxi , China."},{"name":"Qinghua Zheng","slug":"qinghua-zheng","given":"Qinghua","family":"Zheng","affiliation":"SPKLSTN Lab , Department of Computer Science , Xi'an Jiaotong University , Shaanxi , China."}],"abstract":"Semi-supervised learning plays a significant role in multi-class classification, where a small number of labeled data are more deterministic while substantial unlabeled data might cause large uncertainties and potential threats. \r\nIn this paper, we distinguish the label fitting of labeled and unlabeled training data through a probabilistic vector with an adaptive parameter, which always ensures the significant importance of labeled data and characterizes the contribution of unlabeled instance according to its uncertainty. Instead of using traditional least squares regression (LSR) for classification, we develop a new discriminative LSR by equipping each label with an adjustment vector. This strategy avoids incorrect penalization on samples that are far away from the boundary and simultaneously facilitates multi-class classification by enlarging the geometrical distance of instances belonging to different classes. An efficient alternative algorithm is exploited to solve the proposed model with closed form solution for each updating rule. We also analyze the convergence and complexity of the proposed algorithm theoretically. Experimental results on several benchmark datasets demonstrate the effectiveness and superiority of the proposed model for multi-class classification tasks.","doi":"10.24963/ijcai.2017/337","url":"https://ijcai.org/proceedings/2017/337","pdf":"https://ijcai.org/proceedings/2017/0337.pdf","markdown":"https://ijcai.org/proceedings/2017/337.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/337","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":337,"pages":{"first":2421,"last":2427},"keywords":["Machine Learning: Semi-Supervised Learning","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Belkin et al., 2006] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled exam- ples. J. Mach. Learn. Res., 7:2399-2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"Mikhail Belkin, Partha Niyogi, Vikas Sindhwani","year":"2006","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bishop, 2006] Christopher M Bishop. Pattern recognition. Mach. Learn., 128:1-58, 2006.","title":"Pattern recognition","authors":"M Christopher, Bishop","year":"2006","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2007a] Deng Cai, Xiaofei He, and Jiawei Han. Semi-supervised discriminant analysis. In ICCV, 2007.","title":"Semi-supervised discriminant analysis","authors":"Cai","year":"2007","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2007b] Deng Cai, Xiaofei He, Yuxiao Hu, Jiawei Han, and Thomas Huang. Learning a spatially smooth sub- space for face recognition. In CVPR, 2007.","title":"Learning a spatially smooth subspace for face recognition","authors":"Cai","year":"2007","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chang and Yang, 2016] Xiaojun Chang and Yi Yang. Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Trans. Neural Netw. Learn. Syst., 2016.","title":"Semisupervised feature analysis by mining correlations among multiple tasks","authors":"Xiaojun Chang, Yi Yang","year":"2016","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chang et al., 2014] Xiaojun Chang, Feiping Nie, Yi Yang, and Heng Huang. A convex formulation for semi- supervised multi-label feature selection. In AAAI, 2014.","title":"A convex formulation for semisupervised multi-label feature selection","authors":"Xiaojun Chang, Feiping Nie, Yi Yang, Heng Huang","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chapelle et al., 2008] Olivier Chapelle, Vikas Sindhwani, and Sathiya S Keerthi. Optimization techniques for semi- supervised support vector machines. J. Mach. Learn. Res., 9:203-233, 2008.","title":"Optimization techniques for semisupervised support vector machines","authors":"Olivier Chapelle, Vikas Sindhwani, Sathiya S Keerthi","year":"2008","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chapelle et al., 2009] Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. Semi-supervised learning. IEEE Trans. Neural Netw., 20(3):542-542, 2009.","title":"Semi-supervised learning","authors":"Olivier Chapelle, Bernhard Scholkopf, Alexander Zien","year":"2009","venue":"IEEE Trans. Neural Netw","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fan et al., 2005] Rong-En Fan, Pai-Hsuen Chen, and Chih- Jen Lin. Working set selection using second order infor- mation for training support vector machines. J. Mach. Learn. Res., 6:1889-1918, 2005.","title":"Working set selection using second order information for training support vector machines","authors":"Rong-En Fan, Chih-Jen Pai-Hsuen Chen, Lin","year":"2005","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Georghiades et al., 2001] Athinodoros S Georghiades, Pe- ter N Belhumeur, and David J Kriegman. From few to many: Illumination cone models for face recognition un- der variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell., 23(6):643-660, 2001.","title":"From few to many: Illumination cone models for face recognition under variable lighting and pose","authors":"Peter N Athinodoros S Georghiades, David J Belhumeur, Kriegman","year":"2001","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2011] Xiaobai Liu, Xiaotong Yuan, Shuicheng Yan, and Hai Jin. Multi-class semi-supervised svms with positiveness exclusive regularization. In ICCV, 2011.","title":"Multi-class semi-supervised svms with positiveness exclusive regularization","authors":"Xiaobai Liu, Xiaotong Yuan, Shuicheng Yan, Hai Jin","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Luo et al., 2017] Minnan Luo, Xiaojun Chang, Liqiang Nie, Yi Yang, Alexander G Hauptmann, and Qinghua Zheng. An adaptive semisupervised feature analysis for video se- mantic recognition. IEEE Trans. Cybern., 2017.","title":"An adaptive semisupervised feature analysis for video semantic recognition","authors":"Luo","year":"2017","venue":"IEEE Trans. Cybern","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nie et al., 2010] Feiping Nie, Dong Xu, Ivor Wai-Hung Tsang, and Changshui Zhang. Flexible manifold em- bedding: A framework for semi-supervised and unsuper- vised dimension reduction. IEEE Trans. Image Process., 19(7):1921-1932, 2010.","title":"Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction","authors":"Nie","year":"2010","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nie et al., 2011a] Feiping Nie, Hua Wang, Heng Huang, and Chris Ding. Unsupervised and semi-supervised learn- ing via ℓ 1 -norm graph. In ICCV, 2011.","title":"Unsupervised and semi-supervised learning via ℓ 1 -norm graph","authors":"Nie","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nie et al., 2011b] Feiping Nie, Dong Xu, Xuelong Li, and Shiming Xiang. Semisupervised dimensionality reduction and classification through virtual label regression. IEEE Trans. Systems, Man, and Cybernetics, Part B, 41(3):675- 685, 2011.","title":"Semisupervised dimensionality reduction and classification through virtual label regression","authors":"Nie","year":"2011","venue":"IEEE Trans. Systems, Man, and Cybernetics, Part B","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sim et al., 2002] Terence Sim, Simon Baker, and Maan Bsat. The cmu pose, illumination, and expression (pie) database. In FG, 2002.","title":"The cmu pose, illumination, and expression (pie) database","authors":"Terence Sim, Simon Baker, Maan Bsat","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sindhwani and Keerthi, 2006] Vikas Sindhwani and S Sathiya Keerthi. Large scale semi-supervised linear svms. In ACM SIGIR, 2006.","title":"Large scale semi-supervised linear svms","authors":"Vikas Sindhwani, S Sathiya, Keerthi","year":"2006","venue":"ACM SIGIR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Subramanya and Talukdar, 2014] Amarnag Subramanya and Partha Pratim Talukdar. Graph-based semi-supervised learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 8(4):1-125, 2014.","title":"Amarnag Subramanya and Partha Pratim Talukdar","authors":"Talukdar Subramanya","year":"2014","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2014] De Wang, Feiping Nie, and Heng Huang. Large-scale adaptive semi-supervised learning via unified inductive and transductive model. In SIGKDD, 2014.","title":"Large-scale adaptive semi-supervised learning via unified inductive and transductive model","authors":"De Wang, Feiping Nie, Heng Huang","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xiang et al., 2012] Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, and Changshui Zhang. Discrimi- native least squares regression for multiclass classification and feature selection. IEEE Trans. Neural Netw. Learn. Syst., 23(11):1738-1754, 2012.","title":"Discriminative least squares regression for multiclass classification and feature selection","authors":"Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, Changshui Zhang","year":"2012","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yan et al., 2007] Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, and Stephen Lin. Graph embedding and extensions: a general framework for di- mensionality reduction. IEEE Trans. Pattern Anal. Mach. Intell., 29(1):40-51, 2007.","title":"Graph embedding and extensions: a general framework for dimensionality reduction","authors":"Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, Stephen Lin","year":"2007","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2015] Yi Yang, Zhigang Ma, Feiping Nie, Xiao- jun Chang, and Alexander G Hauptmann. Multi-class ac- tive learning by uncertainty sampling with diversity maxi- mization. Int. J. Comput. Vision, 113(2):113-127, 2015.","title":"Multi-class active learning by uncertainty sampling with diversity maximization","authors":"Yi Yang, Zhigang Ma, Feiping Nie, Xiaojun Chang, Alexander G Hauptmann","year":"2015","venue":"Int. J. Comput. Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhou et al., 2004] Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. NIPS, 2004.","title":"Learning with local and global consistency","authors":"Zhou","year":"2004","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhu and Goldberg, 2009] Xiaojin Zhu and Andrew B Gold- berg. Introduction to semi-supervised learning. Morgan & Claypool Publishers, 2009.","title":"Introduction to semi-supervised learning","authors":"Xiaojin Zhu, Andrew B Goldberg","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhu et al., 2003] Xiaojin Zhu, Zoubin Ghahramani, John Lafferty, et al. Semi-supervised learning using gaussian fields and harmonic functions. In ICML, 2003.","title":"Semi-supervised learning using gaussian fields and harmonic functions","authors":"Xiaojin Zhu, Zoubin Ghahramani, John Lafferty","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2004] Xiaojin Zhu, Jaz Kandola, Zoubin Ghahramani, and John D Lafferty. Nonparametric trans- forms of graph kernels for semi-supervised learning. In NIPS, 2004.","title":"Nonparametric transforms of graph kernels for semi-supervised learning","authors":"Xiaojin Zhu, Jaz Kandola, Zoubin Ghahramani, John D Lafferty","year":"2004","venue":"NIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}