{"type":"ScholarlyArticle","title":"Convolutional 2D LDA for Nonlinear Dimensionality Reduction","authors":[{"name":"Qi Wang","slug":"qi-wang","given":"Qi","family":"Wang","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xian 710072 , Shaanxi , P. R. China; Unmanned System Research Institute (USRI) , Northwestern Polytechnical University , Xian 710072 , Shaanxi , P. R. China"},{"name":"Zequn Qin","slug":"zequn-qin","given":"Zequn","family":"Qin","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xian 710072 , Shaanxi , P. R. China"},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xian 710072 , Shaanxi , P. R. China"},{"name":"Yuan Yuan","slug":"yuan-yuan","given":"Yuan","family":"Yuan","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xian 710072 , Shaanxi , P. R. China"}],"abstract":"Representing high-volume and high-order data is an essential problem, especially in machine learning field. Although existing two-dimensional (2D) discriminant analysis achieves promising performance, the single and linear projection features make it difficult to analyze more complex data. In this paper, we propose a novel convolutional two-dimensional linear discriminant analysis (2D LDA) method for data representation. In order to deal with nonlinear data, a specially designed Convolutional Neural Networks (CNN) is presented, which can be proved having the equivalent objective function with common 2D LDA. In this way, the discriminant ability can benefit from not only the nonlinearity of Convolutional Neural Networks, but also the powerful learning process. Experiment results on several datasets show that the proposed method performs better than other state-of-the-art methods in terms of classification accuracy.","doi":"10.24963/ijcai.2017/408","url":"https://ijcai.org/proceedings/2017/408","pdf":"https://ijcai.org/proceedings/2017/0408.pdf","markdown":"https://ijcai.org/proceedings/2017/408.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/408","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":408,"pages":{"first":2929,"last":2935},"keywords":["Machine Learning: Feature Selection/Construction","Machine Learning: Machine Learning","Machine Learning: Classification","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, and Michael Isard. Tensorflow: A system for large-scale machine learn- ing. 2016.","title":"Tensorflow: A system for large-scale machine learning","authors":"Martn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In ICML, pages 1247-1255, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Jeff Bilmes, Karen Livescu","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Belhumeur et al., 1997] Peter N. Belhumeur, João P Hes- panha, and David J. Kriegman. Eigenfaces vs. fisher- faces: Recognition using class specific linear projection. IEEE Transactions on pattern analysis and machine intel- ligence, 19(7):711-720, 1997.","title":"Eigenfaces vs. fisherfaces: Recognition using class specific linear projection","authors":"N Peter, João P Belhumeur, David J Hespanha, Kriegman","year":"1997","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chan et al., 2014] Tsung Han Chan, Kui Jia, Shenghua Gao, and Jiwen Lu. Pcanet: A simple deep learning base- line for image classification? IEEE Transactions on Image Processing, 24(12):5017-5032, 2014.","title":"Pcanet: A simple deep learning baseline for image classification","authors":"Tsung Han Chan, Kui Jia, Shenghua Gao, Jiwen Lu","year":"2014","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chang et al., 2015] Xiaojun Chang, Feiping Nie, Sen Wang, Yi Yang, Xiaofang Zhou, and Chengqi Zhang. Compound rank-k projections for bilinear analysis. 2015. [Diem et al., 2013]","title":"Compound rank-k projections for bilinear analysis","authors":"Xiaojun Chang, Feiping Nie, Sen Wang, Yi Yang, Xiaofang Zhou, Chengqi Zhang","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"Markus Diem, Stefan Fiel, Angelika Garz, Manuel Keglevic, Florian Kleber, and Robert Sab- latnig. Icdar 2013 competition on handwritten digit recog- nition (hdrc 2013). pages 1422-1427, 2013.","title":"Icdar 2013 competition on handwritten digit recognition","authors":"Markus Diem, Stefan Fiel, Angelika Garz, Manuel Keglevic, Florian Kleber, Robert Sablatnig","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dorfer et al., 2015] Matthias Dorfer, Rainer Kelz, and Ger- hard Widmer. Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707, 2015.","title":"Deep linear discriminant analysis","authors":"Matthias Dorfer, Rainer Kelz, Gerhard Widmer","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow et al., 2013] Ian J Goodfellow, David Warde- Farley, Mehdi Mirza, Aaron C Courville, and Yoshua Ben- gio. Maxout networks. ICML (3), 28:1319-1327, 2013.","title":"Maxout networks","authors":"Ian J Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron C Courville, Yoshua Bengio","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Graham, 2014] Benjamin Graham. Spatially-sparse convo- lutional neural networks. arXiv preprint arXiv:1409.6070, 2014.","title":"Spatially-sparse convolutional neural networks","authors":"Benjamin Graham","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2005] Xiaofei He, Deng Cai, and Partha Niyogi. Tensor subspace analysis. In Advances in neural informa- tion processing systems, pages 499-506, 2005.","title":"Tensor subspace analysis","authors":"Xiaofei He, Deng Cai, Partha Niyogi","year":"2005","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In IEEE Conference on Computer Vision and Pat- tern Recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hinton et al., 2012] Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhut- dinov. Improving neural networks by preventing co- adaptation of feature detectors. Computer Science, 3(4):pgs. 212-223, 2012.","title":"Improving neural networks by preventing coadaptation of feature detectors","authors":"Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R Salakhutdinov","year":"2012","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hou et al., 2012] Y. Hou, I. Song, H. K. Min, and C. H. Park. Complexity-reduced scheme for feature extraction with linear discriminant analysis. IEEE Transactions on Neural Networks & Learning Systems, 23(6):1003-1009, 2012.","title":"Complexity-reduced scheme for feature extraction with linear discriminant analysis","authors":"Y Hou, I Song, H K Min, C H Park","year":"2012","venue":"IEEE Transactions on Neural Networks & Learning Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Inoue and Urahama, 2006] Kohei Inoue and Kiichi Ura- hama. Non-iterative two-dimensional linear discriminant analysis. In 18th International Conference on Pattern Recognition (ICPR'06), volume 2, pages 540-543. IEEE, 2006.","title":"Non-iterative two-dimensional linear discriminant analysis","authors":"Kohei Inoue, Kiichi Urahama","year":"2006","venue":"18th International Conference on Pattern Recognition (ICPR'06)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. Computer Science, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Sergey Ioffe, Christian Szegedy","year":"2015","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lecun et al., 1998] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Y Lecun, L Bottou, Y Bengio, P Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li and Yuan, 2005] Ming Li and Baozong Yuan. 2d-lda: A statistical linear discriminant analysis for image matrix. Pattern Recognition Letters, 26(5):527-532, 2005.","title":"2d-lda: A statistical linear discriminant analysis for image matrix","authors":"Li, Yuan, Ming Li, Baozong Yuan","year":"2005","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin et al., 2013] Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. arXiv preprint arXiv:1312.4400, 2013.","title":"Network in network","authors":"Lin","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ma et al., 2013] Zhigang Ma, Yi Yang, Feiping Nie, and Nicu Sebe. Thinking of images as what they are: com- pound matrix regression for image classification. In Inter- national Joint Conference on Artificial Intelligence, pages 1530-1536, 2013.","title":"Thinking of images as what they are: compound matrix regression for image classification","authors":"Zhigang Ma, Yi Yang, Feiping Nie, Nicu Sebe","year":"2013","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2013/228"},{"position":21,"raw":"[Mahanta et al., 2013] Mohammad Shahin Mahanta, Amirhossein S. Aghaei, and Konstantinos N. Plataniotis. Regularized lda based on separable scatter matrices for classification of spatio-spectral eeg patterns. 32(3):1237- 1241, 2013.","title":"Regularized lda based on separable scatter matrices for classification of spatio-spectral eeg patterns","authors":"Amirhossein S Mohammad Shahin Mahanta, Konstantinos N Aghaei, Plataniotis","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pirsiavash et al., 2009] Hamed Pirsiavash, Deva Ramanan, and Charless C Fowlkes. Bilinear classifiers for visual recognition. In Advances in neural information process- ing systems, pages 1482-1490, 2009.","title":"Bilinear classifiers for visual recognition","authors":"Hamed Pirsiavash, Deva Ramanan, Charless C Fowlkes","year":"2009","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ren et al., 2012] Chuan-Xian Ren, Dao-Qing Dai, and Hong Yan. Coupled kernel embedding for low-resolution face image recognition. IEEE Transactions on Image Pro- cessing, 21(8):3770-3783, 2012.","title":"Coupled kernel embedding for low-resolution face image recognition","authors":"Dao-Qing Chuan-Xian Ren, Hong Dai, Yan","year":"2012","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ren et al., 2015] Chuan-Xian Ren, DAI Dao-Qing, Xiaofei He, and Hong Yan. Sample weighting: An inherent approach for outlier suppressing discriminant analysis. IEEE Transactions on Knowledge and Data Engineering, 27(11):3070-3083, 2015.","title":"Sample weighting: An inherent approach for outlier suppressing discriminant analysis","authors":"Chuan-Xian Ren, Xiaofei Dai Dao-Qing, Hong He, Yan","year":"2015","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sutskever et al., 2013] I. Sutskever, J. Martens, G. Dahl, and G. Hinton. On the importance of initialization and momentum in deep learning. In International Conference on Machine Learning, 2013.","title":"On the importance of initialization and momentum in deep learning","authors":"Sutskever","year":"2013","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2004] Jian Yang, David Zhang, Alejandro F Frangi, and Jing-yu Yang. Two-dimensional pca: a new approach to appearance-based face representation and recognition. IEEE transactions on pattern analysis and machine intelligence, 26(1):131-137, 2004.","title":"Two-dimensional pca: a new approach to appearance-based face representation and recognition","authors":"Jian Yang, David Zhang, Alejandro F Frangi, Jing-Yu Yang","year":"2004","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ye et al., 2004] Jieping Ye, Ravi Janardan, and Qi Li. Two- dimensional linear discriminant analysis. In Advances in neural information processing systems, pages 1569-1576, 2004.","title":"Twodimensional linear discriminant analysis","authors":"Ye","year":"2004","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}