{"type":"ScholarlyArticle","title":"Spectral Feature Scaling Method for Supervised Dimensionality Reduction","authors":[{"name":"Momo Matsuda","slug":"momo-matsuda","given":"Momo","family":"Matsuda","affiliation":"University of Tsukuba"},{"name":"Keiichi Morikuni","slug":"keiichi-morikuni","given":"Keiichi","family":"Morikuni","affiliation":"University of Tsukuba"},{"name":"Tetsuya Sakurai","slug":"tetsuya-sakurai","given":"Tetsuya","family":"Sakurai","affiliation":"University of Tsukuba; JST/CREST"}],"abstract":"Spectral dimensionality reduction methods enable linear separations of complex data with high-dimensional features in a reduced space. However, these methods do not always give the desired results due to irregularities or uncertainties of the data. Thus, we consider aggressively modifying the scales of the features to obtain the desired classification. Using prior knowledge on the labels of partial samples to specify the Fiedler vector, we formulate an eigenvalue problem of a linear matrix pencil whose eigenvector has the feature scaling factors. The resulting factors can modify the features of entire samples to form clusters in the reduced space, according to the known labels. In this study, we propose new dimensionality reduction methods supervised using the feature scaling associated with the spectral clustering. Numerical experiments show that the proposed methods outperform well-established supervised methods for toy problems with more samples than features, and are more robust regarding clustering than existing methods. Also, the proposed methods outperform existing methods regarding classification for real-world problems with more features than samples of gene expression profiles of cancer diseases. Furthermore, the feature scaling tends to improve the clustering and classification accuracies of existing unsupervised methods, as the proportion of training data increases.","doi":"10.24963/ijcai.2018/355","url":"https://ijcai.org/proceedings/2018/355","pdf":"https://ijcai.org/proceedings/2018/0355.pdf","markdown":"https://ijcai.org/proceedings/2018/355.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/355","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":355,"pages":{"first":2560,"last":2566},"keywords":["Machine Learning: Classification","Machine Learning: Machine Learning","Machine Learning: Knowledge-based Learning","Machine Learning: Data Mining","Machine Learning: Dimensionality Reduction and Manifold Learning","Machine Learning: Clustering"],"videos":[],"references":[{"position":1,"raw":"[Bach and Jordan, 2003] Francis R. Bach and Michael I. Jor- dan. Learning spectral clustering. In Advances in Neural Information Processing Systems 16, pages 305-312, Van- couver, Canada, June 2003.","title":"Learning spectral clustering","authors":"Francis R Bach, Michael I Jordan","year":"2003","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Banerjee et al., 2005] Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon, and Joydeep Ghosh. Clustering with bregman divergences. Journal of Machine Learning Re- search, 6:1705-1749, October 2005.","title":"Clustering with bregman divergences","authors":"Arindam Banerjee, Srujana Merugu, Inderjit S Dhillon, Joydeep Ghosh","year":"2005","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bar-Hillel et al., 2006] Aharon Bar-Hillel, Adam Spiro, and Eran Stark. Spike sorting: Bayesian clustering of non-stationary data. Journal of Neuroscience Methods, 157(2):303-316, October 2006.","title":"Aharon Bar-Hillel, Adam Spiro, and Eran Stark. Spike sorting: Bayesian clustering of non-stationary data","authors":"Bar-Hillel","year":"2006","venue":"Journal of Neuroscience Methods","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Belhumeur et al., 1997] Peter N. Belhumeur, João P. Hes- panha, and David J. Kriegman. Eigenfaces vs. fisher- faces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine In- telligence, 19(7):711-720, July 1997.","title":"Eigenfaces vs. fisherfaces: recognition using class specific linear projection","authors":"N Peter, João P Belhumeur, David J Hespanha, Kriegman","year":"1997","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chung, 1996] Fan R. K. Chung. Spectral Graph Theory. American Mathematical Society, Providence, USA, 1996.","title":"Spectral Graph Theory","authors":"R K Fan, Chung","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cui and Fan, 2012] Yan Cui and Liya Fan. A novel super- vised dimensionality reduction algorithm: Graph-based Fisher analysis. Pattern Recognition, 45(4):1471-1481, April 2012.","title":"A novel supervised dimensionality reduction algorithm: Graph-based Fisher analysis","authors":"Yan Cui, Liya Fan","year":"2012","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fiedler, 1973] Miroslav Fiedler. Algebraic connectivity of graphs. Czechoslovak Mathematical Journal, 23(98):298- 305, 1973.","title":"Algebraic connectivity of graphs","authors":"Miroslav Fiedler, Fiedler","year":"1973","venue":"Czechoslovak Mathematical Journal","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He and Niyogi, 2003] Xiaofei He and Partha Niyogi. Local- ity preserving projections. In Advances in Neural Informa- tion Processing Systems 16, pages 153-160, Vancouver, Canada, June 2003.","title":"Xiaofei He and Partha Niyogi. Locality preserving projections","authors":"Niyogi He","year":"2003","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Isaksson et al., 2008] Anders Isaksson, Mikael Wallman, Hanna Göransson, and Mats G. Gustafsson. Cross- validation and bootstrapping are unreliable in small sample classification. Pattern Recognition Letters, 29(14):1960-1965, July 2008.","title":"Crossvalidation and bootstrapping are unreliable in small sample classification","authors":"Anders Isaksson, Mikael Wallman, Hanna Göransson, Mats G Gustafsson","year":"2008","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ito and Murota, 2016] Shinji Ito and Kazuo Murota. An algorithm for the generalized eigenvalue problem for nonsquare matrix pencils by minimal perturbation ap- proach. SIAM Journal on Matrix Analysis and Applica- tions, 37(1):409-419, June 2016.","title":"An algorithm for the generalized eigenvalue problem for nonsquare matrix pencils by minimal perturbation approach","authors":"Murota ; ] Shinji Ito, Kazuo Ito, Murota","year":"2016","venue":"SIAM Journal on Matrix Analysis and Applications","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jurs et al., 2000] Peter C. Jurs, Gregory A. Bakke, and Heidi E. McClellands. Computational methods for the analysis of chemical sensor array data from volatile ana- lytes. Chemical Reviews, 100(7):2649-2678, June 2000.","title":"Computational methods for the analysis of chemical sensor array data from volatile analytes","authors":"C Peter, Gregory A Jurs, Heidi E Bakke, Mcclellands","year":"2000","venue":"Chemical Reviews","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kamvar et al., 2003] Sepandar D. Kamvar, Dan Klein, and Christopher D. Manning. Spectral learning. In Proceed- ings of the 18th International Joint Conference on Artifi- cial Intelligence, pages 561-566, Acapulco, Mexico, Au- gust 2003.","title":"Spectral learning","authors":"D Sepandar, Dan Kamvar, Christopher D Klein, Manning","year":"2003","venue":"Proceedings of the 18th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kulis et al., 2005] Brian Kulis, Sugato Basu, Inderjit Dhillon, and Raymond Mooney. Semi-supervised graph clustering: a kernel approach. In Proceedings of the 22th International Conference on Machine Learning, pages 457-464, Bonn, Germany, July 2005.","title":"Semi-supervised graph clustering: a kernel approach","authors":"Brian Kulis, Sugato Basu, Inderjit Dhillon, Raymond Mooney","year":"2005","venue":"Proceedings of the 22th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2017] Xuelong Li, Mulin Chen, Feiping Nie, and Qi Wang. Locality adaptive discriminant analysis. In Pro- ceedings of the 26th International Joint Conference on Ar- tificial Intelligence, pages 2201-2207, Melbourne, Aus- tralia, August 2017.","title":"Locality adaptive discriminant analysis","authors":"Li","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/306"},{"position":15,"raw":"[Morikuni, 2016] Keiichi Morikuni. Contour integral-type method for eigenproblems of rectangular matrix pencils. In Annual Meeting 2016 of the Japan Society for Industrial and Applied Mathematics, pages 352-353, Kitakyushu, Japan, September 2016.","title":"Contour integral-type method for eigenproblems of rectangular matrix pencils","authors":"Keiichi Morikuni","year":"2016","venue":"Annual Meeting 2016 of the Japan Society for Industrial and Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rand, 1971] William M. Rand. Objective criteria for the evaluation of clustering methods. Journal of the Ameri- can Statistical Association, 66(336):846-850, December 1971.","title":"Objective criteria for the evaluation of clustering methods","authors":"William M Rand","year":"1971","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Schleif and Tino, 2015] Frank-Michael Schleif and Peter Tino. Indefinite proximity learning: A review. Journal of Neural Computation, 27(10):2039-2096, October 2015.","title":"Indefinite proximity learning: A review","authors":"Tino ; Frank-Michael Schleif, Peter Schleif, Tino","year":"2015","venue":"Journal of Neural Computation","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shi and Malik, 2000] Jianbo Shi and Jitendra Malik. Nor- malized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888- 905, August 2000.","title":"Normalized cuts and image segmentation","authors":"Jianbo Shi, Jitendra Malik","year":"2000","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sogh, 2006] Tarek S. Sogh. Wired and wireless intru- sion detection system: Classifications, good characteris- tics and state-of-the-art. Computer Standards and Inter- faces, 28(6):670-694, September 2006.","title":"Wired and wireless intrusion detection system: Classifications, good characteristics and state-of-the-art","authors":"S Tarek, Sogh","year":"2006","venue":"Computer Standards and Interfaces","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Song et al., 2008] Yangqiu Song, Feiping Nie, Changshui Zhang, and Shiming Xiang. A unified framework for semi- supervised dimensionality reduction. Pattern Recognition, 41(9):2789-2799, September 2008.","title":"Yangqiu Song, Feiping Nie, Changshui Zhang, and Shiming Xiang. A unified framework for semisupervised dimensionality reduction","authors":"Song","year":"2008","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Spielman and Teng, 2014] Daniel A. Spielman and Shang- Hua Teng. Nearly linear time algorithms for precondi- tioning and solving symmetric, diagonally dominant linear systems. SIAM Journal on Matrix Analysis and Applica- tions, 35(3):853-885, July 2014.","title":"Nearly linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems","authors":"Daniel A Spielman, Shang-Hua Teng","year":"2014","venue":"SIAM Journal on Matrix Analysis and Applications","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Strehl and Ghosh, 2002] Alexander Strehl and Joydeep Ghosh. Cluster ensembles -a knowledge reuse frame- work for combining multiple partitions. Journal of Ma- chine Learning Research, 3:583-617, December 2002.","title":"Cluster ensembles -a knowledge reuse framework for combining multiple partitions","authors":"Ghosh Strehl, Alexander Strehl, Joydeep Ghosh","year":"2002","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sugiyama, 2007] Masashi Sugiyama. Dimensionality re- duction of multimodal labeled data by local Fisher dis- criminant analysis. Journal of Machine Learning Re- search, 8:1027-1061, May 2007.","title":"Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis","authors":"Masashi Sugiyama, Sugiyama","year":"2007","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tarca et al., 2006] Adi L. Tarca, Robert Remero, and Sorin Draghici. Analysis of microarray experiments of gene ex- pression profiling. American Journal of Obstetrics and Gynecology,, 195(2):373-388, August 2006.","title":"Analysis of microarray experiments of gene expression profiling","authors":"Adi L Tarca, Robert Remero, Sorin Draghici","year":"2006","venue":"American Journal of Obstetrics and Gynecology","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tichy et al., 1979] Noel M. Tichy, Michael L. Tushman, and Charles Fombrun. Social network analysis for organi- zations. Academy of Management Review, 4(4):507-519, October 1979.","title":"Social network analysis for organizations","authors":"M Noel, Michael L Tichy, Charles Tushman, Fombrun","year":"1979","venue":"Academy of Management Review","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang, 2006] Liu Yang. Distance metric learning: A com- prehensive survey. Michigan State University, pages 1-51, May 2006.","title":"Distance metric learning: A comprehensive survey","authors":"Liu Yang","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}