{"type":"ScholarlyArticle","title":"Dimensionality Reduction with Generalized Linear Models","authors":[{"name":"Mo Chen","slug":"mo-chen","given":"Mo","family":"Chen","affiliation":"Department of Electronic Engineering"},{"name":"Wei Li","slug":"wei-li-2","given":"Wei","family":"Li","affiliation":"Department of Electronic Engineering"},{"name":"Xiaogang Wang","slug":"xiaogang-wang","given":"Xiaogang","family":"Wang","affiliation":"Department of Information Engineering The Chinese University of Hong Kong"},{"name":"Wei Zhang","slug":"wei-zhang-4","given":"Wei","family":"Zhang","affiliation":"Department of Electronic Engineering"}],"abstract":"In this paper, we propose a general dimensionality reduction method for data generated from a very broad family of distributions and nonlinear functions based on the generalized linear model, called Generalized Linear Principal Component Analysis (GLPCA). Data of different domains often have very different structures. These data can be modeled by different distributions and reconstruction functions. For example, real valued data can be modeled by the Gaussian distribution with a linear reconstruction function, whereas binary valued data may be more appropriately modeled by the Bernoulli distribution with a logit or probit function. Based on general linear models, we propose a unified framework for extracting features from data of different domains. A general optimization algorithm based on natural gradient ascent on distribution manifold is proposed for obtaining the maximum likelihood solutions. We also present some specific algorithms derived from this framework to deal with specific data modeling problems such as document modeling. Experimental results of these algorithms on several data sets are shown for the validation of GLPCA.","doi":null,"url":"https://ijcai.org/proceedings/2013/191","pdf":"https://ijcai.org/proceedings/2013/0191.pdf","markdown":"https://ijcai.org/proceedings/2013/191.md","bibtex":"https://ijcai.org/proceedings/2013/bibtex/191","volume":{"slug":"2013","title":"Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2013","url":"https://ijcai.org/proceedings/2013/","year":2013,"location":"Beijing, China","dates":"3–9 August 2013","editors":"Francesca Rossi","isbn":"978-1-57735-633-2","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":191,"pages":{"first":1267,"last":1271},"keywords":[],"videos":[],"references":[{"position":1,"raw":"[Amari and Nagaoka, 2000] S. Amari and H. Nagaoka. Methods of information geometry. Amer Mathematical Society, 2000.","title":"Methods of information geometry","authors":"S Amari, H Nagaoka","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amari, 1998] Shun-Ichi Amari. Natural gradient works ef- ficiently in learning. Neural Computation, 10(2):251-276, 1998.","title":"Shun-Ichi Amari. Natural gradient works efficiently in learning","authors":"Amari","year":"1998","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Azoury and Warmuth, 2001] K.S. Azoury and MK War- muth. Relative loss bounds for on-line density estima- tion with the exponential family of distributions. Machine Learning, 43(3):211-246, 2001.","title":"Relative loss bounds for on-line density estimation with the exponential family of distributions","authors":"K S Azoury, Warmuth","year":"2001","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Box and Cox, 1964] GEP Box and DR Cox. An analysis of transformations. Journal of the Royal Statistical Society. Series B (Methodological), pages 211-252, 1964.","title":"GEP Box and DR Cox. An analysis of transformations","authors":"Cox Box","year":"1964","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collins et al., 2002] M. Collins, S. Dasgupta, and R.E. Schapire. A generalization of principal component anal- ysis to the exponential family. Advances in Neural Infor- mation Processing Systems, 1:617-624, 2002.","title":"A generalization of principal component analysis to the exponential family","authors":"M Collins, S Dasgupta, R E Schapire","year":"2002","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Forster and Warmuth, 2002] J. Forster and M.K. Warmuth. Relative expected instantaneous loss bounds. Journal of Computer and System Sciences, 64(1):76-102, 2002.","title":"Relative expected instantaneous loss bounds","authors":"Warmuth Forster, ] J Forster, M K Warmuth","year":"2002","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gordon, 2003] Geoffrey J Gordon. Generalized 2 linear 2 models. Advances in Neural Information Processing Sys- tems, pages 593-600, 2003.","title":"Generalized 2 linear 2 models","authors":"Geoffrey J Gordon","year":"2003","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kivinen and Warmuth, 2001] Jyrki Kivinen and Manfred K. Warmuth. Relative loss bounds for multidimensional re- gression problems. Machine Learning, 45(3):301-329, 2001.","title":"Relative loss bounds for multidimensional regression problems","authors":"Warmuth ; ] Jyrki Kivinen, Manfred K Kivinen, Warmuth","year":"2001","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kolda and Bader, 2009] T.G. Kolda and B.W. Bader. Ten- sor decompositions and applications. SIAM review, 51(3):455-500, 2009.","title":"Tensor decompositions and applications","authors":"T G Kolda, B W Bader","year":"2009","venue":"SIAM review","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Nelder and Wedderburn, 1972] JA Nelder and RWM Wed- derburn. Generalized linear models. Journal of the Royal Statistical Society. Series A (General), 135(3):370-384, 1972.","title":"JA Nelder and RWM Wedderburn. Generalized linear models","authors":"Wedderburn Nelder","year":"1972","venue":"Journal of the Royal Statistical Society. Series A (General)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Rish et al., 2008] Irina Rish, Genady Grabarnik, Guillermo Cecchi, Francisco Pereira, and Geoffrey J Gordon. Closed- form supervised dimensionality reduction with general- ized linear models. pages 832-839, 2008.","title":"Closedform supervised dimensionality reduction with generalized linear models","authors":"Irina Rish, Genady Grabarnik, Guillermo Cecchi, Francisco Pereira, Geoffrey J Gordon","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Roweis, 1998] S. Roweis. EM algorithms for PCA and SPCA. Advances in Neural Information Processing Sys- tems, pages 626-632, 1998.","title":"EM algorithms for PCA and SPCA","authors":"S Roweis","year":"1998","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Schein et al., 2003] A.I. Schein, L.K. Saul, and L.H. Ungar. A generalized linear model for principal component anal- ysis of binary data. page 546431, 2003.","title":"A generalized linear model for principal component analysis of binary data","authors":"A I Schein, L K Saul, L H Ungar","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Wainwright and Jordan, 2008] M.J. Wainwright and M.I. Jordan. Graphical models, exponential families, and vari- ational inference. Foundations and Trends in Machine Learning, 1(1-2):1-305, 2008.","title":"Graphical models, exponential families, and variational inference","authors":"M J Wainwright, M I Jordan","year":"2008","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}