{"type":"ScholarlyArticle","title":"Multimodal Linear Discriminant Analysis via Structural Sparsity","authors":[{"name":"Yu Zhang","slug":"yu-zhang","given":"Yu","family":"Zhang","affiliation":"Department of Computer Science and Engineering , Hong Kong University of Science and Technology"},{"name":"Yuan Jiang","slug":"yuan-jiang","given":"Yuan","family":"Jiang","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University"}],"abstract":"Linear discriminant analysis (LDA) is a widely used supervised dimensionality reduction technique. Even though the LDA method has many real-world applications, it has some limitations such as the single-modal problem that each class follows a normal distribution. To solve this problem, we propose a method called multimodal linear discriminant analysis (MLDA). By generalizing the between-class and within-class scatter matrices, the MLDA model can allow each data point to have its own class mean which is called the instance-specific class mean. Then in each class, data points which share the same or similar instance-specific class means are considered to form one cluster or modal. In order to learn the instance-specific class means, we use the ratio of the proposed generalized between-class scatter measure over the proposed generalized within-class scatter measure, which encourages the class separability, as a criterion. The observation that each class will have a limited number of clusters inspires us to use a structural sparse regularizor to control the number of unique instance-specific class means in each class. Experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed MLDA method.","doi":"10.24963/ijcai.2017/482","url":"https://ijcai.org/proceedings/2017/482","pdf":"https://ijcai.org/proceedings/2017/0482.pdf","markdown":"https://ijcai.org/proceedings/2017/482.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/482","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":482,"pages":{"first":3448,"last":3454},"keywords":["Machine Learning: Machine Learning","Machine Learning: Feature Selection/Construction"],"videos":[],"references":[{"position":1,"raw":"[Bach et al., 2012a] Francis R. Bach, Rodolphe Jenatton, Julien Mairal, and Guillaume Obozinski. Optimization with sparsity-inducing penalties. Foundations and Trends in Machine Learning, 4(1):1-106, 2012.","title":"Optimization with sparsity-inducing penalties","authors":"Bach","year":"2012","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bach et al., 2012b] Francis R. Bach, Rodolphe Jenatton, Julien Mairal, and Guillaume Obozinski. Structured s- parsity through convex optimization. Statistical Science, 27(4):450-468, 2012.","title":"Structured sparsity through convex optimization","authors":"Bach","year":"2012","venue":"Statistical Science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Belhumeur et al., 1997] Peter N. Belhumeur, João P. Hes- panha, and David J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711-720, 1997.","title":"Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection","authors":"N Peter, João P Belhumeur, David J Hespanha, Kriegman","year":"1997","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gong et al., 2013] Pinghua Gong, Changshui Zhang, Zhaosong Lu, Jianhua Huang, and Jieping Ye. A general iterative shrinkage and thresholding algorithm for non- convex regularized optimization problems. In Proceedings of the 30th International Conference on Machine Learning, pages 37-45, 2013.","title":"A general iterative shrinkage and thresholding algorithm for nonconvex regularized optimization problems","authors":"Gong","year":"2013","venue":"Proceedings of the 30th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Graham and Allinson, 1998] Daniel B. Graham and Nigel Allinson. Characterizing virtual eigensignatures for general purpose face recognition. In Face Recognition: From The- ory to Applications, volume 163, pages 446-456. Springer, 1998.","title":"Characterizing virtual eigensignatures for general purpose face recognition","authors":"Daniel B Graham, Nigel Allinson","year":"1998","venue":"Face Recognition: From Theory to Applications","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hastie and Tibshirani, 1996] Trevor Hastie and Robert Tib- shirani. Discriminant analysis by Gaussian mixtures. Jour- nal of the Royal Statistical Society, Series B, 58(1):155-176, 1996.","title":"Trevor Hastie and Robert Tibshirani. Discriminant analysis by Gaussian mixtures","authors":"Tibshirani Hastie","year":"1996","venue":"Journal of the Royal Statistical Society, Series B","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hocking et al., 2011] Toby Hocking, Jean-Philippe Vert, Francis R. Bach, and Armand Joulin. Clusterpath: An algorithm for clustering using convex fusion penalties. In Proceedings of the 28th International Conference on Ma- chine Learning, pages 745-752, 2011.","title":"Clusterpath: An algorithm for clustering using convex fusion penalties","authors":"Toby Hocking, Jean-Philippe Vert, Francis R Bach, Armand Joulin","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kim and Kittler, 2005] Tae-Kyun Kim and Josef Kittler. Lo- cally linear discriminant analysis for multimodally distribut- ed classes for face recognition with a single model image. IEEE Transactions on Pattern Analysis and Machine Intel- ligence, 27(3):318-327, 2005.","title":"Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image","authors":"Kittler ; Tae-Kyun Kim, Josef Kim, Kittler","year":"2005","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kuo and Landgrebe, 2004] B.-C. Kuo and D. A. Landgrebe. Nonparametric weighted feature extraction for classifica- tion. IEEE Transactions on Geoscience and Remote Sens- ing, 42(5):1096-1105, 2004.","title":"Nonparametric weighted feature extraction for classification","authors":"B.-C Kuo, D A Landgrebe","year":"2004","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[LeCun and Cortes, 1998] Yann LeCun and Corinna Cortes. The MNIST database of handwritten digits, 1998.","title":"The MNIST database of handwritten digits","authors":"Lecun, Cortes, Yann Lecun, Corinna Cortes","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Leibe and Schiele, 2003] Bastian Leibe and Bernt Schiele. Analyzing appearance and contour based methods for ob- ject categorization. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recog- nition, pages 409-415, 2003.","title":"Analyzing appearance and contour based methods for object categorization","authors":"Schiele Leibe, Bastian Leibe, Bernt Schiele","year":"2003","venue":"Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2009] Zhifeng Li, Dahua Lin, and Xiaoou Tang. Nonparametric discriminant analysis for face recognition. IEEE Transactions on Pattern Analysis and Machine Intel- ligence, 31(4):755-761, 2009.","title":"Nonparametric discriminant analysis for face recognition","authors":"Zhifeng Li, Dahua Lin, Xiaoou Tang","year":"2009","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Martínez and Benavente, 1998] Aleix M. Martínez and Robert Benavente. The AR-face database. Technical Re- port 24, CVC, 1998.","title":"The AR-face database","authors":"M Aleix, Robert Martínez, Benavente","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Martínez and Zhu, 2005] Aleix M. Martínez and Manli Zhu. Where are linear feature extraction methods applicable? IEEE Transactions on Pattern Analysis and Machine Intel- ligence, 27(12):1934-1944, 2005.","title":"Where are linear feature extraction methods applicable","authors":"Zhu ; ] Martínez, M Aleix, Manli Martínez, Zhu","year":"2005","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sugiyama, 2006] Masashi Sugiyama. Local Fisher discrimi- nant analysis for supervised dimensionality reduction. In Proceedings of the 23rd International Conference on Ma- chine Learning, pages 905-912, 2006.","title":"Local Fisher discriminant analysis for supervised dimensionality reduction","authors":"Masashi Sugiyama, Sugiyama","year":"2006","venue":"Proceedings of the 23rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sugiyama, 2007] Masashi Sugiyama. Dimensionality reduc- tion of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research, 8:1027- 1061, 2007.","title":"Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis","authors":"Masashi Sugiyama, Sugiyama","year":"2007","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Yu, 2013] Yaoliang Yu. Better approximation and faster algorithm using the proximal average. In Advances in Neural Information Processing Systems 26, pages 458-466, 2013.","title":"Better approximation and faster algorithm using the proximal average","authors":"Yu ;, Yaoliang Yu","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhong and Kwok, 2014] Wenliang Zhong and James T. K- wok. Gradient descent with proximal average for non- convex and composite regularization. In Proceedings of the 28th AAAI Conference on Artificial Intelligence, pages 2206-2212, 2014.","title":"Gradient descent with proximal average for nonconvex and composite regularization","authors":"Wenliang Zhong, James T Kwok","year":"2014","venue":"Proceedings of the 28th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhu and Martínez, 2006] Manli Zhu and Aleix M. Martínez. Subclass discriminant analysis. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 28(8):1274-1286, 2006.","title":"Subclass discriminant analysis","authors":"Manli Zhu, Aleix M Martínez","year":"2006","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}