{"type":"ScholarlyArticle","title":"Reconstruction Weighting Principal Component Analysis with Fusion Contrastive Learning","authors":[{"name":"Qianqian Wang","slug":"qianqian-wang","given":"Qianqian","family":"Wang","affiliation":"Key Laboratory of Measurement and Control of Complex Systems of Engineering , Ministry of Education , Southeast University , Nanjing , China, 210096."},{"name":"Meiling Liu","slug":"meiling-liu","given":"Meiling","family":"Liu","affiliation":"School of Telecommunications Engineering , Xidian University , Xi'an , China"},{"name":"Wei Feng","slug":"wei-feng","given":"Wei","family":"Feng","affiliation":"School of Computer Science and Technology , Xi'an Jiaotong University , Xi'an , China"},{"name":"Mengping Jiang","slug":"mengping-jiang","given":"Mengping","family":"Jiang","affiliation":"School of Telecommunications Engineering , Xidian University , Xi'an , China"},{"name":"Haiming Xu","slug":"haiming-xu","given":"Haiming","family":"Xu","affiliation":"School of Telecommunications Engineering , Xidian University , Xi'an , China"},{"name":"Quanxue Gao","slug":"quanxue-gao","given":"Quanxue","family":"Gao","affiliation":"School of Telecommunications Engineering , Xidian University , Xi'an , China"}],"abstract":"Principal component analysis (PCA) is a popular unsupervised dimensionality reduction method to extract the principal components of data. However, there are two problems with the existing PCA: (1) Traditional PCA methods treat each sample equally and ignore sample differences. (2) They fail to extract the discriminative features required by recognition tasks. To overcome these problems, we incorporate contrastive learning to develop a novel weighted PCA algorithm. Specifically, our method weights the reconstruction error of individual samples to reduce the influence of outliers. Besides, it integrates contrastive learning into PCA to increase inter-class distances and reduce intra-class distance, which helps to improve PCA's discriminative capability. We further develop an unsupervised strategy to select positive and negative samples, which eliminates pseudo-negative samples guided by clustering labels. Specifically, it employs confidence level to distinguish positive and negative samples. Consequently, our method achieves higher recognition accuracy on benchmark datasets.","doi":"10.24963/ijcai.2024/563","url":"https://ijcai.org/proceedings/2024/563","pdf":"https://ijcai.org/proceedings/2024/0563.pdf","markdown":"https://ijcai.org/proceedings/2024/563.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/563","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":563,"pages":{"first":5091,"last":5098},"keywords":["Computer Vision: CV: Recognition (object detection, categorization)","Machine Learning: ML: Feature extraction, selection and dimensionality reduction","Machine Learning: ML: Classification","Computer Vision: CV: Representation learning"],"videos":[],"references":[{"position":1,"raw":"[Abdi and Williams, 2010] Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisci- plinary reviews: computational statistics, 2(4):433-459, 2010.","title":"Principal component analysis","authors":"Williams ; Abdi, Hervé Abdi, Lynne J Williams","year":"2010","venue":"Wiley interdisciplinary reviews: computational statistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arora et al., 2019] Sanjeev Arora, Hrishikesh Khande- parkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saunshi. A theoretical analysis of contrastive unsupervised representation learning. arXiv preprint arXiv:1902.09229, 2019.","title":"A theoretical analysis of contrastive unsupervised representation learning","authors":"Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, Nikunj Saunshi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Belhumeur et al., 1997] Peter N. Belhumeur, Joao P Hes- panha, and David J. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE TPAMI, 19(7):711-720, 1997.","title":"Eigenfaces vs. fisherfaces: Recognition using class specific linear projection","authors":"N Peter, Joao P Belhumeur, David J Hespanha, Kriegman","year":"1997","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ding et al., 2006] Chris Ding, Ding Zhou, Xiaofeng He, and Hongyuan Zha. R 1-pca: rotational invariant l 1-norm principal component analysis for robust subspace factor- ization. In ICML, pages 281-288, 2006.","title":"R 1-pca: rotational invariant l 1-norm principal component analysis for robust subspace factorization","authors":"Chris Ding, Ding Zhou, Xiaofeng He, Hongyuan Zha","year":"2006","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Doersch et al., 2015] Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learn- ing by context prediction. In IEEE ICCV, pages 1422- 1430, 2015.","title":"Unsupervised visual representation learning by context prediction","authors":"Carl Doersch, Abhinav Gupta, Alexei A Efros","year":"2015","venue":"IEEE ICCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Georghiades et al., 2001] Athinodoros S. Georghiades, Pe- ter N. Belhumeur, and David J. Kriegman. From few to many: Illumination cone models for face recognition un- der variable lighting and pose. IEEE TPAMI, 23(6):643- 660, 2001.","title":"From few to many: Illumination cone models for face recognition under variable lighting and pose","authors":"Athinodoros S Georghiades, Peter N Belhumeur, David J Kriegman","year":"2001","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He and Niyogi, 2003] Xiaofei He and Partha Niyogi. Lo- cality preserving projections. NeurIPS, 16, 2003.","title":"Locality preserving projections","authors":"Xiaofei He, Partha Niyogi","year":"2003","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2005] Xiaofei He, Deng Cai, Shuicheng Yan, and Hong-Jiang Zhang. Neighborhood preserving embedding. In IEEE ICCV, volume 2, pages 1208-1213. IEEE, 2005.","title":"Neighborhood preserving embedding","authors":"Xiaofei He, Deng Cai, Shuicheng Yan, Hong-Jiang Zhang","year":"2005","venue":"IEEE ICCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2011] Ran He, Bao-Gang Hu, Wei-Shi Zheng, and Xiang-Wei Kong. Robust principal component analy- sis based on maximum correntropy criterion. IEEE TIP, 20(6):1485-1494, 2011.","title":"Robust principal component analysis based on maximum correntropy criterion","authors":"Ran He, Bao-Gang Hu, Wei-Shi Zheng, Xiang-Wei Kong","year":"2011","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jin et al., 2001] Zhong Jin, Jing-Yu Yang, Zhong-Shan Hu, and Zhen Lou. Face recognition based on the uncorrelated discriminant transformation. PR, 34(7):1405-1416, 2001.","title":"Face recognition based on the uncorrelated discriminant transformation","authors":"Jin Zhong, Jing-Yu Yang, Zhong-Shan Hu, Zhen Lou","year":"2001","venue":"PR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ke and Kanade, 2005] Qifa Ke and Takeo Kanade. Robust l/sub 1/norm factorization in the presence of outliers and missing data by alternative convex programming. In IEEE CVPR, volume 1, pages 739-746. IEEE, 2005.","title":"Robust l/sub 1/norm factorization in the presence of outliers and missing data by alternative convex programming","authors":"Kanade ; ] Qifa Ke, Takeo Ke, Kanade","year":"2005","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kwak, 2008] Nojun Kwak. Principal component anal- ysis based on l1-norm maximization. IEEE TPAMI, 30(9):1672-1680, 2008.","title":"Principal component analysis based on l1-norm maximization","authors":"Nojun Kwak, Kwak","year":"2008","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2018] Xuelong Li, Han Zhang, Rui Zhang, Yun Liu, and Feiping Nie. Generalized uncorrelated regres- sion with adaptive graph for unsupervised feature selec- tion. IEEE TNNLS, 30(5):1587-1595, 2018.","title":"Generalized uncorrelated regression with adaptive graph for unsupervised feature selection","authors":"Xuelong Li, Han Zhang, Rui Zhang, Yun Liu, Feiping Nie","year":"2018","venue":"IEEE TNNLS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Martinez and Benavente, 1998] Aleix Martinez and Robert Benavente. The ar face database: Cvc technical report, 24. CVC, 1998.","title":"The ar face database: Cvc technical report","authors":"Benavente Martinez, Aleix Martinez, Robert Benavente","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nie et al., 2014] Feiping Nie, Jianjun Yuan, and Heng Huang. Optimal mean robust principal component anal- ysis. In ICML, pages 1062-1070. PMLR, 2014.","title":"Optimal mean robust principal component analysis","authors":"Nie","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nie et al., 2020] Feiping Nie, Danyang Wu, Rong Wang, and Xuelong Li. Truncated robust principle component analysis with a general optimization framework. IEEE TPAMI, 44(2):1081-1097, 2020.","title":"Truncated robust principle component analysis with a general optimization framework","authors":"Nie","year":"2020","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sim et al., 2002] Terence Sim, Simon Baker, and Maan Bsat. The cmu pose, illumination, and expression (pie) database. In IEEE international conference on automatic face gesture recognition, pages 53-58. IEEE, 2002.","title":"The cmu pose, illumination, and expression (pie) database","authors":"Terence Sim, Simon Baker, Maan Bsat","year":"2002","venue":"IEEE international conference on automatic face gesture recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sun et al., 2019] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197, 2019.","title":"Rotate: Knowledge graph embedding by relational rotation in complex space","authors":"Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, Jian Tang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Turk and Pentland, 1991] Matthew Turk and Alex Pentland. Eigenfaces for recognition. Journal of cognitive neuro- science, 3(1):71-86, 1991.","title":"Matthew Turk and Alex Pentland. Eigenfaces for recognition","authors":"Pentland Turk","year":"1991","venue":"Journal of cognitive neuroscience","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2014] Rong Wang, Feiping Nie, Xiaojun Yang, Feifei Gao, and Minli Yao. Robust 2dpca with non-greedy ℓ 1 -norm maximization for image analysis. IEEE TC, 45(5):1108-1112, 2014.","title":"Robust 2dpca with non-greedy ℓ 1 -norm maximization for image analysis","authors":"Rong Wang, Feiping Nie, Xiaojun Yang, Feifei Gao, Minli Yao","year":"2014","venue":"IEEE TC","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2017a] Qianqian Wang, Quanxue Gao, Xinbo Gao, and Feiping Nie. Angle principal component analy- sis. In IJCAI, pages 2936-2942, 2017.","title":"Angle principal component analysis","authors":"Wang","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/409"},{"position":22,"raw":"[Wang et al., 2017b] Qianqian Wang, Quanxue Gao, Xinbo Gao, and Feiping Nie. ℓ 2,p -norm based pca for image recognition. IEEE TIP, 27(3):1336-1346, 2017.","title":"ℓ 2,p -norm based pca for image recognition","authors":"Wang","year":"2017","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2020] Xudong Wang, Ziwei Liu, and X Yu Stella. Unsupervised feature learning by cross-level dis- crimination between instances and groups. arXiv preprint arXiv:2008.03813, 1(2):4, 2020.","title":"Unsupervised feature learning by cross-level discrimination between instances and groups","authors":"Xudong Wang, Ziwei Liu, X Yu, Stella","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wei et al., 2020] Chen Wei, Huiyu Wang, Wei Shen, and Alan Yuille. Co2: Consistent contrast for unsuper- vised visual representation learning. arXiv preprint arXiv:2010.02217, 2020.","title":"Co2: Consistent contrast for unsupervised visual representation learning","authors":"Chen Wei, Huiyu Wang, Wei Shen, Alan Yuille","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ye et al., 2019] Mang Ye, Xu Zhang, Pong C Yuen, and Shih-Fu Chang. Unsupervised embedding learning via in- variant and spreading instance feature. In IEEE CVPR, pages 6210-6219, 2019.","title":"Unsupervised embedding learning via invariant and spreading instance feature","authors":"Mang Ye, Xu Zhang, Pong C Yuen, Shih-Fu Chang","year":"2019","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang and Tong, 2019] Rui Zhang and Hanghang Tong. Robust principal component analysis with adaptive neigh- bors. NeurIPS, 32, 2019.","title":"Robust principal component analysis with adaptive neighbors","authors":"Rui Zhang, Hanghang Tong","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2020] Rui Zhang, Yunxing Zhang, and Xue- long Li. Unsupervised feature selection via adaptive graph learning and constraint. IEEE TNNLS, 33(3):1355-1362, 2020.","title":"Unsupervised feature selection via adaptive graph learning and constraint","authors":"Rui Zhang, Yunxing Zhang, Xuelong Li","year":"2020","venue":"IEEE TNNLS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2022] Hongjie Zhang, Wenwen Qiang, Jinxin Zhang, Yingyi Chen, and Ling Jing. Unified fea- ture extraction framework based on contrastive learning. Knowledge-Based Systems, 258:110028, 2022.","title":"Unified feature extraction framework based on contrastive learning","authors":"Hongjie Zhang, Wenwen Qiang, Jinxin Zhang, Yingyi Chen, Ling Jing","year":"2022","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}