{"type":"ScholarlyArticle","title":"Generalized Discriminative Deep Non-Negative Matrix Factorization Based on Latent Feature and Basis Learning","authors":[{"name":"Zijian Yang","slug":"zijian-yang","given":"Zijian","family":"Yang","affiliation":"ShanghaiTech University"},{"name":"Zhiwei Li","slug":"zhiwei-li","given":"Zhiwei","family":"Li","affiliation":"ShanghaiTech University"},{"name":"Lu Sun","slug":"lu-sun","given":"Lu","family":"Sun","affiliation":"ShanghaiTech University"}],"abstract":"As a powerful tool for data representation, deep NMF has attracted much attention in recent years. Current deep NMF builds the multi-layer structure by decomposing either basis matrix or feature matrix into multiple factors, and probably complicates the learning process when data is insufficient or exhibits simple structure. To overcome the limitations, a novel method called Generalized Deep Non-negative Matrix Factorization (GDNMF) is proposed, which generalizes several NMF and deep NMF methods in a unified framework. GDNMF simultaneously performs decomposition on both features and bases, which learns a hierarchical data representation based on multi-level basis. To further improve the latent representation and enhance its flexibility, GDNMF mutually reinforces shallow linear model and deep non-linear model. Moreover, semi-supervised GDNMF is proposed by treating partial label information as soft constraints in the multi-layer structure. An efficient two-phase optimization algorithm is developed, and experiments on five real-world datesets verify its superior performance compared with state-of-the-art methods.","doi":"10.24963/ijcai.2023/499","url":"https://ijcai.org/proceedings/2023/499","pdf":"https://ijcai.org/proceedings/2023/0499.pdf","markdown":"https://ijcai.org/proceedings/2023/499.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/499","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":499,"pages":{"first":4486,"last":4494},"keywords":["Machine Learning: ML: Weakly supervised learning","Machine Learning: ML: Clustering"],"videos":[],"references":[{"position":1,"raw":"[Babaee et al., 2016] Mohammadreza Babaee, Stefanos Tsoukalas, Maryam Babaee, Gerhard Rigoll, and Mihai Datcu. Discriminative nonnegative matrix factoriza- tion for dimensionality reduction. Neurocomputing, 173:212-223, 2016.","title":"Discriminative nonnegative matrix factorization for dimensionality reduction","authors":"Mohammadreza Babaee, Stefanos Tsoukalas, Maryam Babaee, Gerhard Rigoll, Mihai Datcu","year":"2016","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baraniuk, 2011] Richard G Baraniuk. More is less: Signal processing and the data deluge. Science, 331(6018):717- 719, 2011.","title":"More is less: Signal processing and the data deluge","authors":"G Richard, Baraniuk","year":"2011","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Buckler et al., 2018] Mark Buckler, Philip Bedoukian, Suren Jayasuriya, and Adrian Sampson. Eva 2 : Exploit- ing temporal redundancy in live computer vision. In 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), pages 533-546. IEEE, 2018.","title":"Eva 2 : Exploiting temporal redundancy in live computer vision","authors":"Mark Buckler, Philip Bedoukian, Suren Jayasuriya, Adrian Sampson","year":"2018","venue":"ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2008] Deng Cai, Xiaofei He, Xiaoyun Wu, and Jiawei Han. Non-negative matrix factorization on man- ifold. In 2008 eighth IEEE international conference on data mining, pages 63-72. IEEE, 2008.","title":"Non-negative matrix factorization on manifold","authors":"Deng Cai, Xiaofei He, Xiaoyun Wu, Jiawei Han","year":"2008","venue":"2008 eighth IEEE international conference on data mining","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cai et al., 2010] Deng Cai, Xiaofei He, Jiawei Han, and Thomas S Huang. Graph regularized nonnegative matrix factorization for data representation. IEEE transactions on pattern analysis and machine intelligence, 33(8):1548- 1560, 2010.","title":"Graph regularized nonnegative matrix factorization for data representation","authors":"Deng Cai, Xiaofei He, Jiawei Han, Thomas S Huang","year":"2010","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2019] Zhensong Chen, Yong Shi, and Zhiquan Qi. Constrained matrix factorization for semi-weakly learning with label proportions. Pattern Recognition, 91:13-24, 2019.","title":"Constrained matrix factorization for semi-weakly learning with label proportions","authors":"Chen","year":"2019","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2022] Wen-Sheng Chen, Qianwen Zeng, and Binbin Pan. A survey of deep nonnegative matrix factor- ization. Neurocomputing, 491:305-320, 2022.","title":"A survey of deep nonnegative matrix factorization","authors":"Wen-Sheng Chen, Qianwen Zeng, Binbin Pan","year":"2022","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ding et al., 2008] Chris HQ Ding, Tao Li, and Michael I Jordan. Convex and semi-nonnegative matrix factoriza- tions. IEEE transactions on pattern analysis and machine intelligence, 32(1):45-55, 2008.","title":"Convex and semi-nonnegative matrix factorizations","authors":"Tao Chris Hq Ding, Michael I Li, Jordan","year":"2008","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 2018] Hao Fang, Aihua Li, Tao Wang, Hong- wei Chang, and Huoxi Xu. Hyperspectral unmixing using graph-regularized and sparsity-constrained deep nmf. In Optical Sensing and Imaging Technologies and Applica- tions, volume 10846, pages 664-672. SPIE, 2018.","title":"Hyperspectral unmixing using graph-regularized and sparsity-constrained deep nmf","authors":"Aihua Hao Fang, Tao Li, Hongwei Wang, Huoxi Chang, Xu","year":"2018","venue":"Optical Sensing and Imaging Technologies and Applications","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guan et al., 2012] Naiyang Guan, Dacheng Tao, Zhigang Luo, and Bo Yuan. Nenmf: An optimal gradient method for nonnegative matrix factorization. IEEE Transactions on Signal Processing, 60(6):2882-2898, 2012.","title":"Nenmf: An optimal gradient method for nonnegative matrix factorization","authors":"Naiyang Guan, Dacheng Tao, Zhigang Luo, Bo Yuan","year":"2012","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He and Niyogi, 2003] Xiaofei He and Partha Niyogi. Local- ity preserving projections. Advances in neural information processing systems, 16, 2003.","title":"Locality preserving projections","authors":"Xiaofei He, Partha Niyogi","year":"2003","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Karczmarek et al., 2019] P Karczmarek, W Pedrycz, A Kiersztyn, and M Dolecki. A comprehensive exper- imental comparison of the aggregation techniques for face recognition. Iranian Journal of Fuzzy Systems, 16(4):1-19, 2019.","title":"A comprehensive experimental comparison of the aggregation techniques for face recognition","authors":"Karczmarek, Pedrycz, M Kiersztyn, Dolecki","year":"2019","venue":"Iranian Journal of Fuzzy Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lan et al., 2014] Long Lan, Naiyang Guan, Xiang Zhang, Dacheng Tao, and Zhigang Luo. Soft-constrained nonneg- ative matrix factorization via normalization. In 2014 Inter- national Joint Conference on Neural Networks (IJCNN), pages 3025-3030. IEEE, 2014.","title":"Soft-constrained nonneg-ative matrix factorization via normalization","authors":"Long Lan, Naiyang Guan, Xiang Zhang, Dacheng Tao, Zhigang Luo","year":"2014","venue":"2014 International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lee and Seung, 1999] Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix fac- torization. Nature, 401(6755):788-791, 1999.","title":"Learning the parts of objects by non-negative matrix factorization","authors":"D Daniel, H Lee, Seung Sebastian","year":"1999","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li and Yuan, 2004] Ming Li and Baozong Yuan. A novel statistical linear discriminant analysis for image matrix: two-dimensional fisherfaces. In Proceedings 7th Interna- tional Conference on Signal Processing, 2004. Proceed- ings. ICSP'04. 2004., volume 2, pages 1419-1422. IEEE, 2004.","title":"A novel statistical linear discriminant analysis for image matrix: two-dimensional fisherfaces","authors":"Li, Yuan, Ming Li, Baozong Yuan","year":"2004","venue":"Proceedings 7th International Conference on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2011] Haifeng Liu, Zhaohui Wu, Xuelong Li, Deng Cai, and Thomas S Huang. Constrained nonnega- tive matrix factorization for image representation. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 34(7):1299-1311, 2011.","title":"Constrained nonnegative matrix factorization for image representation","authors":"Haifeng Liu, Zhaohui Wu, Xuelong Li, Deng Cai, Thomas S Huang","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2017] Yalin Liu, Naiyang Guan, and Jie Liu. Deep transductive nonnegative matrix factorization for speech separation. In 2017 16th IEEE International Con- ference on Machine Learning and Applications (ICMLA), pages 249-254. IEEE, 2017.","title":"Deep transductive nonnegative matrix factorization for speech separation","authors":"Yalin Liu, Naiyang Guan, Jie Liu","year":"2017","venue":"16th IEEE International Conference on Machine Learning and Applications (ICMLA)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Marutho et al., 2018] Dhendra Marutho, Sunarna Hendra Handaka, Ekaprana Wijaya, et al. The determination of cluster number at k-mean using elbow method and purity evaluation on headline news. In 2018 international semi- nar on application for technology of information and com- munication, pages 533-538. IEEE, 2018.","title":"The determination of cluster number at k-mean using elbow method and purity evaluation on headline news","authors":"Marutho","year":"2018","venue":"2018 international seminar on application for technology of information and communication","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Meng et al., 2019] Yang Meng, Ronghua Shang, Fanhua Shang, Licheng Jiao, Shuyuan Yang, and Rustam Stolkin. Semi-supervised graph regularized deep nmf with bi- orthogonal constraints for data representation. IEEE transactions on neural networks and learning systems, 31(9):3245-3258, 2019.","title":"Semi-supervised graph regularized deep nmf with biorthogonal constraints for data representation","authors":"Yang Meng, Ronghua Shang, Fanhua Shang, Licheng Jiao, Shuyuan Yang, Rustam Stolkin","year":"2019","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Moon and Lee, 2021] Sehwan Moon and Hyunju Lee. Jd- snmf: Joint deep semi-non-negative matrix factorization for learning integrative representation of molecular signals in alzheimer's disease. Journal of personalized medicine, 11(8):686, 2021.","title":"Sehwan Moon and Hyunju Lee. Jdsnmf: Joint deep semi-non-negative matrix factorization for learning integrative representation of molecular signals in alzheimer's disease","authors":"Lee Moon","year":"2021","venue":"Journal of personalized medicine","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Moore, 1981] Bruce Moore. Principal component analy- sis in linear systems: Controllability, observability, and model reduction. IEEE transactions on automatic control, 26(1):17-32, 1981.","title":"Principal component analysis in linear systems: Controllability, observability, and model reduction","authors":"Bruce Moore","year":"1981","venue":"IEEE transactions on automatic control","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Peng et al., 2021a] Chong Peng, Zhilu Zhang, Zhao Kang, Chenglizhao Chen, and Qiang Cheng. Nonnegative matrix factorization with local similarity learning. Information Sciences, 562:325-346, 2021.","title":"Nonnegative matrix factorization with local similarity learning","authors":"Peng","year":"2021","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Peng et al., 2021b] Siyuan Peng, Wee Ser, Badong Chen, and Zhiping Lin. Robust semi-supervised nonnegative ma- trix factorization for image clustering. Pattern Recogni- tion, 111:107683, 2021.","title":"Robust semi-supervised nonnegative matrix factorization for image clustering","authors":"Peng","year":"2021","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sabzalian and Abolghasemi, 2018] B Sabzalian and V Abolghasemi. Iterative weighted non-smooth non- negative matrix factorization for face recognition. International Journal of Engineering, 31(10):1698-1707, 2018.","title":"Iterative weighted non-smooth nonnegative matrix factorization for face recognition","authors":"B Sabzalian, Abolghasemi","year":"2018","venue":"International Journal of Engineering","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shi et al., 2018] Tian Shi, Kyeongpil Kang, Jaegul Choo, and Chandan K Reddy. Short-text topic modeling via non- negative matrix factorization enriched with local word- context correlations. In Proceedings of the 2018 World Wide Web Conference, pages 1105-1114, 2018.","title":"Short-text topic modeling via nonnegative matrix factorization enriched with local wordcontext correlations","authors":"Tian Shi, Kyeongpil Kang, Jaegul Choo, Chandan K Reddy","year":"2018","venue":"Proceedings of the 2018 World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tong et al., 2019] Ming Tong, Yiran Chen, Mengao Zhao, Haili Bu, and Shengnan Xi. A deep discriminative and robust nonnegative matrix factorization network method with soft label constraint. Neural Computing and Applica- tions, 31(11):7447-7475, 2019.","title":"A deep discriminative and robust nonnegative matrix factorization network method with soft label constraint","authors":"Ming Tong, Yiran Chen, Mengao Zhao, Haili Bu, Shengnan Xi","year":"2019","venue":"Neural Computing and Applications","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Trigeorgis et al., 2016] George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou, and Björn W Schuller. A deep matrix factorization method for learning attribute representations. IEEE transactions on pattern analysis and machine intelligence, 39(3):417-429, 2016.","title":"A deep matrix factorization method for learning attribute representations","authors":"George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou, Björn W Schuller","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2021] Wei Wang, Feiyu Chen, Yongxin Ge, Sheng Huang, Xiaohong Zhang, and Dan Yang. Discrim- inative deep semi-nonnegative matrix factorization net- work with similarity maximization for unsupervised fea- ture learning. Pattern Recognition Letters, 149:157-163, 2021.","title":"Discriminative deep semi-nonnegative matrix factorization network with similarity maximization for unsupervised feature learning","authors":"Wei Wang, Feiyu Chen, Yongxin Ge, Sheng Huang, Xiaohong Zhang, Dan Yang","year":"2021","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wisdom et al., 2017] Scott Wisdom, Thomas Powers, James Pitton, and Les Atlas. Deep recurrent nmf for speech separation by unfolding iterative thresholding. In 2017 IEEE Workshop on Applications of Signal Process- ing to Audio and Acoustics (WASPAA), pages 254-258. IEEE, 2017.","title":"Deep recurrent nmf for speech separation by unfolding iterative thresholding","authors":"Scott Wisdom, Thomas Powers, James Pitton, Les Atlas","year":"2017","venue":"IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wu et al., 2019] Wenhui Wu, Sam Kwong, Junhui Hou, Yuheng Jia, and Horace Ho Shing Ip. Simultane- ous dimensionality reduction and classification via dual embedding regularized nonnegative matrix factorization. IEEE Transactions on Image Processing, 28(8):3836- 3847, 2019.","title":"Simultaneous dimensionality reduction and classification via dual embedding regularized nonnegative matrix factorization","authors":"Wenhui Wu, Sam Kwong, Junhui Hou, Yuheng Jia, Horace Ho, Shing Ip","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2003] Wei Xu, Xin Liu, and Yihong Gong. Doc- ument clustering based on non-negative matrix factoriza- tion. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in infor- maion retrieval, pages 267-273, 2003.","title":"Document clustering based on non-negative matrix factorization","authors":"Wei Xu, Xin Liu, Yihong Gong","year":"2003","venue":"Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yi et al., 2019] Yugen Yi, Jianzhong Wang, Wei Zhou, Caixia Zheng, Jun Kong, and Shaojie Qiao. Non-negative matrix factorization with locality constrained adaptive graph. IEEE Transactions on circuits and systems for video technology, 30(2):427-441, 2019.","title":"Non-negative matrix factorization with locality constrained adaptive graph","authors":"Yugen Yi, Jianzhong Wang, Wei Zhou, Caixia Zheng, Jun Kong, Shaojie Qiao","year":"2019","venue":"IEEE Transactions on circuits and systems for video technology","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhao et al., 2021] Yang Zhao, Huiyang Wang, and Jihong Pei. Deep non-negative matrix factorization architecture based on underlying basis images learning. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 43(6):1897-1913, 2021.","title":"Deep non-negative matrix factorization architecture based on underlying basis images learning","authors":"Yang Zhao, Huiyang Wang, Jihong Pei","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhao et al., 2022] Yang Zhao, Furong Deng, Jihong Pei, and Xuan Yang. Progressive deep non-negative matrix fac- torization architecture with graph convolution-based basis image reorganization. Pattern Recognition, 132:108984, 2022.","title":"Progressive deep non-negative matrix factorization architecture with graph convolution-based basis image reorganization","authors":"Yang Zhao, Furong Deng, Jihong Pei, Xuan Yang","year":"2022","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}