{"type":"ScholarlyArticle","title":"Online Harmonizing Gradient Descent for Imbalanced Data Streams One-Pass Classification","authors":[{"name":"Han Zhou","slug":"han-zhou","given":"Han","family":"Zhou","affiliation":"The School of Automation , Chongqing University , Chongqing , China, 400044."},{"name":"Hongpeng Yin","slug":"hongpeng-yin","given":"Hongpeng","family":"Yin","affiliation":"The School of Automation , Chongqing University , Chongqing , China, 400044."},{"name":"Xuanhong Deng","slug":"xuanhong-deng","given":"Xuanhong","family":"Deng","affiliation":"The School of Automation , Chongqing University , Chongqing , China, 400044."},{"name":"Yuyu Huang","slug":"yuyu-huang","given":"Yuyu","family":"Huang","affiliation":"The School of Automation , Chongqing University , Chongqing , China, 400044."}],"abstract":"Many real-world streaming data are sequentially collected over time and with skew-distributed classes. In this situation, online learning models may tend to favor samples from majority classes, making the wrong decisions for those from minority classes. Previous methods try to balance the instance number of different classes or assign asymmetric cost values. They usually require data-buffers to store streaming data or pre-defined cost parameters. This study alternatively shows that the imbalance of instances can be implied by the imbalance of gradients. Then, we propose the Online Harmonizing Gradient Descent (OHGD) for one-pass online classification. By harmonizing the gradient magnitude occurred by different classes, the method avoids the bias of the proposed method in favor of the majority class. Specifically, OHGD requires no data-buffer, extra parameters, or prior knowledge. It also handles imbalanced data streams the same way that it would handle balanced data streams, which facilitates its easy implementation. On top of a few common and mild assumptions, the theoretical analysis proves that OHGD enjoys a satisfying sub-linear regret bound. Extensive experimental results demonstrate the high efficiency and effectiveness in handling imbalanced data streams.","doi":"10.24963/ijcai.2023/274","url":"https://ijcai.org/proceedings/2023/274","pdf":"https://ijcai.org/proceedings/2023/0274.pdf","markdown":"https://ijcai.org/proceedings/2023/274.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/274","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Data Mining","track":"Main Track","number":274,"pages":{"first":2468,"last":2475},"keywords":["Data Mining: DM: Class imbalance and unequal cost","Data Mining: DM: Mining data streams","Machine Learning: ML: Classification"],"videos":[],"references":[{"position":1,"raw":"[Bernardo et al., 2020] Alessio Bernardo, Heitor Murilo Gomes, Jacob Montiel, Bernhard Pfahringer, Albert Bifet, and Emanuele Della Valle. C-smote: Continuous synthetic minority oversampling for evolving data streams. In 2020 IEEE International Conference on Big Data (Big Data), pages 483-492. IEEE, 2020.","title":"C-smote: Continuous synthetic minority oversampling for evolving data streams","authors":"Alessio Bernardo, Murilo Heitor, Jacob Gomes, Bernhard Montiel, Albert Pfahringer, Emanuele Della Bifet, Valle","year":"2020","venue":"IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chawla et al., 2002] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: syn- thetic minority over-sampling technique. Journal of artifi- cial intelligence research, 16:321-357, 2002.","title":"Smote: synthetic minority over-sampling technique","authors":"Kevin W Nitesh V Chawla, Lawrence O Bowyer, Philip Hall, Kegelmeyer","year":"2002","venue":"Journal of artificial intelligence research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Crammer et al., 2006] Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. On- line passive-aggressive algorithms. J. Mach. Learn. Res., 7:551-585, dec 2006.","title":"Online passive-aggressive algorithms","authors":"Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, Yoram Singer","year":"2006","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dredze et al., 2008] Mark Dredze, Koby Crammer, and Fer- nando Pereira. Confidence-weighted linear classification. In Proceedings of the 25th international conference on Machine learning, pages 264-271, 2008.","title":"Confidence-weighted linear classification","authors":"Mark Dredze, Koby Crammer, Fernando Pereira","year":"2008","venue":"Proceedings of the 25th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Du et al., 2021] Hongle Du, Yan Zhang, Ke Gang, Lin Zhang, and Yeh-Cheng Chen. Online ensemble learning algorithm for imbalanced data stream. Applied Soft Com- puting, 107:107378, 2021.","title":"Online ensemble learning algorithm for imbalanced data stream","authors":"Hongle Du, Yan Zhang, Ke Gang, Lin Zhang, Yeh-Cheng Chen","year":"2021","venue":"Applied Soft Computing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Galar et al., 2011] Mikel Galar, Alberto Fernandez, Edurne Barrenechea, Humberto Bustince, and Francisco Herrera. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4):463-484, 2011.","title":"A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches","authors":"Mikel Galar, Alberto Fernandez, Edurne Barrenechea, Humberto Bustince, Francisco Herrera","year":"2011","venue":"IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Grzyb et al., 2021] Joanna Grzyb, Jakub Klikowski, and Michał Woźniak. Hellinger distance weighted ensemble for imbalanced data stream classification. Journal of Com- putational Science, 51:101314, 2021.","title":"Hellinger distance weighted ensemble for imbalanced data stream classification","authors":"Joanna Grzyb, Jakub Klikowski, Michał Woźniak","year":"2021","venue":"Journal of Computational Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hoi et al., 2021] Steven CH Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao. Online learning: A comprehensive sur- vey. Neurocomputing, 459:249-289, 2021.","title":"Online learning: A comprehensive survey","authors":"C H Steven, Doyen Hoi, Jing Sahoo, Peilin Lu, Zhao","year":"2021","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Klikowski and Woźniak, 2022] Jakub Klikowski and Michał Woźniak. Deterministic sampling classifier with weighted bagging for drifted imbalanced data stream classification. Applied Soft Computing, page 108855, 2022.","title":"Deterministic sampling classifier with weighted bagging for drifted imbalanced data stream classification","authors":"Woźniak Klikowski, Jakub Klikowski, Michał Woźniak","year":"2022","venue":"Applied Soft Computing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Loezer et al., 2020] Lucas Loezer, Fabrício Enembreck, Jean Paul Barddal, and Alceu de Souza Britto Jr. Cost- sensitive learning for imbalanced data streams. In Pro- ceedings of the 35th Annual ACM Symposium on Applied Computing, pages 498-504, 2020.","title":"Costsensitive learning for imbalanced data streams","authors":"Lucas Loezer, Fabrício Enembreck, Jean Paul Barddal, Alceu De, Souza Britto","year":"2020","venue":"Proceedings of the 35th Annual ACM Symposium on Applied Computing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lu et al., 2019] Yang Lu, Yiu-Ming Cheung, and Yuan Yan Tang. Adaptive chunk-based dynamic weighted major- ity for imbalanced data streams with concept drift. IEEE Transactions on Neural Networks and Learning Systems, 31(8):2764-2778, 2019.","title":"Adaptive chunk-based dynamic weighted majority for imbalanced data streams with concept drift","authors":"Yang Lu, Yiu-Ming Cheung, Yuan Yan Tang","year":"2019","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Razavi-Far et al., 2021] Roozbeh Razavi-Far, Maryam Farajzadeh-Zanajni, Boyu Wang, Mehrdad Saif, and Shiladitya Chakrabarti. Imputation-based ensemble tech- niques for class imbalance learning. IEEE Transactions on Knowledge and Data Engineering, 33(5):1988-2001, 2021.","title":"Imputation-based ensemble techniques for class imbalance learning","authors":"Roozbeh Razavi-Far, Maryam Farajzadeh-Zanajni, Boyu Wang, Mehrdad Saif, Shiladitya Chakrabarti","year":"2021","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Sovilj et al., 2020] Dušan Sovilj, Paul Budnarain, Scott Sanner, Geoff Salmon, and Mohan Rao. A comparative evaluation of unsupervised deep architectures for intrusion detection in sequential data streams. Expert Systems with Applications, 159:113577, 2020.","title":"A comparative evaluation of unsupervised deep architectures for intrusion detection in sequential data streams","authors":"Dušan Sovilj, Paul Budnarain, Scott Sanner, Geoff Salmon, Mohan Rao","year":"2020","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sun et al., 2007] Yanmin Sun, Mohamed S Kamel, An- drew KC Wong, and Yang Wang. Cost-sensitive boosting for classification of imbalanced data. Pattern recognition, 40(12):3358-3378, 2007.","title":"Cost-sensitive boosting for classification of imbalanced data","authors":"Yanmin Sun, Mohamed S Kamel, Andrew Kc Wong, Yang Wang","year":"2007","venue":"Pattern recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Varnali, 2021] Kaan Varnali. Online behavioral advertising: An integrative review. Journal of Marketing Communica- tions, 27(1):93-114, 2021.","title":"Kaan Varnali. Online behavioral advertising: An integrative review","authors":"Varnali","year":"2021","venue":"Journal of Marketing Communications","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang and Pineau, 2016] Boyu Wang and Joelle Pineau. Online bagging and boosting for imbalanced data streams. IEEE Transactions on Knowledge and Data Engineering, 28(12):3353-3366, 2016.","title":"Online bagging and boosting for imbalanced data streams","authors":"Wang, Pineau ; ] Boyu Wang, Joelle Pineau","year":"2016","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2013a] Jialei Wang, Peilin Zhao, and Steven CH Hoi. Cost-sensitive online classification. IEEE Transactions on Knowledge and Data Engineering, 26(10):2425-2438, 2013.","title":"Cost-sensitive online classification","authors":"Wang","year":"2013","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2013b] Shuo Wang, Leandro L Minku, and Xin Yao. A learning framework for online class imbalance learning. In 2013 IEEE Symposium on Computational In- telligence and Ensemble Learning (CIEL), pages 36-45. IEEE, 2013.","title":"A learning framework for online class imbalance learning","authors":"Wang","year":"2013","venue":"IEEE Symposium on Computational Intelligence and Ensemble Learning (CIEL)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2014] Shuo Wang, Leandro L Minku, and Xin Yao. Resampling-based ensemble methods for online class imbalance learning. IEEE Transactions on Knowledge and Data Engineering, 27(5):1356-1368, 2014.","title":"Resampling-based ensemble methods for online class imbalance learning","authors":"Shuo Wang, Leandro L Minku, Xin Yao","year":"2014","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2021] Nan Wang, Ruozhou Liang, Xibin Zhao, and Yue Gao. Cost-sensitive hypergraph learning with f- measure optimization. IEEE Transactions on Cybernetics, 2021.","title":"Cost-sensitive hypergraph learning with fmeasure optimization","authors":"Nan Wang, Ruozhou Liang, Xibin Zhao, Yue Gao","year":"2021","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yu et al., 2018] Hualong Yu, Xibei Yang, Shang Zheng, and Changyin Sun. Active learning from imbalanced data: A solution of online weighted extreme learning machine. IEEE transactions on neural networks and learning sys- tems, 30(4):1088-1103, 2018.","title":"Active learning from imbalanced data: A solution of online weighted extreme learning machine","authors":"Hualong Yu, Xibei Yang, Shang Zheng, Changyin Sun","year":"2018","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2018] Chong Zhang, Kay Chen Tan, Haizhou Li, and Geok Soon Hong. A cost-sensitive deep belief network for imbalanced classification. IEEE transactions on neural networks and learning systems, 30(1):109-122, 2018.","title":"A cost-sensitive deep belief network for imbalanced classification","authors":"Chong Zhang, Kay Chen Tan, Haizhou Li, Geok Soon Hong","year":"2018","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2019] Yifan Zhang, Peilin Zhao, Shuaicheng Niu, Qingyao Wu, Jiezhang Cao, Junzhou Huang, and Mingkui Tan. Online adaptive asymmetric active learning with limited budgets. IEEE Transactions on Knowledge and Data Engineering, 33(6):2680-2692, 2019.","title":"Online adaptive asymmetric active learning with limited budgets","authors":"Yifan Zhang, Peilin Zhao, Shuaicheng Niu, Qingyao Wu, Jiezhang Cao, Junzhou Huang, Mingkui Tan","year":"2019","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2018] Peilin Zhao, Yifan Zhang, Min Wu, Steven CH Hoi, Mingkui Tan, and Junzhou Huang. Adap- tive cost-sensitive online classification. IEEE Transac- tions on Knowledge and Data Engineering, 31(2):214- 228, 2018.","title":"Adaptive cost-sensitive online classification","authors":"Peilin Zhao, Yifan Zhang, Min Wu, C H Steven, Mingkui Hoi, Junzhou Tan, Huang","year":"2018","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2020] Han Zhou, Hongpeng Yin, Hengyi Zheng, and Yanxia Li. A survey on multi-modal social event detection. Knowledge-Based Systems, 195:105695, 2020.","title":"A survey on multi-modal social event detection","authors":"Han Zhou, Hongpeng Yin, Hengyi Zheng, Yanxia Li","year":"2020","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2022] Han Zhou, Hongpeng Yin, Dandan Zhao, and Li Cai. Incremental learning and conditional drift adaptation for non-stationary industrial process fault diag- nosis. IEEE Transactions on Industrial Informatics, 2022.","title":"Incremental learning and conditional drift adaptation for non-stationary industrial process fault diagnosis","authors":"Han Zhou, Hongpeng Yin, Dandan Zhao, Li Cai","year":"2022","venue":"IEEE Transactions on Industrial Informatics","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zinkevich, 2003] Martin Zinkevich. Online convex pro- gramming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on ma- chine learning (icml-03), pages 928-936, 2003.","title":"Online convex programming and generalized infinitesimal gradient ascent","authors":"Martin Zinkevich","year":"2003","venue":"Proceedings of the 20th international conference on machine learning (icml-03)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zong et al., 2013] Weiwei Zong, Guang-Bin Huang, and Yiqiang Chen. Weighted extreme learning machine for im- balance learning. Neurocomputing, 101:229-242, 2013.","title":"Weighted extreme learning machine for imbalance learning","authors":"Weiwei Zong, Guang-Bin Huang, Yiqiang Chen","year":"2013","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zyblewski et al., 2021] Paweł Zyblewski, Robert Sabourin, and Michał Woźniak. Preprocessed dynamic classifier ensemble selection for highly imbalanced drifted data streams. Information Fusion, 66:138-154, 2021.","title":"Preprocessed dynamic classifier ensemble selection for highly imbalanced drifted data streams","authors":"Paweł Zyblewski, Robert Sabourin, Michał Woźniak","year":"2021","venue":"Information Fusion","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}