{"type":"ScholarlyArticle","title":"Dynamic Weighted Majority for Incremental Learning of Imbalanced Data Streams with Concept Drift","authors":[{"name":"Yang Lu","slug":"yang-lu","given":"Yang","family":"Lu","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong SAR , China"},{"name":"Yiu-ming Cheung","slug":"yiu-ming-cheung","given":"Yiu-ming","family":"Cheung","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong SAR , China; HKBU Institute of Research and Continuing Education , Shenzhen , China"},{"name":"Yuan Yan Tang","slug":"yuan-yan-tang","given":"Yuan Yan","family":"Tang","affiliation":"Department of Computer and Information Science , University of Macau , Macau SAR , China"}],"abstract":"Concept drifts occurring in data streams will jeopardize the accuracy and stability of the online learning process. If the data stream is imbalanced, it will be even more challenging to detect and cure the concept drift. In the literature, these two problems have been intensively addressed separately, but have yet to be well studied when they occur together. In this paper, we propose a chunk-based incremental learning method called Dynamic Weighted Majority for Imbalance Learning (DWMIL) to deal with the data streams with concept drift and class imbalance problem. DWMIL utilizes an ensemble framework by dynamically weighting the base classifiers according to their performance on the current data chunk. Compared with the existing methods, its merits are four-fold: (1) it can keep stable for non-drifted streams and quickly adapt to the new concept; (2) it is totally incremental, i.e. no previous data needs to be stored; (3) it keeps a limited number of classifiers to ensure high efficiency; and (4) it is simple and needs only one thresholding parameter. Experiments on both synthetic and real data sets with concept drift show that DWMIL performs better than the state-of-the-art competitors, with less computational cost.","doi":"10.24963/ijcai.2017/333","url":"https://ijcai.org/proceedings/2017/333","pdf":"https://ijcai.org/proceedings/2017/0333.pdf","markdown":"https://ijcai.org/proceedings/2017/333.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/333","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra, IIIA-CSIC","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":333,"pages":{"first":2393,"last":2399},"keywords":["Machine Learning: Data Mining","Machine Learning: Online Learning"],"videos":[],"references":[{"position":1,"raw":"[Branco et al., 2016] Paula Branco, Luís Torgo, and Rita P Ribeiro. A survey of predictive modeling on imbalanced domains. ACM Computing Surveys (CSUR), 49(2):31, 2016.","title":"A survey of predictive modeling on imbalanced domains","authors":"Paula Branco, Luís Torgo, Rita P Ribeiro","year":"2016","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Breiman et al., 1984] Leo Breiman, Jerome Friedman, Charles J Stone, and Richard A Olshen. Classification and regression trees. CRC press, 1984.","title":"Classification and regression trees","authors":"Breiman","year":"1984","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and He, 2009] Sheng Chen and Haibo He. Sera: se- lectively recursive approach towards nonstationary imbal- anced stream data mining. In Proceedings of the 2009 In- ternational Joint Conference on Neural Networks, pages 522-529. IEEE, 2009.","title":"Sera: selectively recursive approach towards nonstationary imbalanced stream data mining","authors":"Sheng Chen, Haibo He","year":"2009","venue":"Proceedings of the 2009 International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen and He, 2011] Sheng Chen and Haibo He. Toward- s incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach. Evolv- ing Systems, 2(1):35-50, 2011.","title":"Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach","authors":"Sheng Chen, Haibo He","year":"2011","venue":"Evolving Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ditzler and Polikar, 2013] Gregory Ditzler and Robi Po- likar. Incremental learning of concept drift from streaming imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 25(10):2283-2301, 2013.","title":"Incremental learning of concept drift from streaming imbalanced data","authors":"Gregory Ditzler, Robi Polikar","year":"2013","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ditzler et al., 2015] Gregory Ditzler, Manuel Roveri, Ce- sare Alippi, and Robi Polikar. Learning in nonstationary environments: A survey. IEEE Computational Intelligence Magazine, 10(4):12-25, 2015.","title":"Learning in nonstationary environments: A survey","authors":"Gregory Ditzler, Manuel Roveri, Cesare Alippi, Robi Polikar","year":"2015","venue":"IEEE Computational Intelligence Magazine","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Elwell and Polikar, 2011] Ryan Elwell and Robi Polikar. Incremental learning of concept drift in nonstationary en- vironments. IEEE Transactions on Neural Networks, 22(10):1517-1531, 2011.","title":"Incremental learning of concept drift in nonstationary environments","authors":"Ryan Elwell, Robi Polikar","year":"2011","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao et al., 2007] Jing Gao, Wei Fan, Jiawei Han, and Philip S. Yu. A general framework for mining concept- drifting data streams with skewed distributions. In Pro- ceedings of the 2007 SIAM International Conference on Data Mining, pages 3-14. SIAM, 2007.","title":"A general framework for mining conceptdrifting data streams with skewed distributions","authors":"Jing Gao, Wei Fan, Jiawei Han, Philip S Yu","year":"2007","venue":"Proceedings of the 2007 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He and Garcia, 2009] Haibo He and Edwardo A Garci- a. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9):1263-1284, 2009.","title":"Learning from imbalanced data","authors":"Haibo He, Edwardo A Garcia","year":"2009","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hoens and Chawla, 2012] Thomas Ryan Hoens and Nitesh V Chawla. Learning in non-stationary environ- ments with class imbalance. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 168-176. ACM, 2012.","title":"Learning in non-stationary environments with class imbalance","authors":"Thomas Ryan, Hoens, Nitesh V Chawla","year":"2012","venue":"Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kolter and Maloof, 2007] J. Zico Kolter and Marcus A. Maloof. Dynamic weighted majority: An ensemble method for drifting concepts. Journal of Machine Learn- ing Research, 8:2755-2790, 2007.","title":"Dynamic weighted majority: An ensemble method for drifting concepts","authors":"Maloof Kolter, ] J Zico Kolter, Marcus A Maloof","year":"2007","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krawczyk et al., 2017] Bartosz Krawczyk, Leandro L Minku, João Gama, Jerzy Stefanowski, and Michał Woźniak. Ensemble learning for data stream analysis: A survey. Information Fusion, 37:132-156, 2017.","title":"Ensemble learning for data stream analysis: A survey","authors":"Bartosz Krawczyk, Leandro L Minku, João Gama, Jerzy Stefanowski, Michał Woźniak","year":"2017","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mirza et al., 2015] Bilal Mirza, Zhiping Lin, and Nan Li- u. Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocom- puting, 149:316-329, 2015.","title":"Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift","authors":"Bilal Mirza, Zhiping Lin, Nan Liu","year":"2015","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Street and Kim, 2001] W Nick Street and YongSeog Kim. A streaming ensemble algorithm (sea) for large-scale clas- sification. In Proceedings of the 7th ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, pages 377-382. ACM, 2001.","title":"A streaming ensemble algorithm (sea) for large-scale classification","authors":"Kim ; Street, Nick Street, Yongseog Kim","year":"2001","venue":"Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sun et al., 2016] Yu Sun, Ke Tang, Leandro L Minku, Shuo Wang, and Xin Yao. Online ensemble learning of data streams with gradually evolved classes. IEEE Transaction- s on Knowledge and Data Engineering, 28(6):1532-1545, 2016.","title":"Online ensemble learning of data streams with gradually evolved classes","authors":"Yu Sun, Ke Tang, Leandro L Minku, Shuo Wang, Xin Yao","year":"2016","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2013a] Shuo Wang, Leandro L Minku, Davide Ghezzi, Daniele Caltabiano, Peter Tino, and Xin Yao. Concept drift detection for online class imbalance learn- ing. In Proceedings of the 2013 International Joint Con- ference on Neural Networks, pages 1-10. IEEE, 2013.","title":"Concept drift detection for online class imbalance learning","authors":"Wang","year":"2013","venue":"Proceedings of the 2013 International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2013b] Shuo Wang, Leandro L Minku, and Xin Yao. Online class imbalance learning and its applications in fault detection. International Journal of Computation- al Intelligence and Applications, 12(04):1 340 001(1-19), 2013.","title":"Online class imbalance learning and its applications in fault detection","authors":"Wang","year":"2013","venue":"International Journal of Computational Intelligence and Applications","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2015] Shuo Wang, Leandro L Minku, and Xin Yao. Resampling-based ensemble methods for online class imbalance learning. IEEE Transactions on Knowledge and Data Engineering, 27(5):1356-1368, 2015.","title":"Resampling-based ensemble methods for online class imbalance learning","authors":"Shuo Wang, Leandro L Minku, Xin Yao","year":"2015","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2016] Shuo Wang, Leandro L Minku, and Xin Yao. Dealing with multiple classes in online class imbal- ance learning. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, pages 2118- 2124. IJCAI/AAAI, 2016.","title":"Dealing with multiple classes in online class imbalance learning","authors":"Shuo Wang, Leandro L Minku, Xin Yao","year":"2016","venue":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2014] Ke Wu, Andrea Edwards, Wei Fan, Jing Gao, and Kun Zhang. Classifying imbalanced data stream- s via dynamic feature group weighting with importance sampling. In Proceedings of the 2014 SIAM International Conference on Data Mining, pages 722-730. SIAM, 2014.","title":"Classifying imbalanced data streams via dynamic feature group weighting with importance sampling","authors":"Ke Wu, Andrea Edwards, Wei Fan, Jing Gao, Kun Zhang","year":"2014","venue":"Proceedings of the 2014 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}