{"type":"ScholarlyArticle","title":"Discriminative Sample Generation for Deep Imbalanced Learning","authors":[{"name":"Ting Guo","slug":"ting-guo","given":"Ting","family":"Guo","affiliation":"Faculty of Eng. & Info. Tech. , University of Technology Sydney , Australia"},{"name":"Xingquan Zhu","slug":"xingquan-zhu","given":"Xingquan","family":"Zhu","affiliation":"Dept. of Computer & Electrical Eng. and Computer Science , Florida Atlantic University , USA"},{"name":"Yang Wang","slug":"yang-wang","given":"Yang","family":"Wang","affiliation":"Faculty of Eng. & Info. Tech. , University of Technology Sydney , Australia"},{"name":"Fang Chen","slug":"fang-chen","given":"Fang","family":"Chen","affiliation":"Faculty of Eng. & Info. Tech. , University of Technology Sydney , Australia"}],"abstract":"In this paper, we propose a discriminative variational autoencoder (DVAE) to assist deep learning from data with imbalanced class distributions. DVAE is designed to alleviate the class imbalance by explicitly learning class boundaries between training samples, and uses learned class boundaries to guide the feature learning and sample generation. To learn class boundaries, DVAE learns a latent two-component mixture distributor, conditioned by the class labels, so the latent features can help differentiate minority class vs. majority class samples. In order to balance the training data for deep learning to emphasize on the minority class, we combine DVAE and generative adversarial networks (GAN) to form a unified model, DVAAN, which generates synthetic instances close to the class boundaries as training data to learn latent features and update the model. Experiments and comparisons confirm that DVAAN significantly alleviates the class imbalance and delivers accurate models for deep learning from imbalanced data.","doi":"10.24963/ijcai.2019/334","url":"https://ijcai.org/proceedings/2019/334","pdf":"https://ijcai.org/proceedings/2019/0334.pdf","markdown":"https://ijcai.org/proceedings/2019/334.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/334","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":334,"pages":{"first":2406,"last":2412},"keywords":["Machine Learning: Deep Learning","Machine Learning: Data Mining","Machine Learning: Learning Generative Models"],"videos":[],"references":[{"position":1,"raw":"[Bao et al., 2017] Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. Cvae-gan: Fine-grained im- age generation through asymmetric training. In Prof. of the IEEE Intl. Conf. on Computer Vision (ICCV), Oct 2017.","title":"Cvae-gan: Fine-grained image generation through asymmetric training","authors":"Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, Gang Hua","year":"2017","venue":"Prof. of the IEEE Intl. Conf. on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barua et al., 2012] Sukarna Barua, Md. Monirul Islam, Xin Yao, and Kazuyuki Murase. Mwmote-majority weighted minority oversampling technique for imbalanced data set learning. IEEE Transactions on knowledge and data engi- neering, 26(2):405-425, 2012.","title":"Mwmote-majority weighted minority oversampling technique for imbalanced data set learning","authors":"Barua","year":"2012","venue":"IEEE Transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chawla et al., 2002] Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. Smote: Synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.","title":"Smote: Synthetic minority over-sampling technique","authors":"V Nitesh, Kevin W Chawla, Lawrence O Bowyer, W Philip Hall, Kegelmeyer","year":"2002","venue":"Journal of artificial intelligence research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen and Shyu, 2011] Chao Chen and Mei-Ling Shyu. Clustering-based binary-class classification for imbal- anced data sets. In Proc. of the 12th IEEE Intl. Conf. on Information Reuse and Integration, pages 384-389, 2011.","title":"Clustering-based binary-class classification for imbalanced data sets","authors":"Chen, Shyu ; ] Chao, Chen, Mei-Ling Shyu","year":"2011","venue":"Proc. of the 12th IEEE Intl. Conf. on Information Reuse and Integration","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cortes and Vapnik, 1995] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20(3):273-297, 1995.","title":"Corinna Cortes and Vladimir Vapnik. Support-vector networks","authors":"Vapnik Cortes","year":"1995","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dosovitskiy and Brox, 2016] Alexey Dosovitskiy and Thomas Brox. Generating images with perceptual similar- ity metrics based on deep networks. In Proc. of the 2016 Advances in Neural Information Proc. Systems, pages 658-666, 2016.","title":"Generating images with perceptual similarity metrics based on deep networks","authors":"Brox Dosovitskiy, Alexey Dosovitskiy, Thomas Brox","year":"2016","venue":"Proc. of the 2016 Advances in Neural Information Proc. Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Proc. of the 2014 Advances in Neural Information Proc. Systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Proc. of the 2014 Advances in Neural Information Proc. Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He and Garcia, 2008] Haibo He and Edwardo A. Garcia. Learning from imbalanced data. IEEE Transactions on Knowledge & Data Engineering, 9:1263-1284, 2008.","title":"Learning from imbalanced data","authors":"Haibo He, Edwardo A Garcia","year":"2008","venue":"IEEE Transactions on Knowledge & Data Engineering","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Isola et al., 2017] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with con- ditional adversarial networks. In Proc.of the 2017 Com- puter Vision and Pattern Recognition, pages 1125-1134, 2017.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros","year":"2017","venue":"Proc.of the 2017 Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Proc. of the International Conf. on Learning Representations, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"Proc. of the International Conf. on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krawczyk, 2016] Bartosz Krawczyk. Learning from im- balanced data: open challenges and future directions. Progress in Artificial Intelligence, 5(4):221-232, 2016.","title":"Learning from imbalanced data: open challenges and future directions","authors":"Bartosz Krawczyk","year":"2016","venue":"Progress in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Larsen et al., 2016] Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. Autoencoding beyond pixels using a learned similarity metric. In Proc. of the 33nd International Conf. on Machine Learning, pages 1558-1566, 2016.","title":"Autoencoding beyond pixels using a learned similarity metric","authors":"Larsen","year":"2016","venue":"Proc. of the 33nd International Conf. on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu and Wong, 2003] Huiqing Liu and Limsoon Wong. Data mining tools for biological sequences. Journal of Bio. and Computational Biology, 1(1):139-167, 2003.","title":"Data mining tools for biological sequences","authors":"Wong Liu, Huiqing Liu, Limsoon Wong","year":"2003","venue":"Journal of Bio. and Computational Biology","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mirza and Osindero, 2014] Mehdi Mirza and Simon Osin- dero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.","title":"Conditional generative adversarial nets","authors":"Mehdi Mirza, Simon Osindero","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Odena et al., 2016] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with aux- iliary classifier gans. arXiv preprint arXiv:1610.09585, 2016.","title":"Conditional image synthesis with auxiliary classifier gans","authors":"Odena","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Schilling et al., 2002] Mark F Schilling, Ann E Watkins, and William Watkins. Is human height bimodal. The American Statistician, 56(3), 2002.","title":"Is human height bimodal","authors":"Ann E Mark F Schilling, William Watkins, Watkins","year":"2002","venue":"The American Statistician","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sohn et al., 2015] Kihyuk Sohn, Xinchen Yan, and Honglak Lee. Learning structured output representation using deep conditional generative models. In Proc. of the 2015 Ad- vances in Neural Information Proc. Systems, pages 3483- 3491, 2015.","title":"Learning structured output representation using deep conditional generative models","authors":"Sohn","year":"2015","venue":"Proc. of the 2015 Advances in Neural Information Proc. Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sun et al., 2007] Yanmin Sun, Mohamed S.Kamel, An- drew K.C. Wong, and Yang Wang. Cost-sensitive boosting for classification of imbalanced data. Pattern Recognition, 40(12):3358-3378, 2007.","title":"Cost-sensitive boosting for classification of imbalanced data","authors":"Yanmin Sun, Mohamed S Kamel, K C Andrew, Yang Wong, Wang","year":"2007","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[van den Oord et al., 2016] Aäron van den Oord, Nal Kalch- brenner, Oriol Vinyals, Lasse Espeholt, Alex Graves, and Koray Kavukcuoglu. Conditional image generation with pixelcnn decoders. In Proc. of the 2016 Advances in Neu- ral Information Proc. Systems, pages 4790-4798, 2016.","title":"Conditional image generation with pixelcnn decoders","authors":"Oord Van Den","year":"2016","venue":"Proc. of the 2016 Advances in Neural Information Proc. Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2015] S. Wang, L. L. Minku, and X. Yao. Resampling-based ensemble methods for online class im- balance learning. IEEE Transactions on Knowledge & Data Engineering, 27(5):1356-1368, 2015.","title":"Resampling-based ensemble methods for online class imbalance learning","authors":"S Wang, L L Minku, X Yao","year":"2015","venue":"IEEE Transactions on Knowledge & Data Engineering","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu et al., 2017] Fei Wu, Xiao-Yuan Jing, Shiguang Shan, Wangmeng Zuo, and Jing-Yu Yang. Multiset feature learn- ing for highly imbalanced data classification. In Proc. of the 31st AAAI Conf. on Artificial Intelligence, pages 1583- 1589, 2017.","title":"Multiset feature learning for highly imbalanced data classification","authors":"Fei Wu, Xiao-Yuan Jing, Shiguang Shan, Wangmeng Zuo, Jing-Yu Yang","year":"2017","venue":"Proc. of the 31st AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yan et al., 2015] Yilin Yan, Min Chen, Mei-Ling Shyu, and Shu-Ching Chen. Deep learning for imbalanced multime- dia data classification. In Proc. of the 2015 IEEE Interna- tional Symposium on Multimedia, pages 483-488, 2015.","title":"Deep learning for imbalanced multimedia data classification","authors":"Yilin Yan, Min Chen, Mei-Ling Shyu, Shu-Ching Chen","year":"2015","venue":"Proc. of the 2015 IEEE International Symposium on Multimedia","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yan et al., 2017] Yan Yan, Tianbao Yang, Yi Yang, and Jianhui Chen. A framework of online learning with im- balanced streaming data. In Proc. of the 31st AAAI Conf. on Artificial Intelligence, pages 2817-2823, 2017.","title":"A framework of online learning with imbalanced streaming data","authors":"Yan Yan, Tianbao Yang, Yi Yang, Jianhui Chen","year":"2017","venue":"Proc. of the 31st AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu et al., 2017] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In Proc. of the 31st AAAI Conf. on Artificial Intelligence, pages 2852-2858, 2017.","title":"Seqgan: Sequence generative adversarial nets with policy gradient","authors":"Lantao Yu, Weinan Zhang, Jun Wang, Yong Yu","year":"2017","venue":"Proc. of the 31st AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}