{"type":"ScholarlyArticle","title":"Robust High-Dimensional Classification From Few Positive Examples","authors":[{"name":"Deepayan Chakrabarti","slug":"deepayan-chakrabarti","given":"Deepayan","family":"Chakrabarti","affiliation":"University of Texas , Austin"},{"name":"Benjamin Fauber","slug":"benjamin-fauber","given":"Benjamin","family":"Fauber","affiliation":"Dell Inc."}],"abstract":"We tackle an extreme form of imbalanced classification, with up to 105 features but as few as 5 samples from the minority class. This problem occurs in predicting predicting tumor types and fraud detection, among others. Standard imbalanced classification methods are not designed for such severe data scarcity. Sampling-based methods need too many samples due to the high-dimensionality, while cost-based methods must place too high a weight on the limited minority samples. Our proposed method, called DIRECT, bypasses sample generation by training the classifier over a robust smoothed distribution of the minority class. DIRECT is fast, simple, robust, parameter-free, and easy to interpret. We validate DIRECT on several real-world datasets spanning document, image, and medical classification. DIRECT is up to 5x − 7x better than SMOTE-like methods, 30−200% better than ensemble methods, 3x − 7x better than cost-sensitive methods. The greatest gains are for settings with the fewest samples in the minority class, where DIRECT’s robustness is most helpful.","doi":"10.24963/ijcai.2022/271","url":"https://ijcai.org/proceedings/2022/271","pdf":"https://ijcai.org/proceedings/2022/0271.pdf","markdown":"https://ijcai.org/proceedings/2022/271.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/271","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":271,"pages":{"first":1952,"last":1958},"keywords":["Data Mining: Class Imbalance and Unequal Cost","Machine Learning: Classification","Machine Learning: Robustness","Data Mining: Applications"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987057","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985200","length_sec":728,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Cao et al., 2019] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss. In NeuRIPS, 2019.","title":"Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss","authors":"Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, Tengyu Ma","year":"2019","venue":"NeuRIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chakrabarti, 2021] Deepayan Chakrabarti. Robust linear classification from limited training data. Machine Learn- ing, 2021.","title":"Deepayan Chakrabarti. Robust linear classification from limited training data","authors":"Chakrabarti","year":"2021","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chawla et al., 2002] Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16:321-357, June 2002.","title":"SMOTE: Synthetic Minority Over-sampling Technique","authors":"V Nitesh, Kevin W Chawla, Lawrence O Bowyer, W Philip Hall, Kegelmeyer","year":"2002","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2004] Chao Chen, Andy Liao, and Leo Breiman. Using Random Forest to Learn Imbalanced Data. Technical Report 666, UC Berkeley, 2004.","title":"Using Random Forest to Learn Imbalanced Data","authors":"Chao Chen, Andy Liao, Leo Breiman","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chung et al., 2016] Yu-An Chung, Hsuan-Tien Lin, and Shao-Wen Yang. Cost-aware pre-training for multiclass cost-sensitive deep learning. In IJCAI, 2016.","title":"Cost-aware pre-training for multiclass cost-sensitive deep learning","authors":"Yu-An Chung, Hsuan-Tien Lin, Shao-Wen Yang","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/203"},{"position":6,"raw":"[Cover and Thomas, 2006] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley- Interscience, USA, 2006.","title":"Elements of Information Theory","authors":"Thomas M Cover, Joy A Thomas","year":"2006","venue":"Wiley Series in Telecommunications and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cunha et al., 2021] Washington Cunha, Vítor Mangaravite, Christian Gomes, Sérgio Canuto, Elaine Resende, Ce- cilia Nascimento, Felipe Viegas, Celso França, Welling- ton Santos Martins, Jussara M. Almeida, Thierson Rosa, Leonardo Rocha, and Marcos André Gonçalves. On the cost-effectiveness of neural and non-neural approaches and representations for text classification: A comprehen- sive comparative study. Information Processing & Man- agement, 58(3):102481, May 2021.","title":"On the cost-effectiveness of neural and non-neural approaches and representations for text classification: A comprehensive comparative study","authors":"Cunha","year":"2021","venue":"Information Processing & Management","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Czarnecki and Rataj, 2015] W. M. Czarnecki and K. Rataj. Compounds Activity Prediction in Large Imbalanced Datasets with Substructural Relations Fingerprint and EEM. In 2015 IEEE Trustcom/BigDataSE/ISPA, vol- ume 2, pages 192-192, August 2015.","title":"Compounds Activity Prediction in Large Imbalanced Datasets with Substructural Relations Fingerprint and EEM","authors":"W M Czarnecki, K Rataj","year":"2015","venue":"IEEE Trustcom/BigDataSE/ISPA","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Davis and Goadrich, 2006] Jesse Davis and Mark Goad- rich. The relationship between Precision-Recall and ROC curves. In ICML, pages 233-240, 2006.","title":"Jesse Davis and Mark Goadrich. The relationship between Precision-Recall and ROC curves","authors":"Goadrich Davis","year":"2006","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Han et al., 2005] Hui Han, Wen-Yuan Wang, and Bing- Huan Mao. Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. In Interna- tional Conference on Intelligent Computing, pages 878- 887, 2005.","title":"Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning","authors":"Hui Han, Wen-Yuan Wang, Bing-Huan Mao","year":"2005","venue":"International Conference on Intelligent Computing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2008] Haibo He, Yang Bai, E. A. Garcia, and Shutao Li. ADASYN: Adaptive synthetic sampling ap- proach for imbalanced learning. In International Joint Conference on Neural Networks, pages 1322-1328, 2008.","title":"ADASYN: Adaptive synthetic sampling approach for imbalanced learning","authors":"Haibo He, Yang Bai, E A Garcia, Shutao Li","year":"2008","venue":"International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ledoit and Wolf, 2004] Olivier Ledoit and Michael Wolf. A well-conditioned estimator for large-dimensional co- variance matrices. Journal of Multivariate Analysis, 88(2):365-411, February 2004.","title":"A well-conditioned estimator for large-dimensional covariance matrices","authors":"Wolf ; Ledoit, Olivier Ledoit, Michael Wolf","year":"2004","venue":"Journal of Multivariate Analysis","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ledoit and Wolf, 2012] Olivier Ledoit and Michael Wolf. Nonlinear shrinkage estimation of large-dimensional co- variance matrices. The Annals of Statistics, 40(2):1024- 1060, April 2012.","title":"Nonlinear shrinkage estimation of large-dimensional co-variance matrices","authors":"Wolf ; Ledoit, Olivier Ledoit, Michael Wolf","year":"2012","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Marcenko and Pastur, 1967] V. A. Marcenko and L. A. Pas- tur. Distribution of Eigenvalues for Some Sets of Random Matrices. Mathematics of the USSR-Sbornik, 1(4):457- 483, 1967.","title":"Distribution of Eigenvalues for Some Sets of Random Matrices","authors":"V A Marcenko, L A Pastur","year":"1967","venue":"Mathematics of the USSR-Sbornik","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Menardi and Torelli, 2014] Giovanna Menardi and Nicola Torelli. Training and assessing classification rules with imbalanced data. Data Mining and Knowledge Discovery, 28(1):92-122, January 2014.","title":"Training and assessing classification rules with imbalanced data","authors":"Torelli Menardi, Giovanna Menardi, Nicola Torelli","year":"2014","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Meng et al., 2018] Yu Meng, Jiaming Shen, Chao Zhang, and Jiawei Han. Weakly-Supervised Neural Text Classi- fication. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pages 983-992, Torino Italy, October 2018. ACM.","title":"Weakly-Supervised Neural Text Classification","authors":"Yu Meng, Jiaming Shen, Chao Zhang, Jiawei Han","year":"2018","venue":"Proceedings of the 27th ACM International Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mohajerin Esfahani and Kuhn, 2018] Peyman Moha- jerin Esfahani and Daniel Kuhn. Data-driven distri- butionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformula- tions. Mathematical Programming, 171(1-2):115-166, September 2018.","title":"Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations","authors":"Peyman Mohajerin, Esfahani, Daniel Kuhn","year":"2018","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Seiffert et al., 2010] C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse, and A. Napolitano. RUSBoost: A Hybrid Approach to Alleviating Class Imbalance. IEEE Transactions on Sys- tems, Man, and Cybernetics -Part A: Systems and Hu- mans, 40(1):185-197, January 2010.","title":"RUSBoost: A Hybrid Approach to Alleviating Class Imbalance","authors":"C Seiffert, T M Khoshgoftaar, J Van Hulse, A Napolitano","year":"2010","venue":"IEEE Transactions on Systems, Man, and Cybernetics -Part A: Systems and Humans","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Silverman, 1986] B. W. Silverman. Density estimation for statistics and data analysis. Number 26 in Monographs on statistics and applied probability. Chapman & Hall/CRC, London, 1986.","title":"Density estimation for statistics and data analysis","authors":"B W Silverman","year":"1986","venue":"Monographs on statistics and applied probability","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tzelepis et al., 2018] Christos Tzelepis, Vasileios Mezaris, and Ioannis Patras. Linear Maximum Margin Classifier for Learning from Uncertain Data. IEEE PAMI, 40(12):2948- 2962, December 2018.","title":"Linear Maximum Margin Classifier for Learning from Uncertain Data","authors":"Christos Tzelepis, Vasileios Mezaris, Ioannis Patras","year":"2018","venue":"IEEE PAMI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wei et al., 2013] Wei Wei, Jinjiu Li, Longbing Cao, Yuming Ou, and Jiahang Chen. Effective detection of sophisticated online banking fraud on extremely imbalanced data. World Wide Web, 16(4):449-475, 2013.","title":"Effective detection of sophisticated online banking fraud on extremely imbalanced data","authors":"Wei Wei, Jinjiu Li, Longbing Cao, Yuming Ou, Jiahang Chen","year":"2013","venue":"World Wide Web","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2009] Huan Xu, Constantine Caramanis, and Shie Mannor. Robustness and Regularization of Support Vec- tor Machines. Journal of Machine Learning Research, 10:1485-1510, 2009.","title":"Robustness and Regularization of Support Vector Machines","authors":"Huan Xu, Constantine Caramanis, Shie Mannor","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang and Xu, 2020] Yuzhe Yang and Zhi Xu. Rethink- ing the Value of Labels for Improving Class-Imbalanced Learning. In NeurIPS, page 12, 2020.","title":"Rethinking the Value of Labels for Improving Class-Imbalanced Learning","authors":"Yuzhe Yang, Zhi Xu","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2019] Wei Yang, Kuang Lu, Peilin Yang, and Jimmy Lin. Critically Examining the \"Neural Hype\": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. In SIGIR, 2019.","title":"Critically Examining the \"Neural Hype\": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models","authors":"Wei Yang, Kuang Lu, Peilin Yang, Jimmy Lin","year":"2019","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yeang et al., 2001] C. H. Yeang, S. Ramaswamy, P. Tamayo, S. Mukherjee, R. M. Rifkin, M. Angelo, M. Reich, E. Lander, J. Mesirov, and T. Golub. Molecular classification of multiple tumor types. Bioinformatics (Oxford, England), 17 Suppl 1:S316-322, 2001.","title":"Molecular classification of multiple tumor types","authors":"C H Yeang, S Ramaswamy, P Tamayo, S Mukherjee, R M Rifkin, M Angelo, M Reich, E Lander, J Mesirov, T Golub","year":"2001","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}