{"type":"ScholarlyArticle","title":"Class Prior Estimation in Active Positive and Unlabeled Learning","authors":[{"name":"Lorenzo Perini","slug":"lorenzo-perini","given":"Lorenzo","family":"Perini","affiliation":"DTAI Research group , KU Leuven , Belgium"},{"name":"Vincent Vercruyssen","slug":"vincent-vercruyssen","given":"Vincent","family":"Vercruyssen","affiliation":"DTAI Research group , KU Leuven , Belgium"},{"name":"Jesse Davis","slug":"jesse-davis","given":"Jesse","family":"Davis","affiliation":"DTAI Research group , KU Leuven , Belgium"}],"abstract":"Estimating the proportion of positive examples (i.e., the class prior) from positive and unlabeled (PU) data is an important task that facilitates learning a classifier from such data.  In this paper, we explore how to tackle this problem when the observed labels were acquired via active learning. This introduces the challenge that the observed labels were not selected completely at random, which is the primary assumption underpinning existing approaches to estimating the class prior from PU data. We analyze this new setting and design an algorithm that is able to estimate the class prior for a given active learning strategy. Empirically, we show that our approach accurately recovers the true class prior on a benchmark of anomaly detection datasets and that it does so more accurately than existing methods.","doi":"10.24963/ijcai.2020/403","url":"https://ijcai.org/proceedings/2020/403","pdf":"https://ijcai.org/proceedings/2020/0403.pdf","markdown":"https://ijcai.org/proceedings/2020/403.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/403","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":403,"pages":{"first":2915,"last":2921},"keywords":["Data Mining: Classification, Semi-Supervised Learning","Machine Learning: Semi-Supervised Learning","Machine Learning: Active Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24818/file_24818.m3u8","length_sec":291,"provider":"hls"}],"references":[{"position":1,"raw":"[Barnabé-Lortie et al., 2015] Vincent Barnabé-Lortie, Colin Bellinger, and Nathalie Japkowicz. Active learning for one-class classification. In 14th International Conference on Machine Learning and Applications, pages 390-395. IEEE, 2015.","title":"Active learning for one-class classification","authors":"Vincent Barnabé-Lortie, Colin Bellinger, Nathalie Japkowicz","year":"2015","venue":"14th International Conference on Machine Learning and Applications","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bekker and Davis, 2018] Jessa Bekker and Jesse Davis. Es- timating the class prior in positive and unlabeled data through decision tree induction. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, 2018.","title":"Estimating the class prior in positive and unlabeled data through decision tree induction","authors":"Davis ; Bekker, Jessa Bekker, Jesse Davis","year":"2018","venue":"Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bekker and Davis, 2020] Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: A survey. Ma- chine Learning, 2020.","title":"Learning from positive and unlabeled data: A survey","authors":"Davis ; Bekker, Jessa Bekker, Jesse Davis","year":"2020","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bekker et al., 2019] Jessa Bekker, Pieter Robberechts, and Jesse Davis. Beyond the selected completely at random as- sumption for learning from positive and unlabeled data. In Proceedings of 2018 Joint European Conference on Ma- chine Learning and Knowledge Discovery in Databases, 2019.","title":"Beyond the selected completely at random assumption for learning from positive and unlabeled data","authors":"Jessa Bekker, Pieter Robberechts, Jesse Davis","year":"2019","venue":"Proceedings of 2018 Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Campos et al., 2016] Guilherme O Campos, Arthur Zimek, Jörg Sander, Ricardo JGB Campello, Barbora Micenková, Erich Schubert, Ira Assent, and Michael E Houle. On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study. Data Mining and Knowl- edge Discovery, 30(4):891-927, 2016.","title":"On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study","authors":"Arthur Guilherme O Campos, Jörg Zimek, Ricardo Jgb Sander, Barbora Campello, Erich Micenková, Ira Schubert, Michael E Assent, Houle","year":"2016","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Du Plessis and Sugiyama, 2014] Marthinus Christoffel Du Plessis and Masashi Sugiyama. Class prior estimation from positive and unlabeled data. IEICE Transactions on Information and Systems, 97(5):1358-1362, 2014.","title":"Class prior estimation from positive and unlabeled data","authors":"Du Plessis, Marthinus Sugiyama, Du Christoffel, Masashi Plessis, Sugiyama","year":"2014","venue":"IEICE Transactions on Information and Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[du Plessis et al., 2015] M.C. du Plessis, G. Niu, and M. Sugiyama. Class-prior estimation for learning from positive and unlabeled data. Proceedings of the 7th Asian Conference on Machine Learning, pages 221-236, 2015.","title":"Class-prior estimation for learning from positive and unlabeled data","authors":"M C Du Plessis, G Niu, M Sugiyama","year":"2015","venue":"Proceedings of the 7th Asian Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Elkan and Noto, 2008] Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 213-220, 2008.","title":"Learning classifiers from only positive and unlabeled data","authors":"Noto Elkan, Charles Elkan, Keith Noto","year":"2008","venue":"Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ghasemi et al., 2016] Alireza Ghasemi, Hamid R. Rabiee, Mohsen Fadaee, Mohammad Taghi Manzuri, and Moham- mad H. Rohban. Active learning from positive and unla- beled data. CoRR, abs/1602.07495, 2016.","title":"Active learning from positive and unlabeled data","authors":"Alireza Ghasemi, Hamid R Rabiee, Mohsen Fadaee, Mohammad Taghi Manzuri, Mohammad H Rohban","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2015] Guoliang He, Yong Duan, Yifei Li, Tieyun Qian, Jinrong He, and Xiangyang Jia. Active learning for multivariate time series classification with positive unla- beled data. In Proceedings of the 27th International Con- ference on Tools with Artificial Intelligence, pages 178- 185, 2015.","title":"Active learning for multivariate time series classification with positive unlabeled data","authors":"Guoliang He, Yong Duan, Yifei Li, Tieyun Qian, Jinrong He, Xiangyang Jia","year":"2015","venue":"Proceedings of the 27th International Conference on Tools with Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jain et al., 2016a] Shantanu Jain, Martha White, and Pre- drag Radivojac. Estimating the class prior and posterior from noisy positives and unlabeled data. In Advances in Neural Information Processing Systems, pages 2693- 2701, 2016.","title":"and Predrag Radivojac. Estimating the class prior and posterior from noisy positives and unlabeled data","authors":"Jain","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jain et al., 2016b] Shantanu Jain, Martha White, Michael W Trosset, and Predrag Radivojac. Nonparametric semi- supervised learning of class proportions. arXiv preprint arXiv:1601.01944, 2016.","title":"Nonparametric semisupervised learning of class proportions","authors":"Jain","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kriegel et al., 2011] Hans-Peter Kriegel, Peer Kroger, Erich Schubert, and Arthur Zimek. Interpreting and unifying outlier scores. In Proceedings of the SIAM International Conference on Data Mining, pages 13-24, 2011.","title":"Interpreting and unifying outlier scores","authors":"Kriegel","year":"2011","venue":"Proceedings of the SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2008] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In Eighth IEEE International Con- ference on Data Mining, pages 413-422. IEEE, 2008.","title":"Isolation forest","authors":"Tony Fei, Kai Liu, Ming Ting, Zhi-Hua Zhou","year":"2008","venue":"Eighth IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ramaswamy et al., 2016] Harish Ramaswamy, Clayton Scott, and Ambuj Tewari. Mixture proportion estimation via kernel embeddings of distributions. In International Conference on Machine Learning, pages 2052-2060, 2016.","title":"Mixture proportion estimation via kernel embeddings of distributions","authors":"Harish Ramaswamy, Clayton Scott, Ambuj Tewari","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Settles, 2012] Burr Settles. Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 6(1):1-114, 2012.","title":"Burr Settles. Active learning","authors":"Settles","year":"2012","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Trittenbach et al., 2018] Holger Trittenbach, Adrian Engl- hardt, and Klemens Böhm. An overview and a benchmark of active learning for outlier detection with one-class clas- sifiers. arXiv preprint arXiv:1808.04759, 2018.","title":"An overview and a benchmark of active learning for outlier detection with one-class classifiers","authors":"Trittenbach","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Trittenbach et al., 2019] Holger Trittenbach, Adrian Engl- hardt, and Klemens Böhm. Validating one-class active learning with user studies-a prototype and open chal- lenges. In ECML PKDD Workshop, page 17, 2019.","title":"Validating one-class active learning with user studies-a prototype and open challenges","authors":"Trittenbach","year":"2019","venue":"ECML PKDD Workshop","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vercruyssen et al., 2018] Vincent Vercruyssen, Meert Wannes, Verbruggen Gust, Maes Koen, Bäumer Ruben, and Davis Jesse. Semi-supervised anomaly detection with an application to water analytics. In Proceedings of the IEEE International Conference on Data Mining, 2018.","title":"Semi-supervised anomaly detection with an application to water analytics","authors":"Vincent Vercruyssen, Meert Wannes, Verbruggen Gust, Bäumer Maes Koen, Davis Ruben, Jesse","year":"2018","venue":"Proceedings of the IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}