Proceedings

A Comparative Survey: Benchmarking for Pool-based Active Learning

Xueying Zhan, Huan Liu, Qing Li, Antoni B. Chan

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Survey Track. Pages 4679-4686.

https://doi.org/10.24963/ijcai.2021/634

Keywords

  • Machine learning: General

Abstract

Active learning (AL) is a subfield of machine learning (ML) in which a learning algorithm aims to achieve good accuracy with fewer training samples by interactively querying the oracles to label new data points. Pool-based AL is well-motivated in many ML tasks, where unlabeled data is abundant, but their labels are hard or costly to obtain. Although many pool-based AL methods have been developed, some important questions remain unanswered such as how to: 1) determine the current state-of-the-art technique; 2) evaluate the relative benefit of new methods for various properties of the dataset; 3) understand what specific problems merit greater attention; and 4) measure the progress of the field over time. In this paper, we survey and compare various AL strategies used in both recently proposed and classic highly-cited methods. We propose to benchmark pool-based AL methods with a variety of datasets and quantitative metric, and draw insights from the comparative empirical results.

Xueying Zhan, Huan Liu, Qing Li, and Antoni B. Chan. 2021. A Comparative Survey: Benchmarking for Pool-based Active Learning. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 4679–4686. https://doi.org/10.24963/ijcai.2021/634
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards