{"type":"ScholarlyArticle","title":"A Comparative Survey: Benchmarking for Pool-based Active Learning","authors":[{"name":"Xueying Zhan","slug":"xueying-zhan","given":"Xueying","family":"Zhan","affiliation":""},{"name":"Huan Liu","slug":"huan-liu","given":"Huan","family":"Liu","affiliation":""},{"name":"Qing Li","slug":"qing-li","given":"Qing","family":"Li","affiliation":""},{"name":"Antoni B. Chan","slug":"antoni-b-chan","given":"Antoni B.","family":"Chan","affiliation":""}],"abstract":"Active learning (AL) is a subfield of machine learning (ML) in which a learning algorithm aims to achieve good accuracy with fewer training samples by interactively querying the oracles to label new data points. Pool-based AL is well-motivated in many ML tasks, where unlabeled data is abundant, but their labels are hard or costly to obtain. Although many pool-based AL methods have been developed, some important questions remain unanswered such as how to: 1) determine the current state-of-the-art technique; 2) evaluate the relative benefit of new methods for various properties of the dataset; 3) understand what specific problems merit greater attention; and 4) measure the progress of the field over time. In this paper, we survey and compare various AL strategies used in both recently proposed and classic highly-cited methods. We propose to benchmark pool-based AL methods with a variety of datasets and quantitative metric, and draw insights from the comparative empirical results.","doi":"10.24963/ijcai.2021/634","url":"https://ijcai.org/proceedings/2021/634","pdf":"https://ijcai.org/proceedings/2021/0634.pdf","markdown":"https://ijcai.org/proceedings/2021/634.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/634","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Survey Track","track":"Survey Track","number":634,"pages":{"first":4679,"last":4686},"keywords":["Machine learning: General"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}