---
title: "Cost-Saving Effect of Crowdsourcing Learning"
authors:
  - "Lu Wang"
  - "Zhi-Hua Zhou"
doi: ""
url: "https://ijcai.org/proceedings/2016/301"
pdf: "https://ijcai.org/proceedings/2016/0301.pdf"
venue: "Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence"
year: 2016
pages: "2111-2116"
track: ""
keywords:
---

# Cost-Saving Effect of Crowdsourcing Learning

Lu Wang, Zhi-Hua Zhou

*Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence*, pages 2111-2116.

## Abstract

Crowdsourcing is widely adopted in many domains as a popular paradigm to outsource work to individuals. In the machine learning community, crowdsourcing is commonly used as a cost-saving way to collect labels for training data. While a lot of effort has been spent on developing methods for inferring labels from a crowd, few work concentrates on the theoretical foundation of crowdsourcing learning. In this paper, we theoretically study the cost-saving effect of crowdsourcing learning, and present an upper bound for the minimally-sufficient number of crowd labels for effective crowdsourcing learning. Our results provide an understanding about how to allocate crowd labels efficiently, and are verified empirically.