---
title: "Toward a Robust and Universal Crowd-Labeling Framework"
authors:
  - "Faiza Khan Khattak"
doi: ""
url: "https://ijcai.org/proceedings/2016/577"
pdf: "https://ijcai.org/proceedings/2016/0577.pdf"
venue: "Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence"
year: 2016
pages: "4006-4006"
track: ""
keywords:
---

# Toward a Robust and Universal Crowd-Labeling Framework

Faiza Khan Khattak

*Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence*, pages 4006-4006.

## Abstract

One of the main challenges in crowd-labeling is to control for or determine in advance the proportion of low-quality/malicious labelers. We propose methods that estimate the labeler and data instance related parameters using frequentist and Bayesian approaches. All these approaches are based on expert-labeled instance (ground truth) for a small percentage of data to learn the parameters. We also derive a lower bound on the number of expert-labeled instances needed to get better quality labels.