Unsupervised Rank Aggregation with Domain-Specific Expertise

Klementiev, Alexandre; Roth, Dan; Small, Kevin; Titov, Ivan

Unsupervised Rank Aggregation with Domain-Specific Expertise

Alexandre Klementiev, Dan Roth, Kevin Small, Ivan Titov

Download BibTeX Cite

Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence. Pages 1101-1105.

Abstract

Consider the setting where a panel of judges is repeatedly asked to (partially) rank sets of objects according to given criteria, and assume that the judges' expertise depends on the objects' domain. Learning to aggregate their rankings with the goal of producing a better joint ranking is a fundamental problem in many areas of Information Retrieval and Natural Language Processing, amongst others. However, supervised ranking data is generally difficult to obtain, especially if coming from multiple domains. Therefore, we propose a framework for learning to aggregate votes of constituent rankers with domain specific expertise without supervision. We apply the learning framework to the settings of aggregating full rankings and aggregating top-k lists, demonstrating significant improvements over a domain-agnostic baseline in both cases.Alexandre Klementiev, Dan Roth, Kevin Small, Ivan Titov

Alexandre Klementiev, Dan Roth, Kevin Small, and Ivan Titov. 2009. Unsupervised Rank Aggregation with Domain-Specific Expertise. In Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, IJCAI-09. IJCAI, 1101–1105. https://ijcai.org/proceedings/2009/186

@inproceedings{ijcai2009p186,
  title     = {Unsupervised Rank Aggregation with Domain-Specific Expertise},
  author    = {Klementiev, Alexandre and Roth, Dan and Small, Kevin and Titov, Ivan},
  booktitle = {Proceedings of the Twenty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-09}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Hiroaki Kitano},
  pages     = {1101--1105},
  year      = {2009},
  url       = {https://ijcai.org/proceedings/2009/186},
}

TY  - CPAPER
TI  - Unsupervised Rank Aggregation with Domain-Specific Expertise
AU  - Klementiev, Alexandre
AU  - Roth, Dan
AU  - Small, Kevin
AU  - Titov, Ivan
T2  - Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, IJCAI-09
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2009
SP  - 1101
EP  - 1105
UR  - https://ijcai.org/proceedings/2009/186
ER  -