Impartial Peer Review

Kurokawa, David; Lev, Omer; Morgenstern, Jamie; Procaccia, Ariel D.

Impartial Peer Review

David Kurokawa, Omer Lev, Jamie Morgenstern, Ariel D. Procaccia

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 582-587.

Abstract

Motivated by a radically new peer review system that the National Science Foundation recently experimented with, we study peer review systems in which proposals are reviewed by PIs who have submitted proposals themselves. An (m, k)-selection mechanism asks each PI to review m proposals, and uses these reviews to select (at most) k proposals. We are interested in impartial mechanisms, which guarantee that the ratings given by a PI to others' proposals do not affect the likelihood of the PI's own proposal being selected. We design an impartial mechanism that selects a k-subset of proposals that is nearly as highly rated as the one selected by the non-impartial (abstract version of) the NSF pilot mechanism, even when the latter mechanism has the unfair advantage of eliciting honest reviews.

David Kurokawa, Omer Lev, Jamie Morgenstern, and Ariel D. Procaccia. 2015. Impartial Peer Review. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 582–587. https://ijcai.org/proceedings/2015/88

@inproceedings{ijcai2015p88,
  title     = {Impartial Peer Review},
  author    = {Kurokawa, David and Lev, Omer and Morgenstern, Jamie and Procaccia, Ariel D.},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {582--587},
  year      = {2015},
  month     = {7},
  note      = {Main Track},
  url       = {https://ijcai.org/proceedings/2015/88},
}

TY  - CPAPER
TI  - Impartial Peer Review
AU  - Kurokawa, David
AU  - Lev, Omer
AU  - Morgenstern, Jamie
AU  - Procaccia, Ariel D.
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 582
EP  - 587
UR  - https://ijcai.org/proceedings/2015/88
ER  -