Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning

Cui, Yuchen; Koppol, Pallavi; Admoni, Henny; Niekum, Scott; Simmons, Reid; Steinfeld, Aaron; Fitzgerald, Tesca

doi:10.24963/ijcai.2021/599

Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning

Yuchen Cui, Pallavi Koppol, Henny Admoni, Scott Niekum, Reid Simmons, Aaron Steinfeld, Tesca Fitzgerald

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Survey Track. Pages 4382-4391.

https://doi.org/10.24963/ijcai.2021/599

Keywords

Humans and AI: General

Abstract

Human-in-the-loop Machine Learning (HIL-ML) is a widely adopted paradigm for instilling human knowledge in autonomous agents. Many design choices influence the efficiency and effectiveness of such interactive learning processes, particularly the interaction type through which the human teacher may provide feedback. While different interaction types (demonstrations, preferences, etc.) have been proposed and evaluated in the HIL-ML literature, there has been little discussion of how these compare or how they should be selected to best address a particular learning problem. In this survey, we propose an organizing principle for HIL-ML that provides a way to analyze the effects of interaction types on human performance and training data. We also identify open problems in understanding the effects of interaction types.

Yuchen Cui, Pallavi Koppol, Henny Admoni, Scott Niekum, Reid Simmons, Aaron Steinfeld, and Tesca Fitzgerald. 2021. Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 4382–4391. https://doi.org/10.24963/ijcai.2021/599

@inproceedings{ijcai2021p599,
  title     = {Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning},
  author    = {Cui, Yuchen and Koppol, Pallavi and Admoni, Henny and Niekum, Scott and Simmons, Reid and Steinfeld, Aaron and Fitzgerald, Tesca},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Prof. Zhi-Hua Zhou},
  pages     = {4382--4391},
  year      = {2021},
  month     = {8},
  note      = {Survey Track},
  doi       = {10.24963/ijcai.2021/599},
  url       = {https://doi.org/10.24963/ijcai.2021/599},
}

TY  - CPAPER
TI  - Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning
AU  - Cui, Yuchen
AU  - Koppol, Pallavi
AU  - Admoni, Henny
AU  - Niekum, Scott
AU  - Simmons, Reid
AU  - Steinfeld, Aaron
AU  - Fitzgerald, Tesca
T2  - Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2021
SP  - 4382
EP  - 4391
DO  - 10.24963/ijcai.2021/599
UR  - https://doi.org/10.24963/ijcai.2021/599
ER  -