Grounding the Meaning of Words through Vision and Interactive Gameplay

Parde, Natalie; Hair, Adam; Papakostas, Michalis; Tsiakas, Konstantinos; Dagioglou, Maria; Karkaletsis, Vangelis; Nielsen, Rodney D.

Grounding the Meaning of Words through Vision and Interactive Gameplay

Natalie Parde, Adam Hair, Michalis Papakostas, Konstantinos Tsiakas, Maria Dagioglou, Vangelis Karkaletsis, Rodney D. Nielsen

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 1895-1900.

Abstract

Currently, there exists a need for simple, easily-accessible methods with which individuals lacking advanced technical training can expand and customize their robot's knowledge. This work presents a means to satisfy that need, by abstracting the task of training robots to learn about the world around them as a vision- and dialogue-based game, I Spy. In our implementation of I Spy, robots gradually learn about objects and the concepts that describe those objects through repeated gameplay. We show that I Spy is an effective approach for teaching robots how to model new concepts using representations comprised of visual attributes. The results from 255 test games show that the system was able to correctly determine which object the human had in mind 67% of the time. Furthermore, a model evaluation showed that the system correctly understood the visual representations of its learned concepts with an average of 65% accuracy. Human accuracy against the same evaluation standard was just 88% on average.

Natalie Parde, Adam Hair, Michalis Papakostas, Konstantinos Tsiakas, Maria Dagioglou, Vangelis Karkaletsis, and Rodney D. Nielsen. 2015. Grounding the Meaning of Words through Vision and Interactive Gameplay. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 1895–1900. https://ijcai.org/proceedings/2015/269

@inproceedings{ijcai2015p269,
  title     = {Grounding the Meaning of Words through Vision and Interactive Gameplay},
  author    = {Parde, Natalie and Hair, Adam and Papakostas, Michalis and Tsiakas, Konstantinos and Dagioglou, Maria and Karkaletsis, Vangelis and Nielsen, Rodney D.},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {1895--1900},
  year      = {2015},
  month     = {7},
  note      = {Main Track},
  url       = {https://ijcai.org/proceedings/2015/269},
}

TY  - CPAPER
TI  - Grounding the Meaning of Words through Vision and Interactive Gameplay
AU  - Parde, Natalie
AU  - Hair, Adam
AU  - Papakostas, Michalis
AU  - Tsiakas, Konstantinos
AU  - Dagioglou, Maria
AU  - Karkaletsis, Vangelis
AU  - Nielsen, Rodney D.
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 1895
EP  - 1900
UR  - https://ijcai.org/proceedings/2015/269
ER  -