{"type":"ScholarlyArticle","title":"What Can You Do with a Rock? Affordance Extraction via Word Embeddings","authors":[{"name":"Nancy Fulda","slug":"nancy-fulda","given":"Nancy","family":"Fulda","affiliation":"Brigham Young University"},{"name":"Daniel Ricks","slug":"daniel-ricks","given":"Daniel","family":"Ricks","affiliation":"Brigham Young University"},{"name":"Ben Murdoch","slug":"ben-murdoch","given":"Ben","family":"Murdoch","affiliation":"Brigham Young University"},{"name":"David Wingate","slug":"david-wingate","given":"David","family":"Wingate","affiliation":"Brigham Young University"}],"abstract":"Autonomous agents must often detect affordances: the set of behaviors enabled by a situation. Affordance extraction is particularly helpful in domains with large action spaces, allowing the agent to prune its search space by avoiding futile behaviors. This paper presents a method for affordance extraction via word embeddings trained on a tagged Wikipedia corpus. The resulting word vectors are treated as a common knowledge database which can be queried using linear algebra. We apply this method to a reinforcement learning agent in a text-only environment and show that affordance-based action selection improves performance in most cases. Our method increases the computational complexity of each learning step but significantly reduces the total number of steps needed. In addition, the agent's action selections begin to resemble those a human would choose.","doi":"10.24963/ijcai.2017/144","url":"https://ijcai.org/proceedings/2017/144","pdf":"https://ijcai.org/proceedings/2017/0144.pdf","markdown":"https://ijcai.org/proceedings/2017/144.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/144","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Knowledge Representation, Reasoning, and Logic","track":"Main track","number":144,"pages":{"first":1039,"last":1045},"keywords":["Machine Learning: Machine Learning","Knowledge Representation, Reasoning, and Logic: Common-Sense Reasoning","Machine Learning: Reinforcement Learning"],"videos":[],"references":[{"position":1,"raw":"[Arkin, 1998] Ronald C. Arkin. Behavior-Based Robotics. MIT Press, 1998.","title":"Behavior-Based Robotics","authors":"Ronald C Arkin","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bolukbasi et al., 2016a] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, NIPS, pages 4349-4357.","title":"Man is to computer programmer as woman is to homemaker? debiasing word embeddings","authors":"Bolukbasi","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Curran Associates, Inc., 2016.","title":null,"authors":"Curran Associates, Inc","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bolukbasi et al., 2016b] Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam Tauman Kalai. Quantifying and reducing stereotypes in word embeddings. CoRR, abs/1606.06121, 2016.","title":"Quantifying and reducing stereotypes in word embeddings","authors":"Bolukbasi","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brokos et al., 2016] Georgios-Ioannis Brokos, Prodromos Malakasiotis, and Ion Androutsopoulos. Using centroids of word embeddings and word mover's distance for biomedical document retrieval in question answering. CoRR, abs/1608.03905, 2016.","title":"Using centroids of word embeddings and word mover's distance for biomedical document retrieval in question answering","authors":"Georgios-Ioannis Brokos, Prodromos Malakasiotis, Ion Androutsopoulos","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Frome et al., 2013] Andrea Frome, Greg S. Corrado, Jonathon Shlens, Samy Bengio, Jeffrey Dean, and Tomas Mikolov. Devise: A deep visual-semantic embedding model. In In NIPS, 2013.","title":"Devise: A deep visual-semantic embedding model","authors":"Andrea Frome, Greg S Corrado, Jonathon Shlens, Samy Bengio, Jeffrey Dean, Tomas Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gibson, 1977] James J. Gibson. The theory of affordances. In Robert Shaw and John Bransford, editors, Perceiving, Acting, and Knowing. 1977.","title":"The theory of affordances","authors":"James J Gibson","year":"1977","venue":"Perceiving, Acting, and Knowing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Karpathy et al., 2014] Andrej Karpathy, Armand Joulin, and Li Fei-fei. Deep fragment embeddings for bidirectional image sentence mapping. In In arXiv:1406.5679, 2014.","title":"Deep fragment embeddings for bidirectional image sentence mapping","authors":"Andrej Karpathy, Armand Joulin, Li Fei-Fei","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kiros et al., 2015] Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. Skip-thought vectors. CoRR, abs/1506.06726, 2015.","title":"Skip-thought vectors","authors":"Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S Zemel, Antonio Torralba, Raquel Urtasun, Sanja Fidler","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Laird and van Lent, 2001] John E. Laird and Michael van Lent. Human-level AI's killer application: Interactive computer games. AI Magazine, 22(2):15-26, 2001.","title":"Human-level AI's killer application: Interactive computer games","authors":"John E Laird, Michael Van Lent","year":"2001","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Le and Mikolov, 2014] Quoc V. Le and Tomas Mikolov. Dis- tributed representations of sentences and documents. CoRR, abs/1405.4053, 2014.","title":"Distributed representations of sentences and documents","authors":"V Quoc, Tomas Le, Mikolov","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu and Singh, 2004] H. Liu and P. Singh. Conceptnet -a prac- tical commonsense reasoning tool-kit. BT Technology Journal, 22(4):211-226, 2004.","title":"Conceptnet -a practical commonsense reasoning tool-kit","authors":"H Liu, P Singh","year":"2004","venue":"BT Technology Journal","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Matuszek et al., 2006] Cynthia Matuszek, John Cabral, Michael Witbrock, and John Deoliveira. An introduction to the syntax and content of cyc. In Proceedings of the 2006 AAAI Spring Sympo- sium on Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question An- swering, pages 44-49, 2006.","title":"An introduction to the syntax and content of cyc","authors":"Cynthia Matuszek, John Cabral, Michael Witbrock, John Deoliveira","year":"2006","venue":"Proceedings of the 2006 AAAI Spring Symposium on Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, NIPS, pages 3111-3119.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"Curran Associates, Inc., 2013.","title":null,"authors":"Curran Associates, Inc","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mikolov et al., 2013c] Tomas Mikolov, Wen tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word represen- tations. Association for Computational Linguistics, May 2013.","title":"Linguistic regularities in continuous space word representations","authors":"Mikolov","year":"2013","venue":"Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Miller, 1995] George A. Miller. Wordnet: A lexical database for english. Commun. ACM, 38(11):39-41, November 1995.","title":"Wordnet: A lexical database for english","authors":"George A Miller","year":"1995","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostro- vski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Montesano et al., 2007] L. Montesano, M. Lopes, A. Bernardino, and J. Santos-Victor. Modeling affordances using bayesian net- works. In 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4102-4107, Oct 2007.","title":"Modeling affordances using bayesian networks","authors":"L Montesano, M Lopes, A Bernardino, J Santos-Victor","year":"2007","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Narasimhan et al., 2015] Karthik Narasimhan, Tejas D. Kulka- rni, and Regina Barzilay. Language understanding for text- based games using deep reinforcement learning. CoRR, abs/1506.08941, 2015.","title":"Language understanding for textbased games using deep reinforcement learning","authors":"Karthik Narasimhan, Tejas D Kulkarni, Regina Barzilay","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Navarro et al., 2012] Stefan Escaida Navarro, Nicolas Gorges, Heinz Wörn, Julian Schill, Tamim Asfour, and Rüdiger Dill- mann. Haptic object recognition for multi-fingered robot hands. In 2012 IEEE Haptics Symposium (HAPTICS), pages 497-502. IEEE, 2012.","title":"Haptic object recognition for multi-fingered robot hands","authors":"Stefan Escaida Navarro, Nicolas Gorges, Heinz Wörn, Julian Schill, Tamim Asfour, Rüdiger Dillmann","year":"2012","venue":"IEEE Haptics Symposium (HAPTICS)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Russ et al., 2011] Thomas A Russ, Cartic Ramakrishnan, Ed- uard H Hovy, Mihail Bota, and Gully APC Burns. Knowledge engineering tools for reasoning with scientific observations and interpretations: a neural connectivity use case. BMC bioinfor- matics, 12(1):351, 2011.","title":"Knowledge engineering tools for reasoning with scientific observations and interpretations: a neural connectivity use case","authors":"Cartic Thomas A Russ, Eduard H Ramakrishnan, Mihail Hovy, Gully Apc Bota, Burns","year":"2011","venue":"BMC bioinformatics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Schenck et al., 2012] Wolfram Schenck, Hendrik Hasenbein, and Ralf Möller. Detecting affordances by mental imagery. In","title":"Detecting affordances by mental imagery","authors":"Schenck","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"Alessandro G. Di Nuovo, Vivian M. de la Cruz, and Davide Marocco, editors, Proceedings of the SAB Workshop on \"Arti- ficial Mental Imagery\", pages 15-18, Odense (Danmark), 2012.","title":"Proceedings of the SAB Workshop on \"Artificial Mental Imagery","authors":null,"year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schenck et al., 2016] Wolfram Schenck, Hendrik Hasenbein, and Ralf Möller. Detecting affordances by visuomotor simulation. arXiv preprint arXiv:1611.00274, 2016.","title":"Detecting affordances by visuomotor simulation","authors":"Schenck","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Socher et al., 2011] Richard Socher, Cliff C. Lin, Chris Manning, and Andrew Y Ng. Parsing natural scenes and natural language with recursive neural networks. ICML, pages 129 -136, 2011.","title":"Parsing natural scenes and natural language with recursive neural networks","authors":"Richard Socher, Cliff C Lin, Chris Manning, Andrew Y Ng","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Song et al., 2011] Hyun Oh Song, Mario Fritz, Chunhui Gu, and Trevor Darrell. Visual grasp affordances from appearance-based cues. In ICCV Workshops, pages 998-1005. IEEE, 2011.","title":"Visual grasp affordances from appearance-based cues","authors":"Hyun Oh Song, Mario Fritz, Chunhui Gu, Trevor Darrell","year":"2011","venue":"ICCV Workshops","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Song et al., 2015] Hyun Oh Song, Mario Fritz, Daniel Goehring, and Trevor Darrell. Learning to detect visual grasp affordance. In IEEE Transactions on Automation Science and Engineering (TASE), 2015.","title":"Learning to detect visual grasp affordance","authors":"Hyun Oh Song, Mario Fritz, Daniel Goehring, Trevor Darrell","year":"2015","venue":"IEEE Transactions on Automation Science and Engineering","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Stoytchev, 2008] Alexander Stoytchev. Learning the Affordances of Tools Using a Behavior-Grounded Approach, pages 140-158. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008.","title":"Learning the Affordances of Tools Using a Behavior-Grounded Approach","authors":"Alexander Stoytchev","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Watkins and Dayan, 1992] Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8(3-4):279-292, 1992.","title":"Q-learning","authors":"Jch Christopher, Peter Watkins, Dayan","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhu et al., 2014] Yuke Zhu, Alireza Fathi, and Li Fei-Fei. Reason- ing about object affordances in a knowledge base representation. In ECCV, 2014.","title":"Reasoning about object affordances in a knowledge base representation","authors":"Yuke Zhu, Alireza Fathi, Li Fei-Fei","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhu et al., 2015] Yukun Zhu, Ryan Kiros, Richard S. Zemel, Rus- lan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. CoRR, abs/1506.06724, 2015.","title":"Aligning books and movies: Towards story-like visual explanations by watching movies and reading books","authors":"Yukun Zhu, Ryan Kiros, Richard S Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, Sanja Fidler","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}