{"type":"ScholarlyArticle","title":"Grounding Abstract Spatial Concepts for Language Interaction with Robots","authors":[{"name":"Rohan Paul","slug":"rohan-paul","given":"Rohan","family":"Paul","affiliation":"Massachusetts Institute of Technology , USA"},{"name":"Jacob Arkin","slug":"jacob-arkin","given":"Jacob","family":"Arkin","affiliation":"University of Rochester , USA"},{"name":"Nicholas Roy","slug":"nicholas-roy","given":"Nicholas","family":"Roy","affiliation":"Massachusetts Institute of Technology , USA"},{"name":"Thomas M. Howard","slug":"thomas-m-howard","given":"Thomas M.","family":"Howard","affiliation":"University of Rochester , USA"}],"abstract":"Our goal is to develop models that allow a robot to understand or ``ground\" natural language instructionsin the context of its world model. Contemporary approaches estimate correspondences between an instruction and possible candidate groundings such as objects, regions and goals for a robot's action. However, these approaches are unable to reason about abstract or hierarchical concepts such as rows, columns and groups that are relevant in a manipulation domain. We introduce a probabilistic model that incorporates an expressive space of abstract spatial concepts as well as notions of cardinality and ordinality. Abstract concepts are introduced as explicit hierarchical symbols correlated with concrete groundings. Crucially, the abstract groundings form a Markov boundary over concrete groundings, effectively de-correlating them from the remaining variables in the graph which reduces the complexity of training and inference in the model. Empirical evaluation demonstrates accurate grounding of abstract concepts embedded in complex natural language instructions commanding a robot manipulator. The proposed inference method leads to significant efficiency gains compared to the baseline, with minimal trade-off in accuracy.","doi":"10.24963/ijcai.2017/696","url":"https://ijcai.org/proceedings/2017/696","pdf":"https://ijcai.org/proceedings/2017/0696.pdf","markdown":"https://ijcai.org/proceedings/2017/696.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/696","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Best Sister Conferences","track":"Best Sister Conferences","number":696,"pages":{"first":4929,"last":4933},"keywords":["Artificial Intelligence: natural language processing","Artificial Intelligence: knowledge representation and reasoning","Artificial Intelligence: robotics"],"videos":[],"references":[{"position":1,"raw":"[Boularias et al., 2015] Abdeslam Boularias, Felix Duvallet, Jean Oh, and Anthony Stentz. Grounding spatial relations for outdoor robot navigation. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 1976-1982, 2015.","title":"Grounding spatial relations for outdoor robot navigation","authors":"Abdeslam Boularias, Felix Duvallet, Jean Oh, Anthony Stentz","year":"2015","venue":"IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Howard et al., 2014a] Thomas M Howard, Istvan Chung, Oron Propp, Matthew R Walter, and Nicholas Roy. Effi- cient natural language interfaces for assistive robots. In IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS) Work. on Rehabilitation and Assistive Robotics, 2014.","title":"Efficient natural language interfaces for assistive robots","authors":"Howard","year":"2014","venue":"IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS) Work. on Rehabilitation and Assistive Robotics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Howard et al., 2014b] Thomas M Howard, Stefanie Tellex, and Nicholas Roy. A natural language planner interface for mobile manipulators. In 2014 IEEE International Con- ference on Robotics and Automation (ICRA), pages 6652- 6659. IEEE, 2014.","title":"A natural language planner interface for mobile manipulators","authors":"Howard","year":"2014","venue":"IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Marcus et al., 1993] Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. Building a large annotated corpus of english: The penn treebank. Computational linguistics, 19(2):313-330, 1993.","title":"Building a large annotated corpus of english: The penn treebank","authors":"Mary Mitchell P Marcus, Ann Marcinkiewicz, Beatrice Santorini","year":"1993","venue":"Computational linguistics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Matuszek et al., 2012] Cynthia Matuszek, Evan Herbst, Luke Zettlemoyer, and Dieter Fox. Learning to parse natu- ral language commands to a robot control system. In Proc- ceedings of the 13th International Symposium on Experi- mental Robotics (ISER), June 2012.","title":"Learning to parse natural language commands to a robot control system","authors":"Cynthia Matuszek, Evan Herbst, Luke Zettlemoyer, Dieter Fox","year":"2012","venue":"Procceedings of the 13th International Symposium on Experimental Robotics (ISER)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Oh et al., 2015] Jean Oh, Arne Suppé, Felix Duvallet, Ab- deslam Boularias, Jerry Vinokurov, Oscar Romero, Chris- tian Lebiere, and Robert Dean. Toward mobile robots reasoning like humans. In Proceedings of the National Conference on Artificial Intelligence (AAAI), pages 1371- 1379, 2015.","title":"Toward mobile robots reasoning like humans","authors":"Jean Oh, Arne Suppé, Felix Duvallet, Abdeslam Boularias, Jerry Vinokurov, Oscar Romero, Christian Lebiere, Robert Dean","year":"2015","venue":"Proceedings of the National Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Paul et al., 2016] Rohan Paul, Jacob Arkin, Nicholas Roy, and Thomas M. Howard. Efficient grounding of abstract spatial concepts for natural language interaction with robot manipulators. In Proceedings of Robotics: Science and Systems, AnnArbor, Michigan, June 2016.","title":"Efficient grounding of abstract spatial concepts for natural language interaction with robot manipulators","authors":"Rohan Paul, Jacob Arkin, Nicholas Roy, Thomas M Howard","year":"2016","venue":"Proceedings of Robotics: Science and Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Tellex et al., 2011] Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R. Walter, Ashis Gopal Banerjee, Seth J Teller, and Nicholas Roy. Understanding natu- ral language commands for robotic navigation and mobile manipulation. In Proceedings of the National Conference on Artificial Intelligence (AAAI), 2011.","title":"Understanding natural language commands for robotic navigation and mobile manipulation","authors":"Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R Walter, Ashis Gopal Banerjee, Seth J Teller, Nicholas Roy","year":"2011","venue":"Proceedings of the National Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Younger, 1967] Daniel H Younger. Recognition and pars- ing of context-free languages in time n 3 . Information and Control, 10(2):189-208, 1967.","title":"Recognition and parsing of context-free languages in time n 3","authors":"H Daniel, Younger","year":"1967","venue":"Information and Control","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}