{"type":"ScholarlyArticle","title":"Learning Hierarchical Symbolic Representations to Support Interactive Task Learning and Knowledge Transfer","authors":[{"name":"James R. Kirk","slug":"james-r-kirk","given":"James R.","family":"Kirk","affiliation":"Computer Science and Engineering , University of Michigan"},{"name":"John E. Laird","slug":"john-e-laird","given":"John E.","family":"Laird","affiliation":"Computer Science and Engineering , University of Michigan"}],"abstract":"Interactive Task Learning (ITL) focuses on learning the definition of tasks through online natural language instruction in real time. Learning the correct grounded meaning of the instructions is difficult due to ambiguous words, lack of common ground, and the presence of distractors in the environment and the agent’s knowledge. We present a learning strategy embodied in an ITL agent that interactively learns in one shot the meaning of task concepts for 40 games and puzzles in ambiguous scenarios. Our approach learns hierarchical symbolic representations of task knowledge rather than learning a mapping directly from perceptual representations. These representations enable the agent to transfer and compose knowledge, analyze and debug multiple interpretations, and communicate efficiently with the teacher to resolve ambiguity. We evaluate the efficiency of the learning by examining the number of words required to teach tasks across cases of no transfer, positive transfer, and interference from prior tasks. Our results show that the agent can correctly generalize, disambiguate, and transfer concepts within variations in language descriptions and world representations of the same task, and across variations in different tasks.","doi":"10.24963/ijcai.2019/844","url":"https://ijcai.org/proceedings/2019/844","pdf":"https://ijcai.org/proceedings/2019/0844.pdf","markdown":"https://ijcai.org/proceedings/2019/844.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/844","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Understanding Intelligence and Human-level AI in the New Machine Learning era","track":"Understanding Intelligence and Human-level AI in the New Machine Learning era","number":844,"pages":{"first":6095,"last":6102},"keywords":["Special Track on Understanding Intelligence and Human-level AI in the New Machine Learning era: Learning knowledge representations (Special Track on Human AI and Machine Learning)","Special Track on Understanding Intelligence and Human-level AI in the New Machine Learning era: Integrating Learning and (any form of) Reasoning (Special Track on Human AI and Machine Learning)","Special Track on Understanding Intelligence and Human-level AI in the New Machine Learning era: Knowledge representations for Learning (Special Track on Human AI and Machine Learning)"],"videos":[],"references":[{"position":1,"raw":"[Azaria et al., 2016] Amos Azaria, Jayant Krishnamurthy, and Tom M. Mitchell. Instructable intelligent personal agent. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI-16, pages 2681-2689. AAAI Press, 2016.","title":"Instructable intelligent personal agent","authors":"Amos Azaria, Jayant Krishnamurthy, Tom M Mitchell","year":"2016","venue":"Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI-16","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barbu et al., 2010] Andrei Barbu, Siddharth Narayanaswamy, and Jeffrey Mark Siskind. Learn- ing physically-instantiated game play through visual observation. In 2010 IEEE International Conference on Robotics and Automation, pages 1879-1886. IEEE, 2010.","title":"Learning physically-instantiated game play through visual observation","authors":"Andrei Barbu, Siddharth Narayanaswamy, Jeffrey Mark Siskind","year":"2010","venue":"IEEE International Conference on Robotics and Automation","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhargava et al., 2017] Dev Bhargava, Gabriel Vega, and Blue Sheffer. Grounded learning of color semantics with autoencoders. Online source, 2017.","title":"Grounded learning of color semantics with autoencoders","authors":"Bhargava","year":"2017","venue":"Online source","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cantrell et al., 2012] Rehj Cantrell, J. Benton, Kartik Tala- madupula, Subbarao Kambhampati, Paul Schermerhorn, and Matthias Scheutz. Tell me when and why to do it! run-time planner model updates via natural language in- struction. In 2012 7th ACM/IEEE International Confer- ence on Human-Robot Interaction (HRI), pages 471-478. IEEE, 2012.","title":"Tell me when and why to do it! run-time planner model updates via natural language instruction","authors":"Cantrell","year":"2012","venue":"7th ACM/IEEE International Conference on Human-Robot Interaction (HRI)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chai et al., 2018] Joyce Y. Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, and Guangyue Xu. Lan- guage to action: Towards interactive task learning with physical agents. In IJCAI, pages 2-9, 2018.","title":"Language to action: Towards interactive task learning with physical agents","authors":"Joyce Y Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, Guangyue Xu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/1"},{"position":6,"raw":"[Chauhan and Lopes, 2011] Aneesh Chauhan and Luís Seabra Lopes. Using spoken words to guide open-ended category formation. Cognitive processing, 12(4):341, 2011.","title":"Using spoken words to guide open-ended category formation","authors":"Aneesh Chauhan, Luís Seabra Lopes","year":"2011","venue":"Cognitive processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dindo and Zambuto, 2010] Haris Dindo and Daniele Zam- buto. A probabilistic approach to learning a visually grounded language model through human-robot interac- tion. In 2010 IEEE/RSJ International Conference on In- telligent Robots and Systems, pages 790-796. IEEE, 2010.","title":"A probabilistic approach to learning a visually grounded language model through human-robot interaction","authors":"Haris Dindo, Daniele Zambuto","year":"2010","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goldwasser and Roth, 2014] Dan Goldwasser and Dan Roth. Learning from natural instructions. Machine learn- ing, 94(2):205-232, 2014.","title":"Learning from natural instructions","authors":"Roth ; Goldwasser, Dan Goldwasser, Dan Roth","year":"2014","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hinrichs and Forbus, 2014] Thomas R. Hinrichs and Ken- neth D. Forbus. X goes first: Teaching simple games through multimodal interaction. Advances in Cognitive Systems, 3:31-46, 2014.","title":"X goes first: Teaching simple games through multimodal interaction","authors":"Thomas R Hinrichs, Kenneth D Forbus","year":"2014","venue":"Advances in Cognitive Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kaiser, 2012] Lukasz Kaiser. Learning games from videos guided by descriptive complexity. In AAAI, 2012.","title":"Learning games from videos guided by descriptive complexity","authors":"Lukasz Kaiser","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kirk and Laird, 2016] James R. Kirk and John E. Laird. Learning general and efficient representations of novel games through interactive instruction. Advances in Cog- nitive Systems, 4, 2016.","title":"Learning general and efficient representations of novel games through interactive instruction","authors":"James R Kirk, John E Laird","year":"2016","venue":"Advances in Cognitive Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Laird et al., 2017] John Laird, Kevin Gluck, John Ander- son, Kenneth Forbus, Odest Chadwicke Jenkins, Chris- tian Lebiere, Dario Salvucci, Matthias Scheutz, Andrea Thomaz, Greg Trafton, Robert Wray, Shiwali Mohan, and James Kirk. Interactive task learning. IEEE Intelligent Systems, 32(4):6-21, 2017.","title":"Interactive task learning","authors":"John Laird, Kevin Gluck, John Anderson, Kenneth Forbus, Chadwicke Odest, Christian Jenkins, Dario Lebiere, Matthias Salvucci, Andrea Scheutz, Greg Thomaz, Robert Trafton, Shiwali Wray, James Mohan, Kirk","year":"2017","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Laird, 2012] John E. Laird. The Soar cognitive architecture. 2012. [Love et al., 2008] Nathaniel Love, Timothy Hinrichs, David Haley, Eric Schkufza, and Michael Genesereth. General game playing: Game description language specification. Technical report, 2008.","title":"The Soar cognitive architecture","authors":"John E Laird","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Matuszek et al., 2012] Cynthia Matuszek, Nicholas FitzGerald, Luke Zettlemoyer, Liefeng Bo, and Di- eter Fox. A joint model of language and percep- tion for grounded attribute learning. arXiv preprint arXiv:1206.6423, 2012.","title":"A joint model of language and perception for grounded attribute learning","authors":"Cynthia Matuszek, Nicholas Fitzgerald, Luke Zettlemoyer, Liefeng Bo, Dieter Fox","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mininger and Laird, 2016] Aaron Mininger and John E. Laird. Interactively learning strategies for handling ref- erences to unseen or unknown objects. Advances in Cog- nitive Systems, 5, 2016.","title":"Interactively learning strategies for handling references to unseen or unknown objects","authors":"Laird ; Mininger, Aaron Mininger, John E Laird","year":"2016","venue":"Advances in Cognitive Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mininger and Laird, 2018] Aaron Mininger and John E. Laird. Interactively learning a blend of goal-based and procedural tasks. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.","title":"Interactively learning a blend of goal-based and procedural tasks","authors":"Laird ; Mininger, Aaron Mininger, John E Laird","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mohan et al., 2012] Shiwali Mohan, Aaron H. Mininger, James R. Kirk, and John E. Laird. Acquiring grounded representations of words with situated interactive instruc- tion. Advances in Cognitive Systems, 2:113-130, 2012.","title":"Acquiring grounded representations of words with situated interactive instruction","authors":"Mohan","year":"2012","venue":"Advances in Cognitive Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Newell, 1993] Allen Newell. Reasoning, problem solving, and decision processes: The problem space as a funda- mental category. MIT Press, 1993.","title":"Reasoning, problem solving, and decision processes: The problem space as a fundamental category","authors":"Allen Newell","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Orhan et al., 2013] Güner Orhan, Sertac ¸Olgunsoylu, Erol S ¸ahin, and Sinan Kalkan. Co-learning nouns and adjec- tives. In 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL), pages 1-6. IEEE, 2013.","title":"Co-learning nouns and adjectives","authors":"Güner Orhan, Sertac ¸olgunsoylu, Sinan Erol S ¸ahin, Kalkan","year":"2013","venue":"IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Roy, 2002] Deb K. Roy. Learning visually grounded words and syntax for a scene description task. Computer speech & language, 16(3-4):353-385, 2002.","title":"Learning visually grounded words and syntax for a scene description task","authors":"Deb K Roy","year":"2002","venue":"Computer speech & language","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Scheutz et al., 2018] Matthias Scheutz, Evan Krause, Bradley Oosterveld, Tyler Frasca, and Robert Platt. Recursive spoken instruction-based one-shot object and action learning. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelli- gence, IJCAI-18, pages 5354-5358. International Joint Conferences on Artificial Intelligence Organization, 2018.","title":"Recursive spoken instruction-based one-shot object and action learning","authors":"Matthias Scheutz, Evan Krause, Bradley Oosterveld, Tyler Frasca, Robert Platt","year":"2018","venue":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18","doi":null,"url":null,"internal_url":"/proceedings/2018/752"},{"position":22,"raw":"[Silver et al., 2018] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A general reinforcement learning algo- rithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140-1144, 2018.","title":"A general reinforcement learning algorithm that masters chess, shogi, and go through self-play","authors":"David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Thomason et al., 2015] Jesse Thomason, Shiqi Zhang, Ray- mond Mooney, and Peter Stone. Learning to interpret nat- ural language commands through human-robot dialog. In Proceedings of the 24th International Conference on Arti- ficial Intelligence, pages 1923-1929. AAAI Press, 2015.","title":"Learning to interpret natural language commands through human-robot dialog","authors":"Jesse Thomason, Shiqi Zhang, Raymond Mooney, Peter Stone","year":"2015","venue":"Proceedings of the 24th International Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}