{"type":"ScholarlyArticle","title":"Recursive Spoken Instruction-Based One-Shot Object and Action Learning","authors":[{"name":"Matthias Scheutz","slug":"matthias-scheutz","given":"Matthias","family":"Scheutz","affiliation":"Human-Robot Interaction Laboratory , Tufts University , Medford , MA 02111 , USA"},{"name":"Evan Krause","slug":"evan-krause","given":"Evan","family":"Krause","affiliation":"Human-Robot Interaction Laboratory , Tufts University , Medford , MA 02111 , USA"},{"name":"Bradley Oosterveld","slug":"bradley-oosterveld","given":"Bradley","family":"Oosterveld","affiliation":"Human-Robot Interaction Laboratory , Tufts University , Medford , MA 02111 , USA"},{"name":"Tyler Frasca","slug":"tyler-frasca","given":"Tyler","family":"Frasca","affiliation":"Human-Robot Interaction Laboratory , Tufts University , Medford , MA 02111 , USA"},{"name":"Robert Platt","slug":"robert-platt","given":"Robert","family":"Platt","affiliation":"Northeastern University , Boston 02115 , USA"}],"abstract":"Learning new knowledge from single instructions and being able to apply it immediately is highly desirable for artificial agents.  We provide the first demonstration of spoken instruction-based one-shot object and action learning in a cognitive robotic architecture and briefly discuss the architectural modifications required to enable such fast learning, demonstrating the new capabilities on a fully autonomous robot.","doi":"10.24963/ijcai.2018/752","url":"https://ijcai.org/proceedings/2018/752","pdf":"https://ijcai.org/proceedings/2018/0752.pdf","markdown":"https://ijcai.org/proceedings/2018/752.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/752","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Sister Conferences Best Papers","track":"Best Sister Conferences","number":752,"pages":{"first":5354,"last":5358},"keywords":["Natural Language Processing: Dialogue","Robotics: Human Robot Interaction","Machine Learning: Online Learning"],"videos":[],"references":[{"position":1,"raw":"Human: Pass the plate. Robot: Sorry, I do not know how to do that. Human: OK, I will teach you how to pass the plate. Robot: OK. Human: Pick up the plate. Robot: Sorry, I do not know how to do that. Human: OK, I will teach you how to pick up the plate. Robot: OK. Human: First, find the plate. Robot: OK. Human: Then grab the plate. Robot: OK. Human: Move the plate up. Robot: OK. Human: That is how you pick up the plate. Robot: OK. Human: Move the plate forward. Robot: OK. Human: Release the plate. Robot: OK. Human: That is how you pass a plate. Robot: OK. References [Aimetti, 2009] Guillaume Aimetti. Modelling early lan- guage acquisition skills: Towards a general statistical learning mechanism. In Proceedings of the 12th Confer- ence of the European Chapter of the Association for Com- putational Linguistics: Student Research Workshop, pages 1-9. Association for Computational Linguistics, 2009.","title":"Modelling early language acquisition skills: Towards a general statistical learning mechanism","authors":"Guillaume Aimetti","year":"2009","venue":"Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cantrell et al., 2011] Rehj Cantrell, Paul Schermerhorn, and Matthias Scheutz. Learning actions from human-robot dialogues. In Proceedings of the 2011 IEEE Symposium on Robot and Human Interactive Communication, July 2011.","title":"Learning actions from human-robot dialogues","authors":"Cantrell","year":"2011","venue":"Proceedings of the 2011 IEEE Symposium on Robot and Human Interactive Communication","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dzifcak et al., 2009] Juraj Dzifcak, Matthias Scheutz, Chitta Baral, and Paul Schermerhorn. What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA '09), Kobe, Japan, May 2009.","title":"What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution","authors":"Juraj Dzifcak, Matthias Scheutz, Chitta Baral, Paul Schermerhorn","year":"2009","venue":"Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA '09)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fei-Fei et al., 2006] L. Fei-Fei, R. Fergus, and P. Perona. One-shot learning of object categories. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 28(4), 2006.","title":"One-shot learning of object categories","authors":"L Fei-Fei, R Fergus, P Perona","year":"2006","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gualtieri et al., 2016] Marcus Gualtieri, Andreas ten Pas, Kate Saenko, and Robert Platt. High precision grasp pose detection in dense clutter. CoRR, abs/1603.01564, 2016.","title":"High precision grasp pose detection in dense clutter","authors":"Gualtieri","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huffman and Laird, 1995] Scott B. Huffman and John E. Laird. Flexibly instructable agents. arXiv preprint cs/9511101, 1995.","title":"Flexibly instructable agents","authors":"B Scott, John E Huffman, Laird","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kirk and Laird, 2014] J. Kirk and J.E. Laird. Interactive task learning for simple games. Advances in Cognitive Systems, (3):11-28, 2014.","title":"Interactive task learning for simple games","authors":"J Kirk, J E Laird","year":"2014","venue":"Advances in Cognitive Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Krause et al., 2014] Evan Krause, Michael Zillich, Thomas Williams, and Matthias Scheutz. Learning to recognize novel objects in one shot through human-robot interactions in natural language dialogues. In Proceedings of Twenty- Eighth AAAI Conference on Artificial Intelligence, 2014.","title":"Learning to recognize novel objects in one shot through human-robot interactions in natural language dialogues","authors":"Evan Krause, Michael Zillich, Thomas Williams, Matthias Scheutz","year":"2014","venue":"Proceedings of Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lake et al., 2012] Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. Concept learning as motor pro- gram induction: A large-scale empirical study. In Cogni- tive Science Conference, 2012.","title":"Concept learning as motor program induction: A large-scale empirical study","authors":"M Brenden, Ruslan Lake, Joshua B Salakhutdinov, Tenenbaum","year":"2012","venue":"Cognitive Science Conference","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lauria et al., 2001] Stanislao Lauria, Guido Bugmann, Theocharis Kyriacou, Johan Bos, and A Klein. Training personal robots using natural language instruction. Intelli- gent Systems, IEEE, 16(5):38-45, 2001.","title":"Training personal robots using natural language instruction","authors":"Stanislao Lauria, Guido Bugmann, Theocharis Kyriacou, Johan Bos, Klein","year":"2001","venue":"Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Meric ¸li et al., 2013] Cetin Meric ¸li, Steven D Klee, Jack Pa- parian, and Manuela Veloso. An interactive approach for situated task teaching through verbal instructions. 2013.","title":"An interactive approach for situated task teaching through verbal instructions","authors":"Cetin Meric ¸li, Jack Steven D Klee, Manuela Paparian, Veloso","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mohan et al., 2012] Shiwali Mohan, Aaron Mininger, James Kirk, and John E Laird. Learning grounded language through situated interactive instruction. In AAAI Fall Symposium Series, pages 30-37, 2012.","title":"Learning grounded language through situated interactive instruction","authors":"Mohan","year":"2012","venue":"AAAI Fall Symposium Series","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Oosterveld et al., 2017] Bradley Oosterveld, Richard Veale, and Matthias Scheutz. A parallelized dynamic program- ming approach to zero resource spoken term discovery. In Proceedings of the 42nd IEEE International Conference on Acoustics, Speech, and Signal Processing, 2017.","title":"A parallelized dynamic programming approach to zero resource spoken term discovery","authors":"Bradley Oosterveld, Richard Veale, Matthias Scheutz","year":"2017","venue":"Proceedings of the 42nd IEEE International Conference on Acoustics, Speech, and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Scheutz et al., 2017] Matthias Scheutz, Evan Krause, Brad Oosterveld, Tyler Frasca, and Robert Platt. Spoken instruction-based one-shot object and action learning in a cognitive robotic architecture. In Proceedings of the 16th International Conference on Autoomous Agents and Mul- tiagent Systems, 2017.","title":"Spoken instruction-based one-shot object and action learning in a cognitive robotic architecture","authors":"Matthias Scheutz, Evan Krause, Brad Oosterveld, Tyler Frasca, Robert Platt","year":"2017","venue":"Proceedings of the 16th International Conference on Autoomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Scheutz, 2014] Matthias Scheutz. \"teach one, teach all\" - the explosive combination of instructible robots connected via cyber systems. In IEEE Cyber, 2014.","title":"teach one, teach all\"the explosive combination of instructible robots connected via cyber systems","authors":"Matthias Scheutz","year":"2014","venue":"IEEE Cyber","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}