{"type":"ScholarlyArticle","title":"A Survey of Reinforcement Learning Informed by Natural Language","authors":[{"name":"Jelena Luketina","slug":"jelena-luketina","given":"Jelena","family":"Luketina","affiliation":"University of Oxford"},{"name":"Nantas Nardelli","slug":"nantas-nardelli","given":"Nantas","family":"Nardelli","affiliation":"University of Oxford; Facebook AI Research"},{"name":"Gregory Farquhar","slug":"gregory-farquhar","given":"Gregory","family":"Farquhar","affiliation":"University of Oxford; Facebook AI Research"},{"name":"Jakob Foerster","slug":"jakob-foerster","given":"Jakob","family":"Foerster","affiliation":"Facebook AI Research"},{"name":"Jacob Andreas","slug":"jacob-andreas","given":"Jacob","family":"Andreas","affiliation":"Massachusetts Institute of Technology"},{"name":"Edward Grefenstette","slug":"edward-grefenstette","given":"Edward","family":"Grefenstette","affiliation":"Facebook AI Research; University College London"},{"name":"Shimon Whiteson","slug":"shimon-whiteson","given":"Shimon","family":"Whiteson","affiliation":"University of Oxford"},{"name":"Tim Rocktäschel","slug":"tim-rocktaschel","given":"Tim","family":"Rocktäschel","affiliation":"Facebook AI Research; University College London"}],"abstract":"To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it to the task at hand. Recent advances in representation learning for language make it possible to build models that acquire world knowledge from text corpora and integrate this knowledge into downstream decision making problems. We thus argue that the time is right to investigate a tight integration of natural language understanding into RL in particular. We survey the state of the field, including work on instruction following, text games, and learning from textual domain knowledge. Finally, we call for the development of new environments as well as further investigation into the potential uses of recent Natural Language Processing (NLP) techniques for such tasks.","doi":"10.24963/ijcai.2019/880","url":"https://ijcai.org/proceedings/2019/880","pdf":"https://ijcai.org/proceedings/2019/0880.pdf","markdown":"https://ijcai.org/proceedings/2019/880.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/880","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Survey track","track":"Survey track","number":880,"pages":{"first":6309,"last":6317},"keywords":["Machine Learning: Deep Learning","Machine Learning: Reinforcement Learning","Natural Language Processing: NLP Applications and Tools","Machine Learning: Transfer, Adaptation, Multi-task Learning"],"videos":[],"references":[{"position":1,"raw":"[Agarwal et al., 2019] Rishabh Agarwal, Chen Liang, Dale Schuurmans, and Mohammad Norouzi. Learning to gen- eralize from sparse and underspecified rewards. CoRR, abs/1902.07198, 2019.","title":"Learning to generalize from sparse and underspecified rewards","authors":"Rishabh Agarwal, Chen Liang, Dale Schuurmans, Mohammad Norouzi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anderson et al., 2018] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision- and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In CVPR, 2018.","title":"Visionand-language navigation: Interpreting visually-grounded navigation instructions in real environments","authors":"Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, Anton Van Den, Hengel","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Andreas and Klein, 2015] Jacob Andreas and Dan Klein. Alignment-based compositional semantics for instruction following. ACL, 2015.","title":"Alignment-based compositional semantics for instruction following","authors":"Andreas, Klein, Jacob Andreas, Dan Klein","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Andreas et al., 2016] Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In CVPR, 2016.","title":"Neural module networks","authors":"Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Andreas et al., 2017] Jacob Andreas, Dan Klein, and Sergey Levine. Modular Multitask Reinforcement Learning with Policy Sketches. In ICML, 2017.","title":"Modular Multitask Reinforcement Learning with Policy Sketches","authors":"Jacob Andreas, Dan Klein, Sergey Levine","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Andreas et al., 2018] Jacob Andreas, Dan Klein, and Sergey Levine. Learning with latent language. In NAACL-HLT, 2018.","title":"Learning with latent language","authors":"Jacob Andreas, Dan Klein, Sergey Levine","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Antol et al., 2015] Stanislaw Antol, Aishwarya Agrawal, Ji- asen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zit- nick, and Devi Parikh. VQA: Visual question answering. In ICCV, 2015.","title":"VQA: Visual question answering","authors":"Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, Devi Parikh","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Artzi and Zettlemoyer, 2013] Yoav Artzi and Luke Zettle- moyer. Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions. In ACL, 2013.","title":"Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions","authors":"Zettlemoyer Artzi","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Arulkumaran et al., 2017] Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. A Brief Survey of Deep Reinforcement Learning. IEEE Signal Proc. Magazine, 34(6):26-38, 2017.","title":"A Brief Survey of Deep Reinforcement Learning","authors":"Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, Anil Anthony Bharath","year":"2017","venue":"IEEE Signal Proc. Magazine","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Babes et al., 2011] Monica Babes, Vukosi Marivate, Kaushik Subramanian, and Michael L Littman. Appren- ticeship learning about multiple intentions. In ICML, 2011.","title":"Apprenticeship learning about multiple intentions","authors":"Monica Babes, Kaushik Vukosi Marivate, Subramanian, Michael L Littman","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Bahdanau et al., 2019] Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Arian Hosseini, Pushmeet Kohli, and Edward Grefenstette. Learning to Understand Goal Specifications by Modelling Reward. In ICLR, 2019.","title":"Learning to Understand Goal Specifications by Modelling Reward","authors":"Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Arian Hosseini, Pushmeet Kohli, Edward Grefenstette","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Banko et al., 2007] Michele Banko, Michael J Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Etzioni. Open information extraction from the web. In IJCAI, 2007.","title":"Open information extraction from the web","authors":"Michele Banko, J Michael, Stephen Cafarella, Matthew Soderland, Oren Broadhead, Etzioni","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/429"},{"position":13,"raw":"[Barto and Mahadevan, 2003] Andrew G Barto and Sridhar Mahadevan. Recent advances in hierarchical reinforcement learning. Discrete event dynamic systems, 13(1-2):41-77, 2003.","title":"Recent advances in hierarchical reinforcement learning","authors":"G Andrew, Sridhar Barto, Mahadevan","year":"2003","venue":"Discrete event dynamic systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Bisk et al., 2016] Yonatan Bisk, Deniz Yuret, and Daniel Marcu. Natural language communication with robots. In ACL, 2016.","title":"Natural language communication with robots","authors":"Yonatan Bisk, Deniz Yuret, Daniel Marcu","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Bouziane et al., 2015] Abdelghani Bouziane, Djelloul Bouchiha, Noureddine Doumi, and Mimoun Malki. Question answering systems: survey and trends. Procedia Computer Science, 73:366-375, 2015.","title":"Question answering systems: survey and trends","authors":"Bouziane","year":"2015","venue":"Procedia Computer Science","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Branavan et al., 2010] S. R. K. Branavan, Luke S Zettle- moyer, and Regina Barzilay. Reading between the lines: Learning to map high-level instructions to commands. In ACL, 2010.","title":"Reading between the lines: Learning to map high-level instructions to commands","authors":"S R K Branavan, Luke S Zettlemoyer, Regina Barzilay","year":"2010","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Branavan et al., 2012] S. R. K. Branavan, David Silver, and Regina Barzilay. Learning to Win by Reading Manuals in a Monte-Carlo Framework. JAIR, 2012.","title":"Learning to Win by Reading Manuals in a Monte-Carlo Framework","authors":"S R K Branavan, David Silver, Regina Barzilay","year":"2012","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Chaplot et al., 2018] Devendra Singh Chaplot, Kan- thashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, and Ruslan Salakhutdinov. Gated- Attention Architectures for Task-Oriented Language Grounding. In AAAI, 2018.","title":"Gated-Attention Architectures for Task-Oriented Language Grounding","authors":"Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, Ruslan Salakhutdinov","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Chen and Mooney, 2011] David L Chen and Raymond J Mooney. Learning to interpret natural language naviga- tion instructions from observations. In AAAI, 2011.","title":"Learning to interpret natural language navigation instructions from observations","authors":"L David, Raymond J Chen, Mooney","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Chen et al., 2017a] Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open- domain questions. In ACL, 2017.","title":"Reading wikipedia to answer opendomain questions","authors":"Chen","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Chen et al., 2017b] Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. A survey on dialogue systems: Recent advances and new frontiers. ACM SIGKDD Ex- plorations Newsletter, 19(2):25-35, 2017.","title":"A survey on dialogue systems: Recent advances and new frontiers","authors":"Chen","year":"2017","venue":"ACM SIGKDD Explorations Newsletter","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Chen et al., 2018] Howard Chen, Alane Suhr, Dipendra Ku- mar Misra, Noah Snavely, and Yoav Artzi. Touchdown: Natural language navigation and spatial reasoning in visual street environments. CoRR, abs/1811.12354, 2018.","title":"Touchdown: Natural language navigation and spatial reasoning in visual street environments","authors":"Howard Chen, Alane Suhr, Dipendra Kumar Misra, Noah Snavely, Yoav Artzi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Chevalier-Boisvert et al., 2019] Maxime Chevalier- Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning. In ICLR, 2019.","title":"BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning","authors":"Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, Yoshua Bengio","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Côté et al., 2018] Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew J. Hausknecht, Layla El Asri, Mahmoud Adada, Wendy Tay, and Adam Trischler. Textworld: A learning environment for text-based games. CoRR, abs/1806.11532, 2018.","title":"Textworld: A learning environment for text-based games","authors":"Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew J Hausknecht, Layla El Asri, Mahmoud Adada, Wendy Tay, Adam Trischler","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Das et al., 2018a] Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Em- bodied Question Answering. In CVPR, 2018.","title":null,"authors":"Das","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Das et al., 2018b] Abhishek Das, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Neural Modular Con- trol for Embodied Question Answering. CoRL, 2018.","title":"Neural Modular Control for Embodied Question Answering","authors":"Das","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Deerwester et al., 1990] Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harsh- man. Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391-407, 1990.","title":"Indexing by latent semantic analysis","authors":"Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, Richard Harshman","year":"1990","venue":"Journal of the American society for information science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[DePristo and Zubek, 2001] Mark A DePristo and Robert Zubek. being-in-the-world. In AAAI, 2001.","title":"being-in-the-world","authors":"A Mark, Robert Depristo, Zubek","year":"2001","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018.","title":"BERT: pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Dinan et al., 2018] Emily Dinan, Stephen Roller, Kurt Shus- ter, Angela Fan, Michael Auli, and Jason Weston. Wizard of wikipedia: Knowledge-powered conversational agents. CoRR, abs/1811.01241, 2018.","title":"Wizard of wikipedia: Knowledge-powered conversational agents","authors":"Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, Jason Weston","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Eisenstein et al., 2009] Jacob Eisenstein, James Clarke, Dan Goldwasser, and Dan Roth. Reading to learn: constructing features from semantic abstracts. In ACL, 2009.","title":"Reading to learn: constructing features from semantic abstracts","authors":"Jacob Eisenstein, James Clarke, Dan Goldwasser, Dan Roth","year":"2009","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Firth, 1957] John R Firth. A synopsis of linguistic theory. Special Volume of the Philological Society, 1957.","title":"A synopsis of linguistic theory","authors":"John R Firth","year":"1957","venue":"Special Volume of the Philological Society","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Frome et al., 2013] Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc Aurelio Ranzato, and Tomas Mikolov. DeViSE: A Deep Visual-Semantic Embedding Model. In NIPS, 2013.","title":"DeViSE: A Deep Visual-Semantic Embedding Model","authors":"Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc Aurelio Ranzato, Tomas Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Fu et al., 2019] Justin Fu, Anoop Korattikara, Sergey Levine, and Sergio Guadarrama. From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Fol- lowing. In ICLR, 2019.","title":"From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following","authors":"Justin Fu, Anoop Korattikara, Sergey Levine, Sergio Guadarrama","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Fulda et al., 2017] Nancy Fulda, Daniel Ricks, Ben Mur- doch, and David Wingate. What can you do with a rock? affordance extraction via word embeddings. CoRR, abs/1703.03429, 2017.","title":"What can you do with a rock? affordance extraction via word embeddings","authors":"Nancy Fulda, Daniel Ricks, Ben Murdoch, David Wingate","year":"2017","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/144"},{"position":36,"raw":"[Goldberg, 2019] Yoav Goldberg. Assessing BERT's Syntac- tic Abilities. CoRR, abs/1901.05287, 2019.","title":"Assessing BERT's Syntactic Abilities","authors":"Yoav Goldberg","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Gopnik and Meltzoff, 1987] Alison Gopnik and Andrew Meltzoff. The development of categorization in the sec- ond year and its relation to other cognitive and linguistic developments. Child development, pages 1523-1531, 1987.","title":"The development of categorization in the second year and its relation to other cognitive and linguistic developments","authors":"Alison Gopnik, Andrew Meltzoff","year":"1987","venue":"Child development","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Gordon et al., 2018] Daniel Gordon, Aniruddha Kembhavi, Mohammad Rastegari, Joseph Redmon, Dieter Fox, and Ali Farhadi. Iqa: Visual question answering in interactive environments. In CVPR, 2018.","title":"Iqa: Visual question answering in interactive environments","authors":"Daniel Gordon, Aniruddha Kembhavi, Mohammad Rastegari, Joseph Redmon, Dieter Fox, Ali Farhadi","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Goyal et al., 2019] Prasoon Goyal, Scott Niekum, and Raymond J. Mooney. Using Natural Language for Reward Shaping in Reinforcement Learning. CoRR, abs/1903.02020, 2019.","title":"Using Natural Language for Reward Shaping in Reinforcement Learning","authors":"Prasoon Goyal, Scott Niekum, Raymond J Mooney","year":"2019","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2019/331"},{"position":40,"raw":"[Hermann et al., 2017] Karl Moritz Hermann, Felix Hill, Si- mon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jader- berg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis, and Phil Blunsom. Grounded language learning in a simulated 3d world. CoRR, abs/1706.06551, 2017.","title":"Grounded language learning in a simulated 3d world","authors":"Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech, Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis, Phil Blunsom","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Ho and Ermon, 2016] Jonathan Ho and Stefano Ermon. Gen- erative Adversarial Imitation Learning. In NIPS, 2016.","title":"Generative Adversarial Imitation Learning","authors":"Jonathan Ho, Stefano Ermon","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Howard and Ruder, 2018] Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text clas- sification. In ACL, 2018.","title":"Universal language model fine-tuning for text classification","authors":"Ruder Howard, Jeremy Howard, Sebastian Ruder","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Hu et al., 2019] Hengyuan Hu, Denis Yarats, Qucheng Gong, Yuandong Tian, and Mike Lewis. Hierarchical deci- sion making by generating and following natural language instructions. arXiv preprint arXiv:1906.00744, 2019. [Infocom, 1980] Infocom. Zork I, 1980. [Janner et al., 2018]","title":"Hierarchical decision making by generating and following natural language instructions","authors":"Hengyuan Hu, Denis Yarats, Qucheng Gong, Yuandong Tian, Mike Lewis","year":"1980","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"Michael Janner, Karthik Narasimhan, and Regina Barzilay. Representation learning for grounded spatial reasoning. TACL, 2018.","title":"Representation learning for grounded spatial reasoning","authors":"Michael Janner, Karthik Narasimhan, Regina Barzilay","year":"2018","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Johnson et al., 2016] Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. The malmo platform for artificial intelligence experimentation. In IJCAI, 2016.","title":"The malmo platform for artificial intelligence experimentation","authors":"Matthew Johnson, Katja Hofmann, Tim Hutton, David Bignell","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/643"},{"position":46,"raw":"[Johnson et al., 2017] Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for com- positional language and elementary visual reasoning. In CVPR, 2017.","title":"Clevr: A diagnostic dataset for compositional language and elementary visual reasoning","authors":"Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, Ross Girshick","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Kollar et al., 2010] Thomas Kollar, Stefanie Tellex, Deb Roy, and Nicholas Roy. Toward understanding natural language directions. In HRI, 2010.","title":"Toward understanding natural language directions","authors":"Thomas Kollar, Stefanie Tellex, Deb Roy, Nicholas Roy","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Kostka et al., 2017] B. Kostka, J. Kwiecieli, J. Kowalski, and P. Rychlikowski. Text-based adventures of the golovin AI agent. In 2017 IEEE Conference on Computational Intelligence and Games, pages 181-188, 2017.","title":"Text-based adventures of the golovin AI agent","authors":"B Kostka, J Kwiecieli, J Kowalski, P Rychlikowski","year":"2017","venue":"IEEE Conference on Computational Intelligence and Games","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Kuhlmann et al., 2004] Gregory Kuhlmann, Peter Stone, Raymond Mooney, and Jude Shavlik. Guiding a Rein- forcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer. In AAAI-2004 workshop on supervisory control of learning and adaptive systems, 2004.","title":"Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer","authors":"Gregory Kuhlmann, Peter Stone, Raymond Mooney, Jude Shavlik","year":"2004","venue":"AAAI-2004 workshop on supervisory control of learning and adaptive systems","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[MacGlashan et al., 2015] James MacGlashan, Monica Babes-Vroman, Marie desJardins, Michael L. Littman, Smaranda Muresan, Shawn Squire, Stefanie Tellex, Dilip Arumugam, and Lei Yang. Grounding english commands to reward functions. In Robotics: Science and Systems XI, 2015.","title":"Grounding english commands to reward functions","authors":"James Macglashan, Monica Babes-Vroman, Marie Desjardins, Michael L Littman, Smaranda Muresan, Shawn Squire, Stefanie Tellex, Dilip Arumugam, Lei Yang","year":"2015","venue":"Robotics: Science and Systems XI","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[MacMahon et al., 2006] Matt MacMahon, Brian Stankiewicz, and Benjamin Kuipers. Walk the talk: Connecting language, knowledge, and action in route instructions. In AAAI, 2006.","title":"Walk the talk: Connecting language, knowledge, and action in route instructions","authors":"Matt Macmahon, Brian Stankiewicz, Benjamin Kuipers","year":"2006","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Massiceti et al., 2018] Daniela Massiceti, N Siddharth, Puneet K Dokania, and Philip HS Torr. Flipdial: A genera- tive model for two-way visual dialogue. In CVPR, 2018.","title":"Flipdial: A generative model for two-way visual dialogue","authors":"Daniela Massiceti, Siddharth, Philip Hs Puneet K Dokania, Torr","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Mei et al., 2016] Hongyuan Mei, Mohit Bansal, and Matthew R. Walter. Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences. AAAI, 2016.","title":"Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences","authors":"Hongyuan Mei, Mohit Bansal, Matthew R Walter","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Mey, 1993] J. Mey. Pragmatics: An Introduction. Blackwell, 1993.","title":"Pragmatics: An Introduction","authors":"J Mey","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed Repre- sentations of Words and Phrases and their Compositionality. In NIPS, 2013.","title":"Distributed Representations of Words and Phrases and their Compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Misra et al., 2017] Dipendra Misra, John Langford, and Yoav Artzi. Mapping Instructions and Visual Observa- tions to Actions with Reinforcement Learning. EMNLP, 2017.","title":"Mapping Instructions and Visual Observations to Actions with Reinforcement Learning","authors":"Dipendra Misra, John Langford, Yoav Artzi","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Narasimhan et al., 2015] Karthik Narasimhan, Tejas D. Kulkarni, and Regina Barzilay. Language understanding for text-based games using deep reinforcement learning. In EMNLP, 2015.","title":"Language understanding for text-based games using deep reinforcement learning","authors":"Karthik Narasimhan, Tejas D Kulkarni, Regina Barzilay","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Narasimhan et al., 2018] Karthik Narasimhan, Regina Barzi- lay, and Tommi Jaakkola. Grounding Language for Transfer in Deep Reinforcement Learning. JAIR, 2018.","title":"Grounding Language for Transfer in Deep Reinforcement Learning","authors":"Karthik Narasimhan, Regina Barzilay, Tommi Jaakkola","year":"2018","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Oh et al., 2017] Junhyuk Oh, Satinder P. Singh, Honglak Lee, and Pushmeet Kohli. Zero-shot task generalization with multi-task deep reinforcement learning. In ICML, 2017.","title":"Zero-shot task generalization with multi-task deep reinforcement learning","authors":"Junhyuk Oh, P Satinder, Honglak Singh, Pushmeet Lee, Kohli","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Osa et al., 2018] Takayuki Osa, Joni Pajarinen, Gerhard Neumann, J Andrew Bagnell, Pieter Abbeel, Jan Peters, et al. An algorithmic perspective on imitation learning. Foundations and Trends® in Robotics, 7(1-2):1-179, 2018.","title":"An algorithmic perspective on imitation learning","authors":"Takayuki Osa, Joni Pajarinen, Gerhard Neumann, Andrew Bagnell, Pieter Abbeel, Jan Peters","year":"2018","venue":"Foundations and Trends® in Robotics","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Peters et al., 2018a] Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word represen- tations. In NAACL, 2018.","title":"Deep contextualized word representations","authors":"Peters","year":"2018","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Peters et al., 2018b] Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. Dissecting contex- tual word embeddings: Architecture and representation. In EMNLP, 2018.","title":"Dissecting contextual word embeddings: Architecture and representation","authors":"Peters","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Lan- guage models are unsupervised multitask learners. 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":64,"raw":"[Shu et al., 2018] Tianmin Shu, Caiming Xiong, and Richard Socher. Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning. ICLR, 2018.","title":"Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning","authors":"Shu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Shusterman et al., 2011] Anna Shusterman, Sang Ah Lee, and Elizabeth Spelke. Cognitive effects of language on human navigation. Cognition, 120(2):186-201, 2011.","title":"Cognitive effects of language on human navigation","authors":"Anna Shusterman, Sang Ah Lee, Elizabeth Spelke","year":"2011","venue":"Cognition","doi":null,"url":null,"internal_url":null},{"position":66,"raw":"[Silver et al., 2017] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Has- sabis. Mastering the game of Go without human knowledge. Nature, 550(7676):354, 2017.","title":"Mastering the game of Go without human knowledge","authors":"David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George Van Den Driessche, Thore Graepel, Demis Hassabis","year":"2017","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":67,"raw":"[Singh et al., 2002] Satinder Singh, Diane Litman, Michael Kearns, and Marilyn Walker. Optimizing dialogue manage- ment with reinforcement learning: Experiments with the njfun system. JAIR, 2002.","title":"Optimizing dialogue management with reinforcement learning: Experiments with the njfun system","authors":"Satinder Singh, Diane Litman, Michael Kearns, Marilyn Walker","year":"2002","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":68,"raw":"[Socher et al., 2013] Richard Socher, Milind Ganjoo, Christo- pher D. Manning, and Andrew Y. Ng. Zero-shot Learning Through Cross-modal Transfer. In NIPS, 2013.","title":"Zero-shot Learning Through Cross-modal Transfer","authors":"Richard Socher, Milind Ganjoo, Christopher D Manning, Andrew Y Ng","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":69,"raw":"[Spelke and Kinzler, 2007] Elizabeth Spelke and Katherine D Kinzler. Core knowledge. Developmental science, 10(1):89- 96, 2007.","title":"Core knowledge","authors":"Kinzler Spelke, Elizabeth Spelke, Katherine D Kinzler","year":"2007","venue":"Developmental science","doi":null,"url":null,"internal_url":null},{"position":70,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":71,"raw":"[Tellex et al., 2011] Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R Walter, Ashis Gopal Banerjee, Seth Teller, and Nicholas Roy. Understanding natural language commands for robotic navigation and mobile manipulation. In AAAI, 2011.","title":"Understanding natural language commands for robotic navigation and mobile manipulation","authors":"Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R Walter, Ashis Gopal Banerjee, Seth Teller, Nicholas Roy","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":72,"raw":"[Tenney et al., 2019] Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. What do you learn from context? probing for sentence structure in contextualized word representations. In ICLR, 2019.","title":"What do you learn from context? probing for sentence structure in contextualized word representations","authors":"Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, Thomas Mccoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, Ellie Pavlick","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":73,"raw":"[Tesauro, 1995] Gerald Tesauro. Temporal Difference Learn- ing and TD-Gammon. Communications of the ACM, 38(3):58-69, 1995.","title":"Temporal Difference Learning and TD-Gammon","authors":"Gerald Tesauro","year":"1995","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":74,"raw":"[Tian et al., 2017] Yuandong Tian, Qucheng Gong, Wenling Shang, Yuxin Wu, and C. Lawrence Zitnick. ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games. NIPS, 2017.","title":"ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games","authors":"Yuandong Tian, Qucheng Gong, Wenling Shang, Yuxin Wu, C Lawrence Zitnick","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":75,"raw":"[Torrado et al., 2018] Ruben Rodriguez Torrado, Philip Bon- trager, Julian Togelius, Jialin Liu, and Diego Perez-Liebana. Deep reinforcement learning for general video game ai. In 2018 IEEE Conference on Computational Intelligence and Games, pages 1-8, 2018.","title":"Deep reinforcement learning for general video game ai","authors":"Ruben Rodriguez Torrado, Philip Bontrager, Julian Togelius, Jialin Liu, Diego Perez-Liebana","year":"2018","venue":"IEEE Conference on Computational Intelligence and Games","doi":null,"url":null,"internal_url":null},{"position":76,"raw":"[Tsividis et al., 2017] Pedro Tsividis, Thomas Pouncy, Jaque- line L. Xu, Joshua B. Tenenbaum, and Samuel J. Gershman. Human learning in atari. In AAAI, 2017.","title":"Human learning in atari","authors":"Pedro Tsividis, Thomas Pouncy, Jaqueline L Xu, Joshua B Tenenbaum, Samuel J Gershman","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":77,"raw":"[Wang et al., 2016] Sida I Wang, Percy Liang, and Christo- pher D Manning. Learning Language Games through Inter- action. In ACL, 2016.","title":"Learning Language Games through Interaction","authors":"I Sida, Percy Wang, Christopher D Liang, Manning","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":78,"raw":"[Wang et al., 2018] Xin Wang, Qiuyuan Huang, Asli C ¸elikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lei Zhang. Reinforced cross-modal matching and self-supervised imitation learn- ing for vision-language navigation. CoRR, abs/1811.10092, 2018.","title":"Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation","authors":"Xin Wang, Qiuyuan Huang, Jianfeng Asli C ¸elikyilmaz, Dinghan Gao, Yuan-Fang Shen, William Wang, Wang Yang, Lei Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":79,"raw":"[White and Sofge, 1992] David Ashley White and Donald A Sofge. Handbook of Intelligent Control: Neural, Fuzzy, and Adaptative Approaches. Van Nostrand Reinhold Company, 1992.","title":"Handbook of Intelligent Control: Neural, Fuzzy, and Adaptative Approaches","authors":"David, Ashley White, Donald A Sofge","year":"1992","venue":null,"doi":null,"url":null,"internal_url":null},{"position":80,"raw":"[Yan et al., 2018] Claudia Yan, Dipendra Kumar Misra, An- drew Bennett, Aaron Walsman, Yonatan Bisk, and Yoav Artzi. CHALET: cornell house agent learning environment. CoRR, abs/1801.07357, 2018.","title":"CHALET: cornell house agent learning environment","authors":"Claudia Yan, Dipendra Kumar Misra, Andrew Bennett, Aaron Walsman, Yonatan Bisk, Yoav Artzi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":81,"raw":"[Yu et al., 2018] Haonan Yu, Haichao Zhang, and Wei Xu. Interactive Grounded Language Acquisition and General- ization in a 2d World. ICLR, 2018.","title":"Interactive Grounded Language Acquisition and Generalization in a 2d World","authors":"Yu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":82,"raw":"[Yuan et al., 2018] Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew J. Hausknecht, and Adam Trischler. Counting to explore and generalize in text-based games. CoRR, abs/1806.11525, 2018.","title":"Counting to explore and generalize in text-based games","authors":"Yuan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":83,"raw":"[Zellers et al., 2018] Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. SWAG: A large-scale adver- sarial dataset for grounded commonsense inference. In EMNLP, 2018.","title":"SWAG: A large-scale adversarial dataset for grounded commonsense inference","authors":"Rowan Zellers, Yonatan Bisk, Roy Schwartz, Yejin Choi","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":84,"raw":"[Ziebart et al., 2008] Brian D Ziebart, Andrew Maas, J An- drew Bagnell, and Anind K Dey. Maximum Entropy In- verse Reinforcement Learning. In AAAI, 2008.","title":"Maximum Entropy Inverse Reinforcement Learning","authors":"Brian D Ziebart, Andrew Maas, Andrew Bagnell, Anind K Dey","year":"2008","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":85,"raw":"[Zipf, 1949] George Kingsley Zipf. Human behavior and the principle of least effort. 1949.","title":"Human behavior and the principle of least effort","authors":"George Kingsley, Zipf","year":"1949","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}