{"type":"ScholarlyArticle","title":"MineRL: A Large-Scale Dataset of Minecraft Demonstrations","authors":[{"name":"William H. Guss","slug":"william-h-guss","given":"William H.","family":"Guss","affiliation":"Carnegie Mellon University , Pittsburgh , PA 15289 , USA"},{"name":"Brandon Houghton","slug":"brandon-houghton","given":"Brandon","family":"Houghton","affiliation":"Carnegie Mellon University , Pittsburgh , PA 15289 , USA"},{"name":"Nicholay Topin","slug":"nicholay-topin","given":"Nicholay","family":"Topin","affiliation":"Carnegie Mellon University , Pittsburgh , PA 15289 , USA"},{"name":"Phillip Wang","slug":"phillip-wang","given":"Phillip","family":"Wang","affiliation":"Carnegie Mellon University , Pittsburgh , PA 15289 , USA"},{"name":"Cayden Codel","slug":"cayden-codel","given":"Cayden","family":"Codel","affiliation":"Carnegie Mellon University , Pittsburgh , PA 15289 , USA"},{"name":"Manuela Veloso","slug":"manuela-veloso","given":"Manuela","family":"Veloso","affiliation":"Carnegie Mellon University , Pittsburgh , PA 15289 , USA"},{"name":"Ruslan Salakhutdinov","slug":"ruslan-salakhutdinov","given":"Ruslan","family":"Salakhutdinov","affiliation":"Carnegie Mellon University , Pittsburgh , PA 15289 , USA"}],"abstract":"The sample inefficiency of standard deep reinforcement learning methods precludes their application to many real-world problems. Methods which leverage human demonstrations require fewer samples but have been researched less. As demonstrated in the computer vision and natural language processing communities, large-scale datasets have the capacity to facilitate research by serving as an experimental and benchmarking platform for new methods. However, existing datasets compatible with reinforcement learning simulators do not have sufficient scale, structure, and quality to enable the further development and evaluation of methods focused on using human examples. Therefore, we introduce a comprehensive, large-scale, simulator-paired dataset of human demonstrations: MineRL. The dataset consists of over 60 million automatically annotated state-action pairs across a variety of related tasks in Minecraft, a dynamic, 3D, open-world environment. We present a novel data collection scheme which allows for the ongoing introduction of new tasks and the gathering of complete state information suitable for a variety of methods. We demonstrate the hierarchality, diversity, and scale of the MineRL dataset. Further, we show the difficulty of the Minecraft domain along with the potential of MineRL in developing techniques to solve key research challenges within it.","doi":"10.24963/ijcai.2019/339","url":"https://ijcai.org/proceedings/2019/339","pdf":"https://ijcai.org/proceedings/2019/0339.pdf","markdown":"https://ijcai.org/proceedings/2019/339.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/339","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":339,"pages":{"first":2442,"last":2448},"keywords":["Humans and AI: Human Computation and Crowdsourcing","Multidisciplinary Topics and Applications: Computer Games","Machine Learning: Reinforcement Learning"],"videos":[],"references":[{"position":1,"raw":"[Andreas et al., 2017] Jacob Andreas, Dan Klein, and Sergey Levine. Modular multitask reinforcement learning with policy sketches. In Proceedings of the 34th ICML-Volume 70, pages 166-175. JMLR. org, 2017.","title":"Modular multitask reinforcement learning with policy sketches","authors":"Jacob Andreas, Dan Klein, Sergey Levine","year":"2017","venue":"Proceedings of the 34th ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrychowicz et al., 2018] Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pa- chocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al. Learning dexterous in-hand manipulation. arXiv preprint arXiv:1808.00177, 2018.","title":"Learning dexterous in-hand manipulation","authors":"Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob Mcgrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bellemare et al., 2013] Marc G Bellemare, Yavar Naddaf, Joel Ve- ness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. JAIR, 47:253-279, 2013.","title":"The arcade learning environment: An evaluation platform for general agents","authors":"Yavar Marc G Bellemare, Joel Naddaf, Michael Veness, Bowling","year":"2013","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[DeepMind, 2018] DeepMind. Alphastar: Mastering the real-time strategy game starcraft ii, 2018.","title":"DeepMind. Alphastar: Mastering the real-time strategy game starcraft ii","authors":"Deepmind","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical im- age database. 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dhariwal et al., 2017] Prafulla Dhariwal, Christopher Hesse, Oleg Klimov, Alex Nichol, Matthias Plappert, Alec Radford, John Schulman, Szymon Sidor, Yuhuai Wu, and Peter Zhokhov. Ope- nai baselines, 2017.","title":"Openai baselines","authors":"Prafulla Dhariwal, Christopher Hesse, Oleg Klimov, Alex Nichol, Matthias Plappert, Alec Radford, John Schulman, Szymon Sidor, Yuhuai Wu, Peter Zhokhov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Geiger et al., 2013] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. IJRR, 32(11):1231-1237, 2013.","title":"Vision meets robotics: The kitti dataset","authors":"Andreas Geiger, Philip Lenz, Christoph Stiller, Raquel Urtasun","year":"2013","venue":"IJRR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Godfrey et al., 1992] John J Godfrey, Edward C Holliman, and Jane McDaniel. Switchboard: Telephone speech corpus for re- search and development. In Acoustics, Speech, and Signal Pro- cessing, 1992. ICASSP-92., 1992 IEEE International Conference on, volume 1, pages 517-520. IEEE, 1992.","title":"Switchboard: Telephone speech corpus for research and development","authors":"John J Godfrey, Edward C Holliman, Jane Mcdaniel","year":"1992","venue":"Acoustics, Speech, and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hessel et al., 2018] Matteo Hessel, Joseph Modayil, Hado Van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, and David Silver. Rain- bow: Combining improvements in deep reinforcement learning. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.","title":"Rainbow: Combining improvements in deep reinforcement learning","authors":"Matteo Hessel, Joseph Modayil, Hado Van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, David Silver","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Johnson et al., 2016] Matthew Johnson, Katja Hofmann, Tim Hut- ton, and David Bignell. The malmo platform for artificial intelli- gence experimentation. In IJCAI, pages 4246-4247, 2016.","title":"The malmo platform for artificial intelligence experimentation","authors":"Matthew Johnson, Katja Hofmann, Tim Hutton, David Bignell","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/643"},{"position":11,"raw":"[Kurin et al., 2017] Vitaly Kurin, Sebastian Nowozin, Katja Hof- mann, Lucas Beyer, and Bastian Leibe. The atari grand challenge dataset. arXiv preprint arXiv:1705.10998, 2017.","title":"The atari grand challenge dataset","authors":"Sebastian Vitaly Kurin, Katja Nowozin, Lucas Hofmann, Bastian Beyer, Leibe","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Levine et al., 2018] Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen. Learning hand-eye coordina- tion for robotic grasping with deep learning and large-scale data collection. IJRR, 37(4-5):421-436, 2018.","title":"Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection","authors":"Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, Deirdre Quillen","year":"2018","venue":"IJRR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2017] Zeming Lin, Jonas Gehring, Vasil Khalidov, and Gabriel Synnaeve. Stardata: A starcraft ai research dataset. In Thirteenth AIDE Conference, 2017.","title":"Stardata: A starcraft ai research dataset","authors":"Lin","year":"2017","venue":"Thirteenth AIDE Conference","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2017] Jerry Liu, Fisher Yu, and Thomas Funkhouser. In- teractive 3d modeling with a generative adversarial network. In 2017 IC3DV, pages 126-134. IEEE, 2017.","title":"Interactive 3d modeling with a generative adversarial network","authors":"Jerry Liu, Fisher Yu, Thomas Funkhouser","year":"2017","venue":"IC3DV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mahler et al., 2019] Jeffrey Mahler, Matthew Matl, Vishal Satish, Michael Danielczuk, Bill DeRose, Stephen McKinley, and Ken Goldberg. Learning ambidextrous robot grasping policies. Sci- ence Robotics, 4(26):eaau4984, 2019.","title":"Learning ambidextrous robot grasping policies","authors":"Jeffrey Mahler, Matthew Matl, Vishal Satish, Michael Danielczuk, Bill Derose, Stephen Mckinley, Ken Goldberg","year":"2019","venue":"Science Robotics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostro- vski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Mnih","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mnih et al., 2016] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In ICML, pages 1928-1937, 2016.","title":"Asynchronous methods for deep reinforcement learning","authors":"Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Oh et al., 2016] Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, and Honglak Lee. Control of memory, active perception, and action in minecraft. arXiv preprint arXiv:1605.09128, 2016.","title":"Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, and Honglak Lee","authors":"Oh","year":"2016","venue":"Control of memory, active perception, and action in minecraft","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[OpenAI, 2018] OpenAI. Openai five, Sep 2018.","title":"OpenAI. Openai five","authors":"Openai","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ross et al., 2011] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured pre- diction to no-regret online learning. In Proceedings of the 14th ICIAS, pages 627-635, 2011.","title":"A reduction of imitation learning and structured prediction to no-regret online learning","authors":"Stéphane Ross, Geoffrey Gordon, Drew Bagnell","year":"2011","venue":"Proceedings of the 14th ICIAS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Salge et al., 2014] Christoph Salge, Cornelius Glackin, and Daniel Polani. Changing the environment based on empowerment as intrinsic motivation. Entropy, 16(5):2789-2819, 2014.","title":"Changing the environment based on empowerment as intrinsic motivation","authors":"Christoph Salge, Cornelius Glackin, Daniel Polani","year":"2014","venue":"Entropy","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shu et al., 2017] Tianmin Shu, Caiming Xiong, and Richard Socher. Hierarchical and interpretable skill acquisition in multi- task reinforcement learning. arXiv preprint arXiv:1712.07294, 2017.","title":"Hierarchical and interpretable skill acquisition in multitask reinforcement learning","authors":"Shu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Silver et al., 2017] David Silver, Julian Schrittwieser, Karen Si- monyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mas- tering the game of go without human knowledge. Nature, 550(7676):354, 2017.","title":"Mastering the game of go without human knowledge","authors":"David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton","year":"2017","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tessler et al., 2017] Chen Tessler, Shahar Givony, Tom Zahavy, Daniel J Mankowitz, and Shie Mannor. A deep hierarchical ap- proach to lifelong learning in minecraft. In Thirty-First AAAI, 2017.","title":"A deep hierarchical approach to lifelong learning in minecraft","authors":"Chen Tessler, Shahar Givony, Tom Zahavy, Daniel J Mankowitz, Shie Mannor","year":"2017","venue":"Thirty-First AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tobin et al., 2017] Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain ran- domization for transferring deep neural networks from simu- lation to the real world. In Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on, pages 23- 30. IEEE, 2017.","title":"Domain randomization for transferring deep neural networks from simulation to the real world","authors":"Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, Pieter Abbeel","year":"2017","venue":"Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2018] Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, pages 8798-8807, 2018.","title":"High-resolution image synthesis and semantic manipulation with conditional gans","authors":"Ting-Chun, Ming-Yu Wang, Jun-Yan Liu, Andrew Zhu, Jan Tao, Bryan Kautz, Catanzaro","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}