{"type":"ScholarlyArticle","title":"Game Engine Learning from Video","authors":[{"name":"Matthew Guzdial","slug":"matthew-guzdial","given":"Matthew","family":"Guzdial","affiliation":"School of Interactive Computing , Georgia Institute of Technology"},{"name":"Boyang Li","slug":"boyang-li","given":"Boyang","family":"Li","affiliation":"School of Interactive Computing , Georgia Institute of Technology"},{"name":"Mark O. Riedl","slug":"mark-o-riedl","given":"Mark O.","family":"Riedl","affiliation":"School of Interactive Computing , Georgia Institute of Technology"}],"abstract":"Intelligent agents need to be able to make predictions about their environment. In this work we present a novel approach to learn a forward simulation model via simple search over pixel input. We make use of a video game, Super Mario Bros., as an initial test of our approach as it represents a physics system that is significantly less complex than reality. We demonstrate the significant improvement of our approach in predicting future states compared with a baseline CNN and apply the learned model to train a game playing agent. Thus we evaluate the algorithm in terms of the accuracy and value of its output model.","doi":"10.24963/ijcai.2017/518","url":"https://ijcai.org/proceedings/2017/518","pdf":"https://ijcai.org/proceedings/2017/0518.pdf","markdown":"https://ijcai.org/proceedings/2017/518.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/518","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":518,"pages":{"first":3707,"last":3713},"keywords":["Multidisciplinary Topics and Applications: Computer Games","Multidisciplinary Topics and Applications: Interactive Entertainment"],"videos":[],"references":[{"position":1,"raw":"[Braylan and Miikkulainen, 2016] Alexander Eric Braylan and Risto Miikkulainen. Object-model transfer in the gen- eral video game domain. In The 12th Artificial Intelligence and Interactive Digital Entertainment Conference, 2016.","title":"Object-model transfer in the general video game domain","authors":"Alexander Eric, Braylan, Risto Miikkulainen","year":"2016","venue":"The 12th Artificial Intelligence and Interactive Digital Entertainment Conference","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brooks, 1999] Rodney Allen Brooks. Cambrian intelli- gence: The early history of the new AI, volume 44. Mit Press Cambridge, MA, 1999.","title":"Cambrian intelligence: The early history of the new AI","authors":"Rodney Allen, Brooks","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Canossa and Smith, 2015] Alessandro Canossa and Gillian Smith. Towards a procedural evaluation technique: Met- rics for level design. The 10th International Conference on the Foundations of Digital Games, page 8, 2015.","title":"Towards a procedural evaluation technique: Metrics for level design","authors":"Alessandro Canossa, Gillian Smith","year":"2015","venue":"The 10th International Conference on the Foundations of Digital Games","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Churchland and Sejnowski, 2016] Patricia S Churchland and Terrence J Sejnowski. The computational brain. MIT press, 2016.","title":"The computational brain","authors":"S Patricia, Terrence J Churchland, Sejnowski","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cook et al., 2013] Michael Cook, Simon Colton, Azalea Raad, and Jeremy Gow. Mechanic miner: Reflection- driven game mechanic discovery and level design. In Eu- ropean Conference on the Applications of Evolutionary Computation, pages 284-293. Springer, 2013.","title":"Mechanic miner: Reflectiondriven game mechanic discovery and level design","authors":"Michael Cook, Simon Colton, Azalea Raad, Jeremy Gow","year":"2013","venue":"European Conference on the Applications of Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ersen and Sariel, 2015] Mustafa Ersen and Sanem Sariel. Learning behaviors of and interactions among objects through spatio-temporal reasoning. IEEE Transactions on Computational Intelligence and AI in Games, 7(1):75-87, 2015.","title":"Learning behaviors of and interactions among objects through spatio-temporal reasoning","authors":"Sariel Ersen, Mustafa Ersen, Sanem Sariel","year":"2015","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gettys and Fisher, 1979] Charles F Gettys and Stanley D Fisher. Hypothesis plausibility and hypothesis genera- tion. Organizational behavior and human performance, 24(1):93-110, 1979.","title":"Hypothesis plausibility and hypothesis generation","authors":"F Charles, Stanley D Gettys, Fisher","year":"1979","venue":"Organizational behavior and human performance","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gow and Corneli, 2015] Jeremy Gow and Joseph Corneli. Towards generating novel games using conceptual blend- ing. In 11th Artificial Intelligence and Interactive Digital Entertainment Conference, 2015.","title":"Towards generating novel games using conceptual blending","authors":"Corneli Gow, Jeremy Gow, Joseph Corneli","year":"2015","venue":"11th Artificial Intelligence and Interactive Digital Entertainment Conference","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guzdial et al., 2016] Matthew Guzdial, Nathan Sturtevant, and Boyang Li. Deep static and dynamic level analysis: A study on infinite mario. In 3rd Experimental AI in Games Workshop, 2016.","title":"Deep static and dynamic level analysis: A study on infinite mario","authors":"Matthew Guzdial, Nathan Sturtevant, Boyang Li","year":"2016","venue":"3rd Experimental AI in Games Workshop","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hafner, 2016] Danijar Hafner. Deep reinforcement learn- ing from raw pixels in doom. arXiv preprint arXiv:1610.02164, 2016.","title":"Deep reinforcement learning from raw pixels in doom","authors":"Danijar Hafner, Hafner","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kober et al., 2013] Jens Kober, J Andrew Bagnell, and Jan Peters. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11):1238- 1274, 2013.","title":"Reinforcement learning in robotics: A survey","authors":"Kober","year":"2013","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Martens et al., 2016] Chris Martens, Adam Summerville, Michael Mateas, Joseph Osborn, Sarah Harmon, Noah Wardrip-Fruin, and Arnav Jhala. Proceduralist readings, procedurally. In 12th Artificial Intelligence and Interac- tive Digital Entertainment Conference, 2016.","title":"Proceduralist readings, procedurally","authors":"Chris Martens, Adam Summerville, Michael Mateas, Joseph Osborn, Sarah Harmon, Noah Wardrip-Fruin, Arnav Jhala","year":"2016","venue":"12th Artificial Intelligence and Interactive Digital Entertainment Conference","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mnih et al., 2013] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and Martin Ried- miller. Playing Atari with deep reinforcement learn- ing. Technical report, Deepmind Technologies, 2013. arXiv:1312.5602.","title":"Playing Atari with deep reinforcement learning","authors":"V Mnih, K Kavukcuoglu, D Silver, A Graves, I Antonoglou, D Wierstra, Martin Riedmiller","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nelson et al., 2016] Mark J Nelson, Julian Togelius, Cameron B Browne, and Michael Cook. Rules and mechanics. In Procedural Content Generation in Games (Computational Synthesis and Creative Systems), pages 99-121. Springer, 2016.","title":"Rules and mechanics","authors":"Julian Mark J Nelson, Cameron B Togelius, Michael Browne, Cook","year":"2016","venue":"Procedural Content Generation in Games (Computational Synthesis and Creative Systems)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ng et al., 2006] Andrew Ng, Adam Coates, Mark Diel, Varun Ganapathi, Jamie Schulte, Ben Tse, Eric Berger, and Eric Liang. Autonomous inverted helicopter flight via reinforcement learning. Experimental Robotics IX, pages 363-372, 2006.","title":"Autonomous inverted helicopter flight via reinforcement learning","authors":"Andrew Ng, Adam Coates, Mark Diel, Varun Ganapathi, Jamie Schulte, Ben Tse, Eric Berger, Eric Liang","year":"2006","venue":"Experimental Robotics IX","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Oh et al., 2016] Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, and Honglak Lee. Control of memory, active perception, and action in minecraft. arXiv preprint arXiv:1605.09128, 2016.","title":"Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, and Honglak Lee","authors":"Oh","year":"2016","venue":"Control of memory, active perception, and action in minecraft","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pulli et al., 2012] Kari Pulli, Anatoly Baksheev, Kirill Ko- rnyakov, and Victor Eruhimov. Real-time computer vision with OpenCV. Commun. ACM, 55(6):61-69, June 2012.","title":"Real-time computer vision with OpenCV","authors":"Kari Pulli, Anatoly Baksheev, Kirill Kornyakov, Victor Eruhimov","year":"2012","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ranzato et al., 2014] MarcAurelio Ranzato, Arthur Szlam, Joan Bruna, Michael Mathieu, Ronan Collobert, and Sumit Chopra. Video (language) modeling: a baseline for generative models of natural videos. arXiv preprint arXiv:1412.6604, 2014.","title":"MarcAurelio Ranzato","authors":"Ranzato","year":"2014","venue":"Video (language) modeling: a baseline for generative models of natural videos","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Selvaraju et al., 2016] Ramprasaath R Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient- based localization. arXiv preprint arXiv:1610.02391, 2016.","title":"Grad-cam: Why did you say that? visual explanations from deep networks via gradientbased localization","authors":"Abhishek Ramprasaath R Selvaraju, Ramakrishna Das, Michael Vedantam, Devi Cogswell, Dhruv Parikh, Batra","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Singh et al., 2002] Push Singh, Thomas Lin, Erik T Mueller, Grace Lim, Travell Perkins, and Wan Li Zhu. Open mind common sense: Knowledge acquisition from the general public. In OTM Confederated International Conferences\" On the Move to Meaningful Internet Sys- tems\", pages 1223-1237. Springer, 2002.","title":"Open mind common sense: Knowledge acquisition from the general public","authors":"Singh","year":"2002","venue":"OTM Confederated International Conferences\" On the Move to Meaningful Internet Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Summerville et al., 2016] Adam Summerville, Matthew Guzdial, Michael Mateas, and Mark Riedl. Learning player tailored content from observation: Platformer level generation from video traces using lstms. In The 12th Ar- tificial Intelligence and Interactive Digital Entertainment Conference, 2016.","title":"Learning player tailored content from observation: Platformer level generation from video traces using lstms","authors":"Adam Summerville, Matthew Guzdial, Michael Mateas, Mark Riedl","year":"2016","venue":"The 12th Artificial Intelligence and Interactive Digital Entertainment Conference","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Summerville et al., 2017a] Adam Summerville, Morteza Behrooz, Michael Mateas, and Arnav Jhala. What does that ?-block do? learning latent causal affordances from mario play traces. In 1st AAAI Workshop on what's next for AI in games, 2017.","title":"What does that ?-block do? learning latent causal affordances from mario play traces","authors":"Summerville","year":"2017","venue":"1st AAAI Workshop on what's next for AI in games","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Summerville et al., 2017b] Adam Summerville, Sam Snod- grass, Matthew Guzdial, Christoffer Holmgård, Amy K Hoover, Aaron Isaksen, Andy Nealen, and Julian To- gelius. Procedural content generation via machine learning (pcgml). arXiv preprint arXiv:1702.00539, 2017.","title":"Procedural content generation via machine learning","authors":"Summerville","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Togelius et al., 2010] Julian Togelius, Sergey Karakovskiy, and Robin Baumgarten. The 2009 mario ai competition. In IEEE Congress on Evolutionary Computation, pages 1-8, 2010.","title":"The 2009 mario ai competition","authors":"Julian Togelius, Sergey Karakovskiy, Robin Baumgarten","year":"2010","venue":"IEEE Congress on Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Togelius, 2011] Julian Togelius. A procedural critique of deontological reasoning. In Proceedings of DiGRA, 2011.","title":"A procedural critique of deontological reasoning","authors":"Julian Togelius","year":"2011","venue":"Proceedings of DiGRA","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Uriarte and Ontanón, 2015] Alberto Uriarte and Santiago Ontanón. Automatic learning of combat models for rts games. In 11th Artificial Intelligence and Interactive Dig- ital Entertainment Conference, 2015.","title":"Automatic learning of combat models for rts games","authors":"Alberto Uriarte, Santiago Ontanón","year":"2015","venue":"11th Artificial Intelligence and Interactive Digital Entertainment Conference","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vukotić et al., 2017] Vedran Vukotić, Silvia-Laura Pin- tea, Christian Raymond, Guillaume Gravier, and Jan Van Gemert. One-step time-dependent future video frame prediction with a convolutional encoder-decoder neural network. arXiv preprint arXiv:1702.04125, 2017.","title":"One-step time-dependent future video frame prediction with a convolutional encoder-decoder neural network","authors":"Vedran Vukotić, Silvia-Laura Pintea, Christian Raymond, Guillaume Gravier, Jan Van Gemert","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zook and Riedl, 2014] Alexander Zook and Mark O Riedl. Automatic game design via mechanic generation. In AAAI, pages 530-537, 2014.","title":"Automatic game design via mechanic generation","authors":"Alexander Zook, Mark O Riedl","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zook et al., 2015] Alexander Zook, Brent Harrison, and Mark O Riedl. Monte-carlo tree search for simulation- based strategy analysis. In Proceedings of the 10th Con- ference on the Foundations of Digital Games, 2015.","title":"Monte-carlo tree search for simulationbased strategy analysis","authors":"Alexander Zook, Brent Harrison, Mark O Riedl","year":"2015","venue":"Proceedings of the 10th Conference on the Foundations of Digital Games","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}