{"type":"ScholarlyArticle","title":"Aesop: A Visual Storytelling Platform for Conversational AI","authors":[{"name":"Tim Meo","slug":"tim-meo","given":"Tim","family":"Meo","affiliation":"SRI International 201 Washington Rd Princeton , NJ08540 , USA"},{"name":"Aswin Raghavan","slug":"aswin-raghavan","given":"Aswin","family":"Raghavan","affiliation":"SRI International 201 Washington Rd Princeton , NJ08540 , USA"},{"name":"David A. Salter","slug":"david-a-salter","given":"David A.","family":"Salter","affiliation":"SRI International 201 Washington Rd Princeton , NJ08540 , USA"},{"name":"Alex Tozzo","slug":"alex-tozzo","given":"Alex","family":"Tozzo","affiliation":"SRI International 201 Washington Rd Princeton , NJ08540 , USA"},{"name":"Amir Tamrakar","slug":"amir-tamrakar","given":"Amir","family":"Tamrakar","affiliation":"SRI International 201 Washington Rd Princeton , NJ08540 , USA"},{"name":"Mohamed Amer","slug":"mohamed-amer","given":"Mohamed","family":"Amer","affiliation":"SRI International 201 Washington Rd Princeton , NJ08540 , USA"}],"abstract":"We present a new collaborative visual storytelling platform, Aesop, for direction and animation. Aesop consists of a language parser, human gesture monitoring, composition graphs, dialogue state manager, and an interactive 3D animation software. Aesop thus enables 3D spatial and temporal reasoning which are both essential for storytelling. Our key innovation is to enable conversational AI using both verbal and non-verbal communication, which enables research in language, vision, and planning.","doi":"10.24963/ijcai.2018/857","url":"https://ijcai.org/proceedings/2018/857","pdf":"https://ijcai.org/proceedings/2018/0857.pdf","markdown":"https://ijcai.org/proceedings/2018/857.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/857","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Demos","track":"Demos","number":857,"pages":{"first":5844,"last":5846},"keywords":["Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Dialogue","Knowledge Representation, Reasoning, and Logic: Geometric, Spatial, and Temporal Reasoning","Multidisciplinary Topics and Applications: Human-Computer Interaction"],"videos":[],"references":[{"position":1,"raw":"[Allen et al., 2008] James F. Allen, Mary Swift, and Will de Beaumont. Deep semantic analysis of text. In Proceed- ings of the 2008 Conference on Semantics in Text Process- ing, STEP 08. Association for Computational Linguistics, 2008.","title":"Deep semantic analysis of text","authors":"James F Allen, Mary Swift, Will De Beaumont","year":"2008","venue":"Proceedings of the 2008 Conference on Semantics in Text Processing, STEP 08","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beattie et al., 2016] Charles Beattie, Joel Z. Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, Julian Schrittwieser, Keith Anderson, Sarah York, Max Cant, Adam Cain, Adrian Bolton, Stephen Gaffney, Helen King, Demis Hassabis, Shane Legg, and Stig Petersen. Deepmind lab. CoRR, abs/1612.03801, 2016.","title":"Deepmind lab","authors":"Charles Beattie, Joel Z Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, Julian Schrittwieser, Keith Anderson, Sarah York, Max Cant, Adam Cain, Adrian Bolton, Stephen Gaffney, Helen King, Demis Hassabis, Shane Legg, Stig Petersen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ferguson and Allen, 2007] George Ferguson and James Allen. Mixed-initiative systems for collaborative problem solving. AI magazine, 28(2):23, 2007.","title":"Mixed-initiative systems for collaborative problem solving","authors":"Ferguson, Allen ;, George Ferguson, James Allen","year":"2007","venue":"AI magazine","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hermann et al., 2017] Karl Moritz Hermann, Felix Hill, Si- mon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis, and Phil Blunsom. Grounded language learning in a simulated 3d world. CoRR, abs/1706.06551, 2017.","title":"Grounded language learning in a simulated 3d world","authors":"Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech, Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis, Phil Blunsom","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hewlett et al., 2007] Daniel Hewlett, Shane Hoversten, Wesley Kerr, Paul R Cohen, and Yu-Han Chang. Wubble world. In AIIDE, pages 20-24, 2007.","title":"Wubble world","authors":"Daniel Hewlett, Shane Hoversten, Wesley Kerr, Yu-Han Paul R Cohen, Chang","year":"2007","venue":"AIIDE","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Johnson et al., 2016] M. Johnson, K. Hofmann, T. Hutton, and D. Bignell. The malmo platform for artificial intelli- gence experimentation. In International Joint Conference on Artificial Intelligence, 2016.","title":"The malmo platform for artificial intelligence experimentation","authors":"M Johnson, K Hofmann, T Hutton, D Bignell","year":"2016","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/643"},{"position":7,"raw":"[Kim et al., 2018] Sujeong Kim, David Salter, Luke DeLuc- cia, Kilho Son, Mohamed R. Amer, and Amir Tam- rakar. Smilee: Symmetric multi-modal interactions with language-gesture enabled (ai) embodiment. In North American Association for Computational Linguists, 2018.","title":"Smilee: Symmetric multi-modal interactions with language-gesture enabled (ai) embodiment","authors":"Kim","year":"2018","venue":"North American Association for Computational Linguists","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Muvizu, 2018] Muvizu. Muvizu animation software. http://www.muvizu.com/, 2018.","title":"Muvizu. Muvizu animation software","authors":"Muvizu","year":"2018","venue":null,"doi":null,"url":"http://www.muvizu.com/,2018","internal_url":null},{"position":9,"raw":"[Perera et al., 2017] Ian E. Perera, James F. Allen, Lucian Galescu, Choh Man Teng, Mark H. Burstein, Scott E. Friedman, David D. McDonald, and Jeffrey M. Rye. Nat- ural language dialogue for building and learning models and structures. In AAAI, 2017.","title":"Natural language dialogue for building and learning models and structures","authors":"Ian E Perera, James F Allen, Lucian Galescu, Man Choh, Mark H Teng, Scott E Burstein, David D Friedman, Jeffrey M Mcdonald, Rye","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Tran et al., 2016] Dustin Tran, Alp Kucukelbir, Adji B. Di- eng, Maja Rudolph, Dawen Liang, and David M. Blei. Ed- ward: a library for probabilistic modeling, inference, and criticism. arXiv preprint arXiv:1610.09787, 2016.","title":"Edward: a library for probabilistic modeling, inference, and criticism","authors":"Dustin Tran, Alp Kucukelbir, B Adji, Maja Dieng, Dawen Rudolph, David M Liang, Blei","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Vicol et al., 2015] Paul Vicol, Makarand Tapaswi, Lluis Castrejon, and Sanja Fidler. Moviegraphs: Towards un- derstanding human-centric situations from videos. CoRR, abs/1712.06761, 2015.","title":"Moviegraphs: Towards understanding human-centric situations from videos","authors":"Paul Vicol, Makarand Tapaswi, Lluis Castrejon, Sanja Fidler","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Winograd, 1972] Terry Winograd. Understanding natural language. Cognitive Psychology, 3(1):1 -191, 1972.","title":"Terry Winograd. Understanding natural language","authors":"Winograd","year":"1972","venue":"Cognitive Psychology","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}