{"type":"ScholarlyArticle","title":"Multimodal Storytelling via Generative Adversarial Imitation Learning","authors":[{"name":"Zhiqian Chen","slug":"zhiqian-chen","given":"Zhiqian","family":"Chen","affiliation":"Computer Science Department , Virginia Tech , Falls Church , Virginia"},{"name":"Xuchao Zhang","slug":"xuchao-zhang","given":"Xuchao","family":"Zhang","affiliation":"Computer Science Department , Virginia Tech , Falls Church , Virginia"},{"name":"Arnold P. Boedihardjo","slug":"arnold-p-boedihardjo","given":"Arnold P.","family":"Boedihardjo","affiliation":"U. S. Army Corps of Engineers"},{"name":"Jing Dai","slug":"jing-dai","given":"Jing","family":"Dai","affiliation":"Google Inc."},{"name":"Chang-Tien Lu","slug":"chang-tien-lu","given":"Chang-Tien","family":"Lu","affiliation":"Computer Science Department , Virginia Tech , Falls Church , Virginia"}],"abstract":"Deriving event storylines is an effective summarization method to succinctly organize extensive information, which can significantly alleviate the pain of information overload. The critical challenge is the lack of widely recognized definition of storyline metric. Prior studies have developed various approaches based on different assumptions about users' interests. These works can extract interesting patterns, but their assumptions do not guarantee that the derived patterns will match users' preference. On the other hand, their exclusiveness of single modality source misses cross-modality information. This paper proposes a method, multimodal imitation learning via Generative Adversarial Networks(MIL-GAN), to directly model users' interests as reflected by various data. In particular, the proposed model addresses the critical challenge by imitating users' demonstrated storylines. Our proposed model is designed to learn the reward patterns given user-provided storylines and then applies the learned policy to unseen data. The proposed approach is demonstrated to be capable of acquiring the user's implicit intent and outperforming competing methods by a substantial margin with a user study.","doi":"10.24963/ijcai.2017/554","url":"https://ijcai.org/proceedings/2017/554","pdf":"https://ijcai.org/proceedings/2017/0554.pdf","markdown":"https://ijcai.org/proceedings/2017/554.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/554","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":554,"pages":{"first":3967,"last":3973},"keywords":["Machine Learning: Deep Learning","Natural Language Processing: Natural Language Summarization","Machine Learning: Reinforcement Learning"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] M. Arjovsky, S. Chintala, and L. Bot- tou. Wasserstein GAN. ArXiv e-prints, jan 2017.","title":null,"authors":"M Arjovsky, S Chintala, L Bottou, Gan Wasserstein","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2015] Samy Bengio, Oriol Vinyals, et al. Scheduled sampling for sequence prediction with recur- rent neural networks. In NIPS, pages 1171-1179, 2015.","title":"Scheduled sampling for sequence prediction with recurrent neural networks","authors":"Samy Bengio, Oriol Vinyals","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fang et al., 2011] Lujun Fang, Anish Das Sarma, Cong Yu, and Philip Bohannon. Rex: explaining relationships be- tween entity pairs. VLDB, 5(3):241-252, 2011.","title":"Rex: explaining relationships between entity pairs","authors":"Fang","year":"2011","venue":"VLDB","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Yoshua Bengio, et al. Generative adversarial nets. In NIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2016] Ji He, Mari Ostendorf, et al. Deep rein- forcement learning with a combinatorial action space for predicting popular reddit threads. EMNLP, 2016.","title":"Deep reinforcement learning with a combinatorial action space for predicting popular reddit threads","authors":"Ji He, Mari Ostendorf","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ho and Ermon, 2016] Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In NIPS, pages 4565-4573, 2016.","title":"Generative adversarial imitation learning","authors":"Jonathan Ho, Stefano Ermon","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hossain et al., 2012] M Shahriar Hossain, Patrick Butler, Arnold P Boedihardjo, and Naren Ramakrishnan. Story- telling in entity networks to support intelligence analysts. In SIGKDD, pages 1375-1383. ACM, 2012.","title":"Storytelling in entity networks to support intelligence analysts","authors":"Patrick Shahriar Hossain, Arnold P Butler, Naren Boedihardjo, Ramakrishnan","year":"2012","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang and Huang, 2013] Lifu Huang and Lian'en Huang. Optimized event storyline generation based on mixture- event-aspect model. In EMNLP, pages 726-735, 2013.","title":"Optimized event storyline generation based on mixtureevent-aspect model","authors":"Huang Huang, Lifu Huang, Lian'en Huang","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim et al., 2014] Gunhee Kim, Leonid Sigal, et al. Joint summarization of large-scale collections of web images and videos for storyline reconstruction. In CVPR, pages 4225-4232, 2014.","title":"Joint summarization of large-scale collections of web images and videos for storyline reconstruction","authors":"Kim","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kiros et al., 2015] Ryan Kiros, Ruslan Salakhutdinov, and Richard S Zemel. Unifying visual-semantic embeddings with multimodal neural language models. TACL, 2015.","title":"Unifying visual-semantic embeddings with multimodal neural language models","authors":"Ryan Kiros, Ruslan Salakhutdinov, Richard S Zemel","year":"2015","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kumar et al., 2008] Deept Kumar, Naren Ramakrishnan, Richard F Helm, and Malcolm Potts. Algorithms for sto- rytelling. KDD, 20(6):736-751, 2008.","title":"Algorithms for storytelling","authors":"Kumar","year":"2008","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lample and Chaplot, 2016] Guillaume Lample and Deven- dra Singh Chaplot. Playing fps games with deep reinforce- ment learning. AAAI, 2016.","title":"Guillaume Lample and Devendra Singh Chaplot. Playing fps games with deep reinforcement learning","authors":"Chaplot Lample","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lee et al., 2012] Heeyoung Lee, Marta Recasens, et al. Joint entity and event coreference resolution across doc- uments. In EMNLP-CONLL, pages 489-500. Association for Computational Linguistics, 2012.","title":"Joint entity and event coreference resolution across documents","authors":"Lee","year":"2012","venue":"EMNLP-CONLL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2012] Chen Lin, Chun Lin, et al. Generating event storylines from microblogs. In CIKM, pages 175- 184. ACM, 2012.","title":"Generating event storylines from microblogs","authors":"Chen Lin, Chun Lin","year":"2012","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. Human-level control through deep re- inforcement learning. Nature, 518(7540):529-533, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Narasimhan et al., 2016] Karthik Narasimhan, Adam Yala, et al. Improving information extraction by acquiring exter- nal evidence with reinforcement learning. EMNLP, 2016.","title":"Improving information extraction by acquiring external evidence with reinforcement learning","authors":"Karthik Narasimhan, Adam Yala","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ng et al., 2000] Andrew Y Ng, Stuart J Russell, et al. Algo- rithms for inverse reinforcement learning. In ICML, pages 663-670, 2000.","title":"Algorithms for inverse reinforcement learning","authors":"Y Andrew, Stuart J Ng, Russell","year":"2000","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ngiam et al., 2011] Jiquan Ngiam, Aditya Khosla, et al. Multimodal deep learning. In ICML, pages 689-696, 2011.","title":"Multimodal deep learning","authors":"Ngiam","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Oh et al., 2015] Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, et al. Action-conditional video prediction using deep net- works in atari games. In NIPS, pages 2863-2871, 2015.","title":"Action-conditional video prediction using deep networks in atari games","authors":"Oh","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Park and Kim, 2015] Cesc C Park and Gunhee Kim. Ex- pressing an image stream with a sequence of natural sen- tences. In NIPS, pages 73-81, 2015.","title":"Expressing an image stream with a sequence of natural sentences","authors":"C Cesc, Gunhee Park, Kim","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pomerleau, 1991] Dean A Pomerleau. Efficient training of artificial neural networks for autonomous navigation. Neu- ral Computation, 3(1):88-97, 1991.","title":"Efficient training of artificial neural networks for autonomous navigation","authors":"A Dean, Pomerleau","year":"1991","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Russell, 1998] Stuart Russell. Learning agents for uncer- tain environments. In Proceedings of the eleventh annual conference on Computational learning theory, pages 101- 103. ACM, 1998.","title":"Learning agents for uncertain environments","authors":"Stuart Russell","year":"1998","venue":"Proceedings of the eleventh annual conference on Computational learning theory","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shahaf and Guestrin, 2010] Dafna Shahaf and Carlos Guestrin. Connecting the dots between news articles. In SIGKDD, pages 623-632. ACM, 2010.","title":"Connecting the dots between news articles","authors":"Guestrin Shahaf, Dafna Shahaf, Carlos Guestrin","year":"2010","venue":"SIGKDD","doi":null,"url":null,"internal_url":"/proceedings/2011/455"},{"position":24,"raw":"[Shahaf et al., 2012a] Dafna Shahaf, Carlos Guestrin, and Eric Horvitz. Metro maps of science. In SIGKDD, pages 1122-1130. ACM, 2012.","title":"Metro maps of science","authors":"Shahaf","year":"2012","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shahaf et al., 2012b] Dafna Shahaf, Carlos Guestrin, and Eric Horvitz. Trains of thought: Generating information maps. In WWW, pages 899-908. ACM, 2012.","title":"Trains of thought: Generating information maps","authors":"Shahaf","year":"2012","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Silver et al., 2016] David Silver, Aja Huang, et al. Master- ing the game of go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.","title":"Mastering the game of go with deep neural networks and tree search","authors":"David Silver, Aja Huang","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Srivastava and Salakhutdinov, 2012] Nitish Srivastava and Ruslan R Salakhutdinov. Multimodal learning with deep boltzmann machines. In NIPS, pages 2222-2230, 2012.","title":"Multimodal learning with deep boltzmann machines","authors":"Nitish Srivastava, Ruslan R Salakhutdinov","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"Sutton et al., ] Richard S Sutton, David A McAllester, Satinder P Singh, Yishay Mansour, et al. Policy gradient methods for reinforcement learning with function approx- imation. [Voskarides et al., 2015]","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"Sutton","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"Nikos Voskarides, Edgar Meij, et al. Learning to explain entity relationships in knowl- edge graphs. 2015.","title":"Learning to explain entity relationships in knowledge graphs","authors":"Nikos Voskarides, Edgar Meij","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2012] Dingding Wang, Tao Li, and Mitsunori Ogihara. Generating pictorial storylines via minimum- weight connected dominating set approximation in multi- view graphs. In AAAI. Citeseer, 2012.","title":"Generating pictorial storylines via minimumweight connected dominating set approximation in multiview graphs","authors":"Dingding Wang, Tao Li, Mitsunori Ogihara","year":"2012","venue":"AAAI. Citeseer","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Williams, 1992] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8(3-4):229-256, 1992.","title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","authors":"Williams Ronald","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yu et al., 2017] L Yu, W Zhang, et al. Seqgan: sequence generative adversarial nets with policy gradient. vol- ume 31. AAAI, 2017.","title":"Seqgan: sequence generative adversarial nets with policy gradient","authors":"Yu, Zhang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang and LeCun, 2015] Xiang Zhang and Yann LeCun. Text understanding from scratch. arXiv preprint arXiv:1502.01710, 2015.","title":"Text understanding from scratch","authors":"Xiang Zhang, Yann Lecun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhou et al., 2015] Deyu Zhou, Liangyu Chen, et al. An un- supervised framework of exploring events on twitter: Fil- tering, extraction and categorization. In AAAI, 2015.","title":"An unsupervised framework of exploring events on twitter: Filtering, extraction and categorization","authors":"Zhou","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}