{"type":"ScholarlyArticle","title":"An AI-empowered Visual Storyline Generator","authors":[{"name":"Chang Liu","slug":"chang-liu","given":"Chang","family":"Liu","affiliation":"School of Computer Science and Engineering , Nanyang Technological University (NTU) , Singapore"},{"name":"Zhao Yong Lim","slug":"zhao-yong-lim","given":"Zhao Yong","family":"Lim","affiliation":"School of Computer Science and Engineering , Nanyang Technological University (NTU) , Singapore"},{"name":"Han Yu","slug":"han-yu","given":"Han","family":"Yu","affiliation":"School of Computer Science and Engineering , Nanyang Technological University (NTU) , Singapore"},{"name":"Zhiqi Shen","slug":"zhiqi-shen","given":"Zhiqi","family":"Shen","affiliation":"School of Computer Science and Engineering , Nanyang Technological University (NTU) , Singapore; Alibaba-NTU Singapore Joint Research Institute , Singapore"},{"name":"Ian Dixon","slug":"ian-dixon","given":"Ian","family":"Dixon","affiliation":"Wee Kim Wee School of Communication and Information , NTU , Singapore"},{"name":"Zhanning Gao","slug":"zhanning-gao","given":"Zhanning","family":"Gao","affiliation":"Alibaba Group , Hangzhou , China"},{"name":"Pan Wang","slug":"pan-wang","given":"Pan","family":"Wang","affiliation":"Alibaba Group , Hangzhou , China"},{"name":"Peiran Ren","slug":"peiran-ren","given":"Peiran","family":"Ren","affiliation":"Alibaba Group , Hangzhou , China"},{"name":"Xuansong Xie","slug":"xuansong-xie","given":"Xuansong","family":"Xie","affiliation":"Alibaba Group , Hangzhou , China"},{"name":"Lizhen Cui","slug":"lizhen-cui","given":"Lizhen","family":"Cui","affiliation":"School of Software , Shandong University (SDU) , China; Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR) , Shandong University , China"},{"name":"Chunyan Miao","slug":"chunyan-miao","given":"Chunyan","family":"Miao","affiliation":"School of Computer Science and Engineering , Nanyang Technological University (NTU) , Singapore"}],"abstract":"Video editing is currently a highly skill- and time-intensive process. One of the most important tasks in video editing is to compose the visual storyline. This paper outlines Visual Storyline Generator (VSG), an artificial intelligence (AI)-empowered system that automatically generates visual storylines based on a set of images and video footages provided by the user. It is designed to produce engaging and persuasive promotional videos with an easy-to-use interface. In addition, users can be involved in refining the AI-generated visual storylines. The editing results can be used as training data to further improve the AI algorithms in VSG.","doi":"10.24963/ijcai.2020/765","url":"https://ijcai.org/proceedings/2020/765","pdf":"https://ijcai.org/proceedings/2020/0765.pdf","markdown":"https://ijcai.org/proceedings/2020/765.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/765","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Demos","track":"Demos","number":765,"pages":{"first":5267,"last":5269},"keywords":["Computer Vision: general","Human-Computer Interactive Systems: general"],"videos":[],"references":[{"position":1,"raw":"[Choi et al., 2016] Jinsoo Choi, Tae-Hyun Oh, and In So Kweon. Video-story composition via plot analysis. In CVPR, pages 3122-3130, 2016.","title":"Video-story composition via plot analysis","authors":"Jinsoo Choi, Tae-Hyun Oh, In So Kweon","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Kim et al., 2018] Hye-Rin Kim, Yeong-Seok Kim, Seon Joo Kim, and In-Kwon Lee. Building Emotional Machines: Recognizing Image Emotions Through Deep Neural Networks. IEEE Transactions on Multimedia, 20(11):2980-2992, November 2018.","title":"Building Emotional Machines: Recognizing Image Emotions Through Deep Neural Networks","authors":"Hye-Rin Kim, Yeong-Seok Kim, Seon Joo Kim, In-Kwon Lee","year":"2018","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Liu et al., 2019] Chang Liu, Yi Dong, Han Yu, Zhiqi Shen, Zhanning Gao, Pan Wang, Changgong Zhang, Peiran Ren, Xuansong Xie, Lizhen Cui, and Chunyan Miao. Generat- ing Persuasive Visual Storylines for Promotional Videos. In CIKM, pages 901-910, 2019.","title":"Generating Persuasive Visual Storylines for Promotional Videos","authors":"Chang Liu, Yi Dong, Han Yu, Zhiqi Shen, Zhanning Gao, Pan Wang, Changgong Zhang, Peiran Ren, Xuansong Xie, Lizhen Cui, Chunyan Miao","year":"2019","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Liu et al., 2020] Chang Liu, Han Yu, Yi Dong, Zhiqi Shen, Yingxue Yu, Ian Dixon, Zhanning Gao, Pan Wang, Chang- gong Zhang, Peiran Ren, Xuansong Xie, Lizhen Cui, and Chunyan Miao. Generating Engaging Promotional Videos for E-commerce Platforms. In AAAI, 2020.","title":"Generating Engaging Promotional Videos for E-commerce Platforms","authors":"Chang Liu, Han Yu, Yi Dong, Zhiqi Shen, Yingxue Yu, Ian Dixon, Zhanning Gao, Pan Wang, Changgong Zhang, Peiran Ren, Xuansong Xie, Lizhen Cui, Chunyan Miao","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Sigurdsson et al., 2016] Gunnar A Sigurdsson, Xinlei Chen, and Abhinav Gupta. Learning visual storylines with skipping recurrent neural networks. In ECCV, pages 71-88, 2016.","title":"Learning visual storylines with skipping recurrent neural networks","authors":"Xinlei Gunnar A Sigurdsson, Abhinav Chen, Gupta","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Talebi and Milanfar, 2018] Hossein Talebi and Peyman Mi- lanfar. NIMA: Neural image assessment. IEEE Transac- tions on Image Processing, 27(8):3998-4011, 2018.","title":"NIMA: Neural image assessment","authors":"Hossein Talebi, Peyman Milanfar","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Wang et al., 2004] Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, and others. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-612, 2004.","title":"Image quality assessment: from error visibility to structural similarity","authors":"Zhou Wang, Alan C Bovik, Eero P Hamid R Sheikh, Simoncelli, Others","year":"2004","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Xu et al., 2019] Dejing Xu, Jun Xiao, Zhou Zhao, Jian Shao, Di Xie, and Yueting Zhuang. Self-Supervised Spa- tiotemporal Learning via Video Clip Order Prediction. In CVPR, pages 10334-10343, 2019.","title":"Self-Supervised Spatiotemporal Learning via Video Clip Order Prediction","authors":"Xu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Yu et al., 2018] Han Yu, Zhiqi Shen, Chunyan Miao, Cyril Leung, Victor R. Lesser, and Qiang Yang. Building Ethics into Artificial Intelligence. In IJCAI, pages 5527-5533, July 2018.","title":"Building Ethics into Artificial Intelligence","authors":"Han Yu, Zhiqi Shen, Chunyan Miao, Cyril Leung, Victor R Lesser, Qiang Yang","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/779"},{"position":10,"raw":"[Zhong et al., 2018] Guangyu Zhong, Yi-Hsuan Tsai, Sifei Liu, Zhixun Su, and Ming-Hsuan Yang. Learning Video- Story Composition via Recurrent Neural Network. In WACV, pages 1727-1735, 2018.","title":"Learning Video-Story Composition via Recurrent Neural Network","authors":"Guangyu Zhong, Yi-Hsuan Tsai, Sifei Liu, Zhixun Su, Ming-Hsuan Yang","year":"2018","venue":"WACV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}