{"type":"ScholarlyArticle","title":"Densely Supervised Hierarchical Policy-Value Network for Image Paragraph Generation","authors":[{"name":"Siying Wu","slug":"siying-wu","given":"Siying","family":"Wu","affiliation":"National Engineering Laboratory for Brain-inspired Intelligence Technology and Application , University of Science and Technology of China"},{"name":"Zheng-Jun Zha","slug":"zheng-jun-zha","given":"Zheng-Jun","family":"Zha","affiliation":"National Engineering Laboratory for Brain-inspired Intelligence Technology and Application , University of Science and Technology of China"},{"name":"Zilei Wang","slug":"zilei-wang","given":"Zilei","family":"Wang","affiliation":"National Engineering Laboratory for Brain-inspired Intelligence Technology and Application , University of Science and Technology of China"},{"name":"Houqiang Li","slug":"houqiang-li","given":"Houqiang","family":"Li","affiliation":"National Engineering Laboratory for Brain-inspired Intelligence Technology and Application , University of Science and Technology of China"},{"name":"Feng Wu","slug":"feng-wu","given":"Feng","family":"Wu","affiliation":"National Engineering Laboratory for Brain-inspired Intelligence Technology and Application , University of Science and Technology of China"}],"abstract":"Image paragraph generation aims to describe an image with a paragraph in natural language.  Compared to image captioning with a single sentence, paragraph generation provides more expressive and fine-grained description for storytelling.  Existing approaches mainly optimize paragraph generator towards minimizing\r\nword-wise cross entropy loss, which neglects linguistic hierarchy of paragraph and results in ``sparse\" supervision for generator learning.  In this paper, we propose a novel Densely Supervised Hierarchical Policy-Value (DHPV) network for effective paragraph generation. We design new hierarchical supervisions consisting of hierarchical rewards and values at both sentence and word levels. The joint exploration of hierarchical rewards and values provides dense supervision cues for learning effective paragraph generator. We propose a new hierarchical policy-value architecture which exploits compositionality at token-to-token and sentence-to-sentence levels simultaneously and can preserve the semantic and syntactic constituent integrity. Extensive experiments on the Stanford image-paragraph benchmark have demonstrated the effectiveness of the proposed DHPV approach with performance improvements over multiple state-of-the-art methods.","doi":"10.24963/ijcai.2019/137","url":"https://ijcai.org/proceedings/2019/137","pdf":"https://ijcai.org/proceedings/2019/0137.pdf","markdown":"https://ijcai.org/proceedings/2019/137.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/137","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":137,"pages":{"first":975,"last":981},"keywords":["Computer Vision: Language and Vision","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Anderson et al., 2018] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for im- age captioning and visual question answering. In CVPR, volume 3, page 6, 2018.","title":"Bottom-up and top-down attention for image captioning and visual question answering","authors":"Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, Lei Zhang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio S, 2015] Jaitly N Shazeer N. Bengio S, Vinyals O. Scheduled sampling for sequence prediction with recur- rent neural networks. In NIPS, pages 1171-1179, 2015.","title":"Scheduled sampling for sequence prediction with recurrent neural networks","authors":"N Jaitly, N Shazeer, S Bengio, O Vinyals","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chatterjee and Schwing, 2018] Moitreya Chatterjee and Alexander G Schwing. Diverse and coherent paragraph generation from images. In ECCV, pages 729-744, 2018.","title":"Diverse and coherent paragraph generation from images","authors":"Schwing Chatterjee, Moitreya Chatterjee, Alexander G Schwing","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Che et al., 2018] Wenbin Che, Xiaopeng Fan, Ruiqin Xiong, and Debin Zhao. Paragraph generation network with visual relationship detection. In 2018 ACM MM, pages 1435-1443, 2018.","title":"Paragraph generation network with visual relationship detection","authors":"Wenbin Che, Xiaopeng Fan, Ruiqin Xiong, Debin Zhao","year":"2018","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Farhadi et al., 2010] Ali Farhadi, Mohsen Hejrati, Moham- mad Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, and David Forsyth. Every picture tells a story: Generating sentences from images. In ECCV, pages 15-29, 2010.","title":"Every picture tells a story: Generating sentences from images","authors":"Ali Farhadi, Mohsen Hejrati, Mohammad Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, David Forsyth","year":"2010","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Frome et al., 2013] Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Tomas Mikolov, et al. Devise: A deep visual-semantic embedding model. In NIPS, pages 2121-2129, 2013.","title":"Devise: A deep visual-semantic embedding model","authors":"Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Tomas Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In NIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Johnson et al., 2016] Justin Johnson, Andrej Karpathy, and Li Fei-Fei. Densecap: Fully convolutional localization networks for dense captioning. In CVPR, pages 4565- 4574, 2016.","title":"Densecap: Fully convolutional localization networks for dense captioning","authors":"Justin Johnson, Andrej Karpathy, Li Fei-Fei","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Karpathy and Fei-Fei, 2015] Andrej Karpathy and Li Fei- Fei. Deep visual-semantic alignments for generating im- age descriptions. In CVPR, pages 3128-3137, 2015.","title":"Deep visual-semantic alignments for generating image descriptions","authors":"Fei-Fei ; Karpathy, Andrej Karpathy, Li Fei-Fei","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krause et al., 2017] Jonathan Krause, Justin Johnson, Ran- jay Krishna, and Li Fei-Fei. A hierarchical approach for generating descriptive image paragraphs. In CVPR, pages 3337-3345. IEEE, 2017.","title":"A hierarchical approach for generating descriptive image paragraphs","authors":"Jonathan Krause, Justin Johnson, Ranjay Krishna, Li Fei-Fei","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kulkarni et al., 2011] Girish Kulkarni, Visruth Premraj, Sagnik Dhar, Siming Li, Yejin Choi, Alexander C Berg, and Tamara L Berg. Baby talk: Understanding and gener- ating image descriptions. In CVPR, 2011.","title":"Baby talk: Understanding and generating image descriptions","authors":"Girish Kulkarni, Visruth Premraj, Sagnik Dhar, Siming Li, Yejin Choi, Alexander C Berg, Tamara L Berg","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2011] Siming Li, Girish Kulkarni, Tamara L Berg, Alexander C Berg, and Yejin Choi. Composing simple im- age descriptions using web-scale n-grams. In ACL, pages 220-228, 2011.","title":"Composing simple image descriptions using web-scale n-grams","authors":"Siming Li, Girish Kulkarni, Tamara L Berg, Alexander C Berg, Yejin Choi","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liang et al., 2017] Xiaodan Liang, Zhiting Hu, Hao Zhang, Chuang Gan, and Eric P Xing. Recurrent topic-transition gan for visual paragraph generation. In ICCV, pages 3362- 3371, 2017.","title":"Recurrent topic-transition gan for visual paragraph generation","authors":"Xiaodan Liang, Zhiting Hu, Hao Zhang, Chuang Gan, Eric P Xing","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lillicrap et al., 2015] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous con- trol with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.","title":"Continuous control with deep reinforcement learning","authors":"Jonathan J Timothy P Lillicrap, Alexander Hunt, Nicolas Pritzel, Tom Heess, Yuval Erez, David Tassa, Daan Silver, Wierstra","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2017] Siqi Liu, Zhenhai Zhu, Ning Ye, Sergio Guadarrama, and Kevin Murphy. Improved image caption- ing via policy gradient optimization of spider. In ICCV, pages 873-881, 2017.","title":"Improved image captioning via policy gradient optimization of spider","authors":"Siqi Liu, Zhenhai Zhu, Ning Ye, Sergio Guadarrama, Kevin Murphy","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mao et al., 2014] Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan Yuille. Deep captioning with multimodal recurrent neural networks (m-rnn). arXiv preprint arXiv:1412.6632, 2014.","title":"Deep captioning with multimodal recurrent neural networks (m-rnn)","authors":"Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, Alan Yuille","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mao et al., 2018] Yuzhao Mao, Chang Zhou, Xiaojie Wang, and Ruifan Li. Show and tell more: Topic-oriented multi- sentence image captioning. In IJCAI-18, pages 4258- 4264, July 2018.","title":"Show and tell more: Topic-oriented multisentence image captioning","authors":"Yuzhao Mao, Chang Zhou, Xiaojie Wang, Ruifan Li","year":"2018","venue":"IJCAI-18","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ren et al., 2017] Zhou Ren, Xiaoyu Wang, Ning Zhang, Xutao Lv, and Li-Jia Li. Deep reinforcement learning- based image captioning with embedding reward. In CVPR, pages 290-298, 2017.","title":"Deep reinforcement learningbased image captioning with embedding reward","authors":"Xiaoyu Zhou Ren, Ning Wang, Xutao Zhang, Li-Jia Lv, Li","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rennie et al., 2017] Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. Self- critical sequence training for image captioning. In CVPR, volume 1, page 3, 2017.","title":"Selfcritical sequence training for image captioning","authors":"Etienne Steven J Rennie, Youssef Marcheret, Jarret Mroueh, Vaibhava Ross, Goel","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vinyals et al., 2015] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neu- ral image caption generator. In CVPR, pages 3156-3164, 2015.","title":"Show and tell: A neural image caption generator","authors":"Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2018] Ziwei Wang, Yadan Luo, Yang Li, Zi Huang, and Hongzhi Yin. Look deeper see richer: Depth-aware image paragraph captioning. In 2018 ACM MM, pages 672-680, 2018.","title":"Look deeper see richer: Depth-aware image paragraph captioning","authors":"Ziwei Wang, Yadan Luo, Yang Li, Zi Huang, Hongzhi Yin","year":"2018","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2015] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In ICML, pages 2048-2057, 2015.","title":"Show, attend and tell: Neural image caption generation with visual attention","authors":"Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, Yoshua Bengio","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[You et al., 2016] Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. Image captioning with semantic attention. In CVPR, pages 4651-4659, 2016.","title":"Image captioning with semantic attention","authors":"You","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zha et al., 2019] Zheng-Jun Zha, Daqing Liu, Hanwang Zhang, Yongdong Zhang, and Feng Wu. Context-aware visual policy network for fine-grained image captioning. IEEE transactions on pattern analysis and machine intel- ligence, 2019.","title":"Context-aware visual policy network for fine-grained image captioning","authors":"Zheng-Jun Zha, Daqing Liu, Hanwang Zhang, Yongdong Zhang, Feng Wu","year":"2019","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2017] Li Zhang, Flood Sung, Feng Liu, Tao Xiang, Shaogang Gong, Yongxin Yang, and Timothy M Hospedales. Actor-critic sequence training for image cap- tioning. arXiv preprint arXiv:1706.09601, 2017.","title":"Actor-critic sequence training for image captioning","authors":"Li Zhang, Flood Sung, Feng Liu, Tao Xiang, Shaogang Gong, Yongxin Yang, Timothy M Hospedales","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}