{"type":"ScholarlyArticle","title":"OSIC: A New One-Stage Image Captioner Coined","authors":[{"name":"Bo Wang","slug":"bo-wang","given":"Bo","family":"Wang","affiliation":"Hefei University of Technology , Hefei , China"},{"name":"Zhao Zhang","slug":"zhao-zhang","given":"Zhao","family":"Zhang","affiliation":"Hefei University of Technology , Hefei , China"},{"name":"Mingbo Zhao","slug":"mingbo-zhao","given":"Mingbo","family":"Zhao","affiliation":"Donghua University , Shanghai , China"},{"name":"Xiaojie Jin","slug":"xiaojie-jin","given":"Xiaojie","family":"Jin","affiliation":"Bytedance Research , USA"},{"name":"Mingliang Xu","slug":"mingliang-xu","given":"Mingliang","family":"Xu","affiliation":"Zhengzhou University , Zhengzhou , China"},{"name":"Meng Wang","slug":"meng-wang","given":"Meng","family":"Wang","affiliation":"Hefei University of Technology , Hefei , China"}],"abstract":"Mainstream image captioning models are usually two-stage captioners, i.e., encoding the region features by a pre-trained detector and then feeding them into a language model to generate the captions. However, such a two-stage procedure will lead to a task-based information gap that decreases the performance, because the region features in the detection task are suboptimal representations and cannot provide all the necessary information for subsequent captions generation. Besides, the region features are usually represented from the last layer of the detectors that lose the local details of images. In this paper, we propose a novel One-Stage Image Captioner (OSIC) with dynamic multi-sight learning, which directly transforms the images into descriptive sentences in one stage for eliminating the information gap. Specifically, to obtain rich features, multi-level features are captured by Swin Transformer, and then fed into a novel dynamic multi-sight embedding module to exploit both the global structure and local texture of input images. To enhance the global modeling capacity of the visual encoder, we propose a new dual-dimensional refining to non-locally model the features interaction. As a result, OSIC can directly obtain rich semantic information to improve the captioner. Extensive comparisons on the benchmark MS-COCO, Flickr8K and Flickr30K datasets verified the superior performance of our method.","doi":"10.24963/ijcai.2024/150","url":"https://ijcai.org/proceedings/2024/150","pdf":"https://ijcai.org/proceedings/2024/0150.pdf","markdown":"https://ijcai.org/proceedings/2024/150.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/150","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":150,"pages":{"first":1353,"last":1361},"keywords":["Computer Vision: CV: Machine learning for vision","Computer Vision: CV: Scene analysis and understanding","Computer Vision: CV: Vision, language and reasoning","Natural Language Processing: NLP: Language generation"],"videos":[],"references":[{"position":1,"raw":"[Anderson et al., 2018] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for im- age captioning and visual question answering. In CVPR, 2018.","title":"Bottom-up and top-down attention for image captioning and visual question answering","authors":"Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, Lei Zhang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrej Karpathy, 2017] Li Fei-Fei Andrej Karpathy. Deep visual-semantic alignments for generating image descrip- tions. TPAMI, 2017.","title":"Deep visual-semantic alignments for generating image descriptions","authors":"Li Fei-Fei, Andrej Karpathy","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Banerjee and Lavie, 2005] Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In ACL, 2005.","title":"Meteor: An automatic metric for mt evaluation with improved correlation with human judgments","authors":"Lavie Banerjee, Satanjeev Banerjee, Alon Lavie","year":"2005","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2017] Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, and Tat-Seng Chua. Sca- cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In CVPR, 2017.","title":"Scacnn: Spatial and channel-wise attention in convolutional networks for image captioning","authors":"Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, Tat-Seng Chua","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cornia et al., 2020] Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, and Rita Cucchiara. Meshed-memory transformer for image captioning. In CVPR, 2020.","title":"Meshed-memory transformer for image captioning","authors":"Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, Rita Cucchiara","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. CORR, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":"CORR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ding et al., 2020] Songtao Ding, Shiru Qu, Yuling Xi, and Shaohua Wan. Stimulus-driven and concept-driven analy- sis for image caption generation. Neurocomputing, 2020.","title":"Stimulus-driven and concept-driven analysis for image caption generation","authors":"Songtao Ding, Shiru Qu, Yuling Xi, Shaohua Wan","year":"2020","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Trans- formers for image recognition at scale. In ICLR, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 2022] Zhiyuan Fang, Jianfeng Wang, Xiaowei Hu, Lin Liang, Zhe Gan, Lijuan Wang, Yezhou Yang, and Zicheng Liu. Injecting semantic concepts into end-to-end image captioning. In CVPR, 2022.","title":"Injecting semantic concepts into end-to-end image captioning","authors":"Zhiyuan Fang, Jianfeng Wang, Xiaowei Hu, Lin Liang, Zhe Gan, Lijuan Wang, Yezhou Yang, Zicheng Liu","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fei et al., 2022] Zhengcong Fei, Junshi Huang, Xiaoming Wei, and Xiaolin Wei. Efficient modeling of future con- text for image captioning. In ACM MM, 2022.","title":"Efficient modeling of future context for image captioning","authors":"Fei","year":"2022","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fei et al., 2023] Zhengcong Fei, Mingyuan Fan, Li Zhu, Junshi Huang, Xiaoming Wei, and Xiaolin Wei. Uncertainty-aware image captioning. In AAAI, 2023.","title":"Uncertainty-aware image captioning","authors":"Fei","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gao et al., 2019] Lianli Gao, Kaixuan Fan, Jingkuan Song, Xianglong Liu, Xing Xu, and Heng Tao Shen. Deliberate attention networks for image captioning. In AAAI, 2019.","title":"Deliberate attention networks for image captioning","authors":"Gao","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gao et al., 2022] Yi-Meng Gao, Ning Wang, Wei Suo, Mengyang Sun, and Peifeng Wang. Improving image cap- tioning via enhancing dual-side context awareness. In ICMR, 2022.","title":"Improving image captioning via enhancing dual-side context awareness","authors":"Yi-Meng Gao, Ning Wang, Wei Suo, Mengyang Sun, Peifeng Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Guo et al., 2021] Longteng Guo, Jing Liu, Xinxin Zhu, Xingjian He, Jie Jiang, and Hanqing Lu. Non- autoregressive image captioning with counterfactuals- critical multi-agent learning. In IJCAI, 2021.","title":"Nonautoregressive image captioning with counterfactualscritical multi-agent learning","authors":"Longteng Guo, Jing Liu, Xinxin Zhu, Xingjian He, Jie Jiang, Hanqing Lu","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Herdade et al., 2019] Simao Herdade, Armin Kappeler, Kofi Boakye, and Joao Soares. Image captioning: Trans- forming objects into words. In NeurIPS, 2019.","title":"Image captioning: Transforming objects into words","authors":"Simao Herdade, Armin Kappeler, Kofi Boakye, Joao Soares","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hodosh et al., 2013] Micah Hodosh, Peter Young, and Ju- lia Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. Journal of Ar- tificial Intelligence Research, 2013.","title":"Framing image description as a ranking task: Data, models and evaluation metrics","authors":"Hodosh","year":"2013","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hu et al., 2018] Han Hu, Jiayuan Gu, Zheng Zhang, Jifeng Dai, and Yichen Wei. Relation networks for object detec- tion. In CVPR, 2018.","title":"Relation networks for object detection","authors":"Han Hu, Jiayuan Gu, Zheng Zhang, Jifeng Dai, Yichen Wei","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Huang et al., 2019] Lun Huang, Wenmin Wang, Jie Chen, and Xiao-Yong Wei. Attention on attention for image cap- tioning. In ICCV, 2019.","title":"Attention on attention for image captioning","authors":"Lun Huang, Wenmin Wang, Jie Chen, Xiao-Yong Wei","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jiang et al., 2018] Wenhao Jiang, Lin Ma, Yu-Gang Jiang, W. Liu, and T. Zhang. Recurrent fusion network for image captioning. In ICCV, 2018.","title":"Recurrent fusion network for image captioning","authors":"Wenhao Jiang, Lin Ma, Yu-Gang Jiang, W Liu, T Zhang","year":"2018","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Krishna et al., 2016] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowd- sourced dense image annotations. IJCV, 2016.","title":"Visual genome: Connecting language and vision using crowdsourced dense image annotations","authors":"Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, Michael S Bernstein, Li Fei-Fei","year":"2016","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kuo and Kira, 2022] Chia-Wen Kuo and Zsolt Kira. Be- yond a pre-trained object detector: Cross-modal textual and visual context for image captioning. In CVPR, 2022.","title":"Beyond a pre-trained object detector: Cross-modal textual and visual context for image captioning","authors":"Chia-Wen Kuo, Zsolt Kira","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge J. Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In ECCV, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge J Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lin, 2004] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In ACL, 2004.","title":"Rouge: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu et al., 2017] Chang Liu, Fuchun Sun, Changhu Wang, Feng Wang, and Alan Loddon Yuille. Mat: A multimodal attentive translator for image captioning. In IJCAI, 2017.","title":"Mat: A multimodal attentive translator for image captioning","authors":"Chang Liu, Fuchun Sun, Changhu Wang, Feng Wang, Alan Loddon, Yuille","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/563"},{"position":25,"raw":"[Liu et al., 2018] Daqing Liu, Zheng-Jun Zha, Hanwang Zhang, Yongdong Zhang, and Feng Wu. Context-aware visual policy network for sequence-level image caption- ing. In ACM MM, 2018.","title":"Context-aware visual policy network for sequence-level image captioning","authors":"Daqing Liu, Zheng-Jun Zha, Hanwang Zhang, Yongdong Zhang, Feng Wu","year":"2018","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lu et al., 2017] Jiasen Lu, Caiming Xiong, Devi Parikh, and Richard Socher. Knowing when to look: Adaptive attention via a visual sentinel for image captioning. In CVPR, 2017.","title":"Knowing when to look: Adaptive attention via a visual sentinel for image captioning","authors":"Lu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Luo et al., 2023] Jianjie Luo, Yehao Li, Yingwei Pan, Ting Yao, Jianlin Feng, Hongyang Chao, and Tao Mei. Semantic-conditional diffusion networks for image cap- tioning. In CVPR, 2023.","title":"Semantic-conditional diffusion networks for image captioning","authors":"Luo","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Pan et al., 2020] Yingwei Pan, Ting Yao, Yehao Li, and Tao Mei. X-linear attention networks for image captioning. In CVPR, 2020.","title":"X-linear attention networks for image captioning","authors":"Pan","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In ACL, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervi- sion. In ICML, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross B. Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. TPAMI, 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Rennie et al., 2017] Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. Self- critical sequence training for image captioning. In CVPR, 2017.","title":"Selfcritical sequence training for image captioning","authors":"Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, Vaibhava Goel","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Song et al., 2021] Zeliang Song, Xiaofei Zhou, Linhua Dong, Jianlong Tan, and Li Guo. Direction relation trans- former for image captioning. In ACM MM, 2021.","title":"Direction relation transformer for image captioning","authors":"Song","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Vedantam et al., 2015] Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. Cider: Consensus- based image description evaluation. In CVPR, 2015.","title":"Cider: Consensusbased image description evaluation","authors":"C Lawrence Ramakrishna Vedantam, Devi Zitnick, Parikh","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Vinyals et al., 2015] Oriol Vinyals, Alexander Toshev, Samy Bengio, and D. Erhan. Show and tell: A neural image caption generator. In CVPR, 2015.","title":"Show and tell: A neural image caption generator","authors":"Oriol Vinyals, Alexander Toshev, Samy Bengio, D Erhan","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wang et al., 2022a] Bo Wang, Zhao Zhang, Jicong Fan, Mingbo Zhao, Choujun Zhan, and Mingliang Xu. Fine- former: Fine-grained adaptive object transformer for im- age captioning. In ICDM, 2022.","title":"Fineformer: Fine-grained adaptive object transformer for image captioning","authors":"Wang","year":"2022","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang et al., 2022b] Yiyu Wang, Jungang Xu, and Yingfei Sun. A visual persistence model for image captioning. Neurocomputing, 2022.","title":"A visual persistence model for image captioning","authors":"Wang","year":"2022","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wang et al., 2023a] Ning Wang, Jiahao Xie, Jihao Wu, Mingbo Jia, and Linlin Li. Controllable image captioning via prompting. In AAAI, 2023.","title":"Controllable image captioning via prompting","authors":"Wang","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wang et al., 2023b] Ning Wang, Jiangrong Xie, Hang Luo, Qinglin Cheng, Jihao Wu, Mingbo Jia, and Linlin Li. Ef- ficient image captioning for edge devices. In AAAI, 2023.","title":"Efficient image captioning for edge devices","authors":"Wang","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Wei et al., 2021a] Haiyang Wei, Zhixin Li, Feicheng Huang, Canlong Zhang, Huifang Ma, and Zhongzhi Shi. Integrating scene semantic knowledge into image caption- ing. ACM Transactions on Multimedia Computing, Com- munications, and Applications, 2021.","title":"Integrating scene semantic knowledge into image captioning","authors":"Wei","year":"2021","venue":"ACM Transactions on Multimedia Computing, Communications, and Applications","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Wei et al., 2021b] Yanyan Wei, Zhao Zhang, Yang Wang, Mingliang Xu, Yi Yang, Shuicheng Yan, and Meng Wang. Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking. TIP, 2021.","title":"Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking","authors":"Wei","year":"2021","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wei et al., 2022] Yanyan Wei, Zhao Zhang, Huan Zheng, Richang Hong, Yi Yang, and Meng Wang. Sginet: To- ward sufficient interaction between single image deraining and semantic segmentation. In ACM MM, 2022.","title":"Sginet: Toward sufficient interaction between single image deraining and semantic segmentation","authors":"Yanyan Wei, Zhao Zhang, Huan Zheng, Richang Hong, Yi Yang, Meng Wang","year":"2022","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Yan et al., 2021] Xu Yan, Zhengcong Fei, Zekang Li, Shuhui Wang, Qingming Huang, and Qi Tian. Semi- autoregressive image captioning. In ACM MM, 2021.","title":"Semiautoregressive image captioning","authors":"Zhengcong Xu Yan, Zekang Fei, Shuhui Li, Qingming Wang, Qi Huang, Tian","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Yang et al., 2019] Xu Yang, Kaihua Tang, Hanwang Zhang, and Jianfei Cai. Auto-encoding scene graphs for image captioning. In CVPR, 2019.","title":"Auto-encoding scene graphs for image captioning","authors":"Xu Yang, Kaihua Tang, Hanwang Zhang, Jianfei Cai","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"Yao et al., ] Ting Yao, Yingwei Pan, Yehao Li, and Tao Mei. Hierarchy parsing for image captioning. In CVPR.","title":"Hierarchy parsing for image captioning","authors":"Yao","year":null,"venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Yao et al., 2018] Ting Yao, Yingwei Pan, Yehao Li, and Tao Mei. Exploring visual relationship for image captioning. In ECCV, 2018.","title":"Exploring visual relationship for image captioning","authors":"Ting Yao, Yingwei Pan, Yehao Li, Tao Mei","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"You et al., ] Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. Image captioning with seman- tic attention. In CVPR.","title":"Image captioning with semantic attention","authors":"You","year":null,"venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Young et al., 2014] Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2014.","title":"From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions","authors":"Peter Young, Alice Lai, Micah Hodosh, Julia Hockenmaier","year":"2014","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Yu et al., 2021] Litao Yu, Jian Zhang, and Qiang Wu. Dual attention on pyramid feature maps for image captioning. TMM, 2021.","title":"Dual attention on pyramid feature maps for image captioning","authors":"Yu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zeng et al., 2022a] Pengpeng Zeng, Haonan Zhang, Jingkuan Song, and Lianli Gao. S2 transformer for image captioning. In IJCAI, 2022.","title":"S2 transformer for image captioning","authors":"Zeng","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/224"},{"position":52,"raw":"[Zeng et al., 2022b] Pengpeng Zeng, Jinkuan Zhu, Jingkuan Song, and Lianli Gao. Progressive tree-structured proto- type network for end-to-end image captioning. In ACM MM, 2022.","title":"Progressive tree-structured prototype network for end-to-end image captioning","authors":"Zeng","year":"2022","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhang et al., 2021] Xuying Zhang, Xiaoshuai Sun, Yun- peng Luo, Jiayi Ji, Yiyi Zhou, Yongjian Wu, Feiyue Huang, and Rongrong Ji. Rstnet: Captioning with adap- tive attention on visual and non-visual words. In CVPR, 2021.","title":"Rstnet: Captioning with adaptive attention on visual and non-visual words","authors":"Zhang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Zhao et al., 2022] Suiyi Zhao, Zhao Zhang, Richang Hong, Mingliang Xu, Yi Yang, and Meng Wang. Fcl-gan: A lightweight and real-time baseline for unsupervised blind image deblurring. In ACM MM, 2022.","title":"Fcl-gan: A lightweight and real-time baseline for unsupervised blind image deblurring","authors":"Suiyi Zhao, Zhao Zhang, Richang Hong, Mingliang Xu, Yi Yang, Meng Wang","year":"2022","venue":"ACM MM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}