Diverse Image Captioning via GroupTalk

Wang, Zhuhao; Wu, Fei; Lu, Weiming; Xiao, Jun; Li, Xi; Zhang, Zitong; Zhuang, Yueting

Diverse Image Captioning via GroupTalk

Zhuhao Wang, Fei Wu, Weiming Lu, Jun Xiao, Xi Li, Zitong Zhang, Yueting Zhuang

Download BibTeX Cite

Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. Pages 2957-2963.

Abstract

Generally speaking, different persons tend to describe images from various aspects due to their individually subjective perception. As a result, generating the appropriate descriptions of images with both diversity and high quality is of great importance. In this paper, we propose a framework called GroupTalk to learn multiple image caption distributions simultaneously and effectively mimic the diversity of the image captions written by human beings. In particular, a novel iterative update strategy is proposed to separate training sentence samples into groups and learn their distributions at the same time. Furthermore, we introduce an efficient classifier to solve the problem brought about by the non-linear and discontinuous nature of language distributions which will impair performance. Experiments on several benchmark datasets show that GroupTalk naturally diversifies the generated captions of each image without sacrificing the accuracy.

Zhuhao Wang, Fei Wu, Weiming Lu, Jun Xiao, Xi Li, Zitong Zhang, and Yueting Zhuang. 2016. Diverse Image Captioning via GroupTalk. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-16. IJCAI, 2957–2963. https://ijcai.org/proceedings/2016/420

@inproceedings{ijcai2016p420,
  title     = {Diverse Image Captioning via GroupTalk},
  author    = {Wang, Zhuhao and Wu, Fei and Lu, Weiming and Xiao, Jun and Li, Xi and Zhang, Zitong and Zhuang, Yueting},
  booktitle = {Proceedings of the Twenty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-16}},
  publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
  editor    = {Subbarao Kambhampati},
  pages     = {2957--2963},
  year      = {2016},
  month     = {7},
  url       = {https://ijcai.org/proceedings/2016/420},
}

TY  - CPAPER
TI  - Diverse Image Captioning via GroupTalk
AU  - Wang, Zhuhao
AU  - Wu, Fei
AU  - Lu, Weiming
AU  - Xiao, Jun
AU  - Li, Xi
AU  - Zhang, Zitong
AU  - Zhuang, Yueting
T2  - Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-16
PB  - AAAI Press / International Joint Conferences on Artificial Intelligence
PY  - 2016
SP  - 2957
EP  - 2963
UR  - https://ijcai.org/proceedings/2016/420
ER  -