{"type":"ScholarlyArticle","title":"Cross-modal Bidirectional Translation via Reinforcement Learning","authors":[{"name":"Jinwei Qi","slug":"jinwei-qi","given":"Jinwei","family":"Qi","affiliation":"Institute of Computer Science and Technology , Peking University , Beijing 100871 , China"},{"name":"Yuxin Peng","slug":"yuxin-peng","given":"Yuxin","family":"Peng","affiliation":"Institute of Computer Science and Technology , Peking University , Beijing 100871 , China"}],"abstract":"The inconsistent distribution and representation of image and text make it quite challenging to measure their similarity, and construct correlation between them. Inspired by neural machine translation to establish a corresponding relationship between two entirely different languages, we attempt to treat images as a special kind of language to provide visual descriptions, so that translation can be conduct between bilingual pair of image and text to effectively explore cross-modal correlation. Thus, we propose Cross-modal Bidirectional Translation (CBT) approach, and further explore the utilization of reinforcement learning to improve the translation process. First, a cross-modal translation mechanism is proposed, where image and text are treated as bilingual pairs, and cross-modal correlation can be effectively captured in both feature spaces of image and text by bidirectional translation training. Second, cross-modal reinforcement learning is proposed to perform a bidirectional game between image and text, which is played as a round to promote the bidirectional translation process. Besides, both inter-modality and intra-modality reward signals can be extracted to provide complementary clues for boosting cross-modal correlation learning. Experiments are conducted to verify the performance of our proposed approach on cross-modal retrieval, compared with 11 state-of-the-art methods on 3 datasets.","doi":"10.24963/ijcai.2018/365","url":"https://ijcai.org/proceedings/2018/365","pdf":"https://ijcai.org/proceedings/2018/0365.pdf","markdown":"https://ijcai.org/proceedings/2018/365.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/365","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":365,"pages":{"first":2630,"last":2636},"keywords":["Computer Vision: Language and Vision","Machine Learning: Reinforcement Learning"],"videos":[],"references":[{"position":1,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff A. Bilmes, and Karen Livescu. Deep canonical correlation analysis. In International Conference on Machine Learn- ing (ICML), pages 1247-1255, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Jeff A Bilmes, Karen Livescu","year":"2013","venue":"International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR), 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Caicedo and Lazebnik, 2015] Juan C. Caicedo and Svetlana Lazebnik. Active object localization with deep reinforce- ment learning. In IEEE International Conference on Com- puter Vision (ICCV), pages 2488-2496, 2015.","title":"Active object localization with deep reinforcement learning","authors":"Juan C Caicedo, Svetlana Lazebnik","year":"2015","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, C ¸aglar Gülc ¸ehre, Dzmitry Bahdanau, and et al. Learn- ing phrase representations using RNN encoder-decoder for statistical machine translation. In Conference on Empiri- cal Methods in Natural Language Processing (EMNLP), pages 1724-1734, 2014.","title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Feng et al., 2014] Fangxiang Feng, Xiaojie Wang, and Ruifan Li. Cross-modal retrieval with correspondence autoencoder. In ACM Conference on Multimedia (ACM- MM), pages 7-16, 2014.","title":"Cross-modal retrieval with correspondence autoencoder","authors":"Feng","year":"2014","venue":"ACM Conference on Multimedia (ACM-MM)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hardoon et al., 2004] David R. Hardoon, Sándor Szedmák, and John Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. Neural Computation, 16(12):2639-2664, 2004.","title":"Canonical correlation analysis: An overview with application to learning methods","authors":"David R Hardoon, Sándor Szedmák, John Shawe-Taylor","year":"2004","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Di He, Yingce Xia, Tao Qin, Liwei Wang, and et al. Dual learning for machine translation. In Ad- vances in Neural Information Processing Systems (NIPS), pages 820-828, 2016.","title":"Dual learning for machine translation","authors":"Di He, Yingce Xia, Tao Qin, Liwei Wang","year":"2016","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kang et al., 2015] Cuicui Kang, Shiming Xiang, Shengcai Liao, Changsheng Xu, and Chunhong Pan. Learning consistent feature representation for cross-modal multime- dia retrieval. IEEE Transactions on Multimedia (TMM), 17(3):370-381, 2015.","title":"Learning consistent feature representation for cross-modal multimedia retrieval","authors":"Kang","year":"2015","venue":"IEEE Transactions on Multimedia (TMM)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. In Conference on Empirical Meth- ods in Natural Language Processing (EMNLP), pages 1746-1751, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":"Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2003] Dongge Li, Nevenka Dimitrova, Mingkun Li, and Ishwar K Sethi. Multimedia content processing through cross-modal association. In ACM Conference on Multimedia (ACM-MM), pages 604-611, 2003.","title":"Multimedia content processing through cross-modal association","authors":"Li","year":"2003","venue":"ACM Conference on Multimedia (ACM-MM)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mnih et al., 2013] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, and et al. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.","title":"Playing atari with deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, and et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Mnih","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Peng et al., 2016] Yuxin Peng, Xin Huang, and Jinwei Qi. Cross-media shared representation by hierarchical learn- ing with multiple deep networks. In International Joint Conference on Artificial Intelligence (IJCAI), pages 3846- 3853, 2016.","title":"Cross-media shared representation by hierarchical learning with multiple deep networks","authors":"Yuxin Peng, Xin Huang, Jinwei Qi","year":"2016","venue":"International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2016/541"},{"position":14,"raw":"[Peng et al., 2017a] Yuxin Peng, Xin Huang, and Yunzhen Zhao. An overview of cross-media retrieval: Concepts, methodologies, benchmarks and challenges. IEEE Trans- actions on Circuits and Systems for Video Technology (TCSVT), 2017.","title":"An overview of cross-media retrieval: Concepts, methodologies, benchmarks and challenges","authors":"Peng","year":"2017","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Peng et al., 2017b] Yuxin Peng, Jinwei Qi, Xin Huang, and Yuxin Yuan. CCL: Cross-modal correlation learning with multi-grained fusion by hierarchical network. IEEE Trans- actions on Multimedia (TMM), 2017.","title":"CCL: Cross-modal correlation learning with multi-grained fusion by hierarchical network","authors":"Peng","year":"2017","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rashtchian et al., 2010] Cyrus Rashtchian, Peter Young, Micah Hodosh, and Julia Hockenmaier. Collecting image annotations using amazon's mechanical turk. In Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, pages 139-147, 2010.","title":"Collecting image annotations using amazon's mechanical turk","authors":"Rashtchian","year":"2010","venue":"Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rasiwasia et al., 2010] Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert RG Lanckriet, Roger Levy, and Nuno Vasconcelos. A new approach to cross-modal multimedia retrieval. In ACM Conference on Multimedia (ACM-MM), pages 251-260, 2010.","title":"A new approach to cross-modal multimedia retrieval","authors":"Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert Rg Lanckriet, Roger Levy, Nuno Vasconcelos","year":"2010","venue":"ACM Conference on Multimedia (ACM-MM)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ren et al., 2017] Zhou Ren, Xiaoyu Wang, Ning Zhang, Xutao Lv, and Li-Jia Li. Deep reinforcement learning- based image captioning with embedding reward. In Conference on Computer Vision and Pattern Recognition (CVPR), 2017.","title":"Deep reinforcement learningbased image captioning with embedding reward","authors":"Xiaoyu Zhou Ren, Ning Wang, Xutao Zhang, Li-Jia Lv, Li","year":"2017","venue":"Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Confer- ence on Learning Representations (ICLR), 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems (NIPS), pages 3104-3112, 2014.","title":"Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sutton et al., 1999] Richard S. Sutton, David A. McAllester, Satinder P. Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Infor- mation Processing Systems (NIPS), pages 1057-1063, 1999.","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"Richard S Sutton, David A Mcallester, P Satinder, Yishay Singh, Mansour","year":"1999","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2017] Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Shen Hengtao. Adversarial cross- modal retrieval. In ACM Conference on Multimedia (ACM-MM), pages 154-162, 2017.","title":"Adversarial crossmodal retrieval","authors":"Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, Shen Hengtao","year":"2017","venue":"ACM Conference on Multimedia (ACM-MM)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wei et al., 2017] Yunchao Wei, Yao Zhao, Canyi Lu, Shikui Wei, Luoqi Liu, Zhenfeng Zhu, and Shuicheng Yan. Cross-modal retrieval with CNN visual features: A new baseline. IEEE Transactions on Cybernetics (TCYB), 47(2):449-460, 2017.","title":"Cross-modal retrieval with CNN visual features: A new baseline","authors":"Yunchao Wei, Yao Zhao, Canyi Lu, Shikui Wei, Luoqi Liu, Zhenfeng Zhu, Shuicheng Yan","year":"2017","venue":"IEEE Transactions on Cybernetics (TCYB)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhai et al., 2014] Xiaohua Zhai, YuXin Peng, and Jianguo Xiao. Learning cross-media joint representation with sparse and semi-supervised regularization. IEEE Trans- actions on Circuits and Systems for Video Technology (TCSVT), 24:965-978, 2014.","title":"Learning cross-media joint representation with sparse and semi-supervised regularization","authors":"Xiaohua Zhai, Yuxin Peng, Jianguo Xiao","year":"2014","venue":"IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}