{"type":"ScholarlyArticle","title":"Lexical-Constraint-Aware Neural Machine Translation via Data Augmentation","authors":[{"name":"Guanhua Chen","slug":"guanhua-chen","given":"Guanhua","family":"Chen","affiliation":"The University of Hong Kong"},{"name":"Yun Chen","slug":"yun-chen","given":"Yun","family":"Chen","affiliation":"Shanghai University of Finance and Economics"},{"name":"Yong Wang","slug":"yong-wang","given":"Yong","family":"Wang","affiliation":"The University of Hong Kong"},{"name":"Victor O.K. Li","slug":"victor-o-k-li","given":"Victor O.K.","family":"Li","affiliation":"The University of Hong Kong"}],"abstract":"Leveraging lexical constraint is extremely significant in domain-specific machine translation and interactive machine translation. Previous studies mainly focus on extending beam search algorithm or augmenting the training corpus by replacing source phrases with the corresponding target translation. These methods either suffer from the heavy computation cost during inference or depend on the quality of the bilingual dictionary pre-specified by user or constructed with statistical machine translation. In response to these problems, we present a conceptually simple and empirically effective data augmentation approach in lexical constrained neural machine translation. Specifically, we make constraint-aware training data by first randomly sampling the phrases of the reference as constraints, and then packing them together into the source sentence with a separation symbol. Extensive experiments on several language pairs demonstrate that our approach achieves superior translation results over the existing systems, improving translation of constrained sentences without hurting the unconstrained ones.","doi":"10.24963/ijcai.2020/496","url":"https://ijcai.org/proceedings/2020/496","pdf":"https://ijcai.org/proceedings/2020/0496.pdf","markdown":"https://ijcai.org/proceedings/2020/496.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/496","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":496,"pages":{"first":3587,"last":3593},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Machine Translation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25275/file_25275.m3u8","length_sec":297,"provider":"hls"}],"references":[{"position":1,"raw":"[Arthur et al., 2016] Philip Arthur, Graham Neubig, and Satoshi Nakamura. Incorporating discrete translation lex- icons into neural machine translation. In Proceedings of EMNLP, pages 1557-1567, 2016.","title":"Incorporating discrete translation lexicons into neural machine translation","authors":"Philip Arthur, Graham Neubig, Satoshi Nakamura","year":"2016","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng et al., 2016] Shanbo Cheng, Shujian Huang, Huadong Chen, Xin-Yu Dai, and Jiajun Chen. PRIMT: A pick-revise framework for interactive machine translation. In Proceedings of NAACL, pages 1240-1249, 2016.","title":"PRIMT: A pick-revise framework for interactive machine translation","authors":"Shanbo Cheng, Shujian Huang, Huadong Chen, Xin-Yu Dai, Jiajun Chen","year":"2016","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of NAACL, pages 4171-4186, 2019.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dinu et al., 2019] Georgiana Dinu, Prashant Mathur, Mar- cello Federico, and Yaser Al-Onaizan. Training neural machine translation to apply terminology constraints. In Proceedings of ACL, pages 3063-3068, 2019.","title":"Training neural machine translation to apply terminology constraints","authors":"Georgiana Dinu, Prashant Mathur, Marcello Federico, Yaser Al-Onaizan","year":"2019","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dyer et al., 2013] Chris Dyer, Victor Chahuneau, and Noah A. Smith. A simple, fast, and effective reparame- terization of IBM Model 2. In Proceedings of NAACL, 2013.","title":"A simple, fast, and effective reparameterization of IBM Model 2","authors":"Chris Dyer, Victor Chahuneau, Noah A Smith","year":"2013","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gulcehre et al., 2016] Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. Pointing the unknown words. In Proceedings of ACL, pages 140-149, 2016.","title":"Pointing the unknown words","authors":"Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, Yoshua Bengio","year":"2016","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hasler et al., 2018] Eva Hasler, Adrià de Gispert, Gonzalo Iglesias, and Bill Byrne. Neural machine translation de- coding with terminology constraints. In Proceedings of NAACL, pages 506-512, 2018.","title":"Neural machine translation decoding with terminology constraints","authors":"Eva Hasler, Adrià De Gispert, Gonzalo Iglesias, Bill Byrne","year":"2018","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hokamp and Liu, 2017] Chris Hokamp and Qun Liu. Lex- ically constrained decoding for sequence generation using grid beam search. In Proceedings of ACL, pages 1535- 1546, 2017.","title":"Lexically constrained decoding for sequence generation using grid beam search","authors":"Chris Hokamp, Qun Liu","year":"2017","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceed- ings of ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Koehn et al., 2003] Philipp Koehn, Franz J. Och, and Daniel Marcu. Statistical phrase-based translation. In Pro- ceedings of NAACL, pages 127-133, 2003.","title":"Statistical phrase-based translation","authors":"Philipp Koehn, Franz J Och, Daniel Marcu","year":"2003","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Koehn et al., 2007] Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Con- stantin, and Evan Herbst. Moses: Open source toolkit for statistical machine translation. In Proceedings of ACL, pages 177-180, 2007.","title":"Moses: Open source toolkit for statistical machine translation","authors":"Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, Evan Herbst","year":"2007","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Koehn, 2009] Philipp Koehn. A process study of computer- aided translation. Machine Translation, 23:241-263, 2009.","title":"A process study of computeraided translation","authors":"Philipp Koehn","year":"2009","venue":"Machine Translation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luong et al., 2015] Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, and Wojciech Zaremba. Addressing the rare word problem in neural machine translation. In Proceed- ings of ACL, pages 11-19, 2015.","title":"Addressing the rare word problem in neural machine translation","authors":"Luong","year":"2015","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Neubig et al., 2019] Graham Neubig, Zi-Yi Dou, Junjie Hu, Paul Michel, Danish Pruthi, and Xinyi Wang. compare- mt: A tool for holistic comparison of language generation systems. In Proceedings of NAACL, 2019.","title":"comparemt: A tool for holistic comparison of language generation systems","authors":"Graham Neubig, Zi-Yi Dou, Junjie Hu, Paul Michel, Danish Pruthi, Xinyi Wang","year":"2019","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ott et al., 2019] Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for se- quence modeling. In Proceedings of NAACL, 2019.","title":"fairseq: A fast, extensible toolkit for sequence modeling","authors":"Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli","year":"2019","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: a method for au- tomatic evaluation of machine translation. In Proceedings of ACL, 2002.","title":"BLEU: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Post and Vilar, 2018] Matt Post and David Vilar. Fast lexi- cally constrained decoding with dynamic beam allocation for neural machine translation. In Proceedings of NAACL, pages 1314-1324, 2018.","title":"Vilar. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation","authors":"Vilar Post, Matt Post, David","year":"2018","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Post, 2018] Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Ma- chine Translation, pages 186-191, 2018.","title":"A call for clarity in reporting BLEU scores","authors":"Matt Post","year":"2018","venue":"Proceedings of the Third Conference on Machine Translation","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sennrich et al., 2016] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of ACL, 2016.","title":"Neural machine translation of rare words with subword units","authors":"Sennrich","year":"2016","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2019] Kai Song, Yue Zhang, Heng Yu, Weihua Luo, Kun Wang, and Min Zhang. Code-switching for en- hancing NMT with pre-specified translation. In Proceed- ings of NAACL, pages 449-459, 2019.","title":"Code-switching for enhancing NMT with pre-specified translation","authors":"Kai Song, Yue Zhang, Heng Yu, Weihua Luo, Kun Wang, Min Zhang","year":"2019","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2019] Tao Wang, Shaohui Kuang, Deyi Xiong, and António Branco. Merging external bilingual pairs into neural machine translation. ArXiv, abs/1912.00567, 2019.","title":"Merging external bilingual pairs into neural machine translation","authors":"Tao Wang, Shaohui Kuang, Deyi Xiong, António Branco","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}