{"type":"ScholarlyArticle","title":"Phrase Table as Recommendation Memory for Neural Machine Translation","authors":[{"name":"Yang Zhao","slug":"yang-zhao","given":"Yang","family":"Zhao","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , CAS , Beijing , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Yining Wang","slug":"yining-wang","given":"Yining","family":"Wang","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , CAS , Beijing , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Jiajun Zhang","slug":"jiajun-zhang","given":"Jiajun","family":"Zhang","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , CAS , Beijing , China; University of Chinese Academy of Sciences , Beijing , China; Beijing Advanced Innovation Center for Language Resources , Beijing , China"},{"name":"Chengqing Zong","slug":"chengqing-zong","given":"Chengqing","family":"Zong","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , CAS , Beijing , China; University of Chinese Academy of Sciences , Beijing , China; CAS Center for Excellence in Brain Science and Intelligence Technology , Beijing , China"}],"abstract":"Neural Machine Translation (NMT) has drawn much attention due to its promising translation performance recently. However, several studies indicate that NMT often generates fluent but unfaithful translations. In this paper, we propose a method to alleviate this problem by using a phrase table as recommendation memory. The main idea is to add bonus to words worthy of recommendation, so that NMT can make correct predictions. Specifically, we first derive a prefix tree to accommodate all the candidate target phrases by searching the phrase translation table according to the source sentence.Then, we construct a recommendation word set by matching between candidate target phrases and previously translated target words by NMT. After that, we determine the specific bonus value for each recommendable word by using the attention vector and phrase translation probability. Finally,we integrate this bonus value into NMT to improve the translation results. The extensive experiments demonstrate that the proposed methods obtain remarkable improvements over the strong attention based NMT.","doi":"10.24963/ijcai.2018/641","url":"https://ijcai.org/proceedings/2018/641","pdf":"https://ijcai.org/proceedings/2018/0641.pdf","markdown":"https://ijcai.org/proceedings/2018/641.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/641","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":641,"pages":{"first":4609,"last":4615},"keywords":["Natural Language Processing: Machine Translation","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Arthur et al., 2016] Philip Arthur, Graham Neubig, and Satoshi Nakamura. Incorporating discrete translation lex- icons into neural machine translation. In Proceedings of EMNLP, 2016.","title":"Incorporating discrete translation lexicons into neural machine translation","authors":"Philip Arthur, Graham Neubig, Satoshi Nakamura","year":"2016","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dahlmann et al., 2017] Leonard Dahlmann, Evgeny Ma- tusov, Pavel Petrushkov, and Shahram Khadivi. Neural machine translation leveraging phrase-based models in a hybrid search. In proceedings of EMNLP, 2017.","title":"Neural machine translation leveraging phrase-based models in a hybrid search","authors":"Leonard Dahlmann, Evgeny Matusov, Pavel Petrushkov, Shahram Khadivi","year":"2017","venue":"proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Feng et al., 2017] Yang Feng, Shiyue Zhang, Andi Zhang, Dong Wang, and Andrew Abel. Memory-augmented neu- ral machine translation. In proceedings of EMNLP, 2017.","title":"Memory-augmented neural machine translation","authors":"Yang Feng, Shiyue Zhang, Andi Zhang, Dong Wang, Andrew Abel","year":"2017","venue":"proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gehring et al., 2017] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. Convo- lutional sequence to sequence learning. arXiv preprint arXiv:1601.03317, 2017.","title":"Convolutional sequence to sequence learning","authors":"Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N Dauphin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Wei He, Zhongjun He, Hua Wu, and Haifeng Wang. Improved neural machine translation with smt features. In Proceedings of AAAI, 2016.","title":"Improved neural machine translation with smt features","authors":"Wei He, Zhongjun He, Hua Wu, Haifeng Wang","year":"2016","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Junczys-Dowmunt et al., 2016] Marcin Junczys-Dowmunt, Tomasz Dwojak, and Hieu Hoang. Is neural machine translation ready for deployment? a case study on 30 trans- lation directions. arXiv preprint arXiv:1610.01108, 2016.","title":"Is neural machine translation ready for deployment? a case study on 30 translation directions","authors":"Marcin Junczys-Dowmunt, Tomasz Dwojak, Hieu Hoang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kalchbrenner and Blunsom, 2013] Nal Kalchbrenner and Phil Blunsom. Recurrent continuous translation models. In Proceedings of EMNLP, 2013.","title":"Recurrent continuous translation models","authors":"Nal Kalchbrenner, Phil Blunsom","year":"2013","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Koehn et al., 2007] Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. Moses: Open source toolkit for statis- tical machine translation. In Proceedings of ACL, 2007.","title":"Moses: Open source toolkit for statistical machine translation","authors":"Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens","year":"2007","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2016] Xiaoqing Li, Jiajun Zhang, and Chengqing Zong. Towards zero unknown word in neural machine translation. In Proceedings of IJCAI, 2016.","title":"Towards zero unknown word in neural machine translation","authors":"Xiaoqing Li, Jiajun Zhang, Chengqing Zong","year":"2016","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/405"},{"position":12,"raw":"[Liu et al., 2016] Lemao Liu, Masao Utiyama, Andrew Finch, and Eiichiro Sumita. Neural machine translation with supervised attention. In Proceedings of COLING, 2016.","title":"Neural machine translation with supervised attention","authors":"Liu","year":"2016","venue":"Proceedings of COLING","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Luong et al., 2015] Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention- based neural machine translation. In Proceedings of EMNLP, 2015.","title":"Effective approaches to attentionbased neural machine translation","authors":"Minh-Thang Luong, Hieu Pham, Christopher D Manning","year":"2015","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mi et al., 2016a] Haitao Mi, Baskaran Sankaran, Zhiguo Wang, and Abe Ittycheriah. Coverage embedding model for neural machine translation. In Proceedings of EMNLP, 2016.","title":"Coverage embedding model for neural machine translation","authors":"Mi","year":"2016","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mi et al., 2016b] Haitao Mi, Zhiguo Wang, and Abe Itty- cheriah. Supervised attentions for neural machine transla- tion. In Proceedings of EMNLP, 2016.","title":"Supervised attentions for neural machine translation","authors":"Mi","year":"2016","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In Proceedings of ACL, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Stahlberg et al., 2016] Felix Stahlberg, Eva Hasler, Aure- lien Waite, and Bill Byrne. Syntactically guided neural machine translation. In Proceedings of ACL, 2016.","title":"Syntactically guided neural machine translation","authors":"Felix Stahlberg, Eva Hasler, Aurelien Waite, Bill Byrne","year":"2016","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Proceedings of NIPS, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tang et al., 2016] Yaohua Tang, Fandong Meng, Zheng- dong Lu, Hang Li, and Philip LH Yu. Neural machine translation with external phrase memory. arXiv preprint arXiv:1606.01792, 2016.","title":"Neural machine translation with external phrase memory","authors":"Yaohua Tang, Fandong Meng, Zhengdong Lu, Hang Li, Philip Lh Yu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tu et al., 2016] Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. Coverage-based neural ma- chine translation. In Proceedings of ACL, 2016.","title":"Coverage-based neural machine translation","authors":"Tu","year":"2016","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tu et al., 2017] Zhaopeng Tu, Yang Liu, Lifeng Shang, Xi- aohua Liu, and Hang Li. Neural machine translation with reconstruction. In proceedings of AAAI, 2017.","title":"Neural machine translation with reconstruction","authors":"Tu","year":"2017","venue":"proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, and ukasz Kaiser. Attention is all you need. arXiv preprint arXiv:1601.03317, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Kaiser","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2017a] Xing Wang, Zhengdong Lu, Zhaopeng Tu, Hang Li, Deyi Xiong, and Min Zhang. Neural machine translation advised by statistical machine translation. In proceedings of AAAI, 2017.","title":"Neural machine translation advised by statistical machine translation","authors":"Wang","year":"2017","venue":"proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017b] Xing Wang, Zhaopeng Tu, Deyi Xiong, and Min Zhang. Translating phrases in neural machine translation. In proceedings of EMNLP, 2017.","title":"Translating phrases in neural machine translation","authors":"Wang","year":"2017","venue":"proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2017c] Yining Wang, Yang Zhao, Jiajun Zhang, Chengqing Zong, and Zhengshan Xue. Towards neural machine translation with partially aligned corpora. In proceedings of IJCNLP, 2017.","title":"Towards neural machine translation with partially aligned corpora","authors":"Wang","year":"2017","venue":"proceedings of IJCNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang and Zong, 2015] Jiajun Zhang and Chengqing Zong. Deep neural networks in machine translation: An overview. IEEE Intelligent Systems, 30(5):16-25, 2015.","title":"Deep neural networks in machine translation: An overview","authors":"Jiajun Zhang, Chengqing Zong","year":"2015","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang and Zong, 2016] Jiajun Zhang and Chengqing Zong. Bridging neural machine translation and bilingual dictio- naries. arXiv preprint arXiv:1610.07272, 2016.","title":"Bridging neural machine translation and bilingual dictionaries","authors":"Jiajun Zhang, Chengqing Zong","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2017] Long Zhou, Wenpeng Hu, Jiajun Zhang, and Chengqing Zong. Neural system combination for ma- chine translation. In Proceedings of ACL, 2017.","title":"Neural system combination for machine translation","authors":"Long Zhou, Wenpeng Hu, Jiajun Zhang, Chengqing Zong","year":"2017","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zoph and Knight, 2016] Barret Zoph and Kevin Knight. Multi-source neural translation. In Proceedings of NAACL, 2016.","title":"Multi-source neural translation","authors":"Knight Zoph, Barret Zoph, Kevin Knight","year":"2016","venue":"Proceedings of NAACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}