{"type":"ScholarlyArticle","title":"Modeling Voting for System Combination in Machine Translation","authors":[{"name":"Xuancheng Huang","slug":"xuancheng-huang","given":"Xuancheng","family":"Huang","affiliation":"Dept. of Comp. Sci. & Tech. , BNRist Center , Institute for AI , Tsinghua University"},{"name":"Jiacheng Zhang","slug":"jiacheng-zhang","given":"Jiacheng","family":"Zhang","affiliation":"Dept. of Comp. Sci. & Tech. , BNRist Center , Institute for AI , Tsinghua University"},{"name":"Zhixing Tan","slug":"zhixing-tan","given":"Zhixing","family":"Tan","affiliation":"Dept. of Comp. Sci. & Tech. , BNRist Center , Institute for AI , Tsinghua University"},{"name":"Derek F. Wong","slug":"derek-f-wong","given":"Derek F.","family":"Wong","affiliation":"CT Lab / Department of Computer and Information Science , University of Macau"},{"name":"Huanbo Luan","slug":"huanbo-luan","given":"Huanbo","family":"Luan","affiliation":"Dept. of Comp. Sci. & Tech. , BNRist Center , Institute for AI , Tsinghua University"},{"name":"Jingfang Xu","slug":"jingfang-xu","given":"Jingfang","family":"Xu","affiliation":"Sogou Inc."},{"name":"Maosong Sun","slug":"maosong-sun","given":"Maosong","family":"Sun","affiliation":"Dept. of Comp. Sci. & Tech. , BNRist Center , Institute for AI , Tsinghua University"},{"name":"Yang Liu","slug":"yang-liu","given":"Yang","family":"Liu","affiliation":"Dept. of Comp. Sci. & Tech. , BNRist Center , Institute for AI , Tsinghua University; Beijing Advanced Innovation Center for Language Resources; Beijing Academy of Artificial Intelligence"}],"abstract":"System combination is an important technique for combining the hypotheses of different machine translation systems to improve translation performance. Although early statistical approaches to system combination have been proven effective in analyzing the consensus between hypotheses, they suffer from the error propagation problem due to the use of pipelines. While this problem has been alleviated by end-to-end training of multi-source sequence-to-sequence models recently, these neural models do not explicitly analyze the relations between hypotheses and fail to capture their agreement because the attention to a word in a hypothesis is calculated independently, ignoring the fact that the word might occur in multiple hypotheses. In this work, we propose an approach to modeling voting for system combination in machine translation. The basic idea is to enable words in hypotheses from different systems to vote on words that are representative and should get involved in the generation process. This can be done by quantifying the influence of each voter and its preference for each candidate. Our approach combines the advantages of statistical and neural methods since it can not only analyze the relations between hypotheses but also allow for end-to-end training. Experiments show that our approach is capable of better taking advantage of the consensus between hypotheses and achieves significant improvements over state-of-the-art baselines on Chinese-English and English-German machine translation tasks.","doi":"10.24963/ijcai.2020/511","url":"https://ijcai.org/proceedings/2020/511","pdf":"https://ijcai.org/proceedings/2020/0511.pdf","markdown":"https://ijcai.org/proceedings/2020/511.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/511","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":511,"pages":{"first":3694,"last":3701},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Machine Translation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25077/file_25077.m3u8","length_sec":281,"provider":"hls"}],"references":[{"position":1,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015. [Bangalore et al., 2001] Bangalore Bangalore, Germán Bor- del, and Giuseppe Riccardi. Computing consensus trans- lation from multiple machine translation systems. ASRU, 2001.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2001","venue":"Computing consensus translation from multiple machine translation systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barrault et al., 2019] Loïc Barrault, Ondřej Bojar, Marta R Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, et al. Findings of the 2019 conference on ma- chine translation (wmt19). In WMT, 2019.","title":"Findings of the 2019 conference on machine translation (wmt19)","authors":"Loïc Barrault, Ondřej Bojar, Marta R Costa-Jussà, Christian Federmann, Mark Fishel, Yvette Graham","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brown et al., 1993] Peter E. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. The mathematics of statistical machine transla- tion: Parameter estimation. CL, 1993.","title":"The mathematics of statistical machine translation: Parameter estimation","authors":"Peter E Brown, Stephen A Della Pietra, Vincent J Della Pietra, Robert L Mercer","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2009] Boxing Chen, Min Zhang, Haizhou Li, and AiTi Aw. A comparative study of hypothesis align- ment and its improvement for machine translation system combination. In ACL/IJCNLP, 2009.","title":"A comparative study of hypothesis alignment and its improvement for machine translation system combination","authors":"Chen","year":"2009","venue":"ACL/IJCNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Du and Way, 2010] Jinhua Du and Andy Way. Using terp to augment the system combination for smt. 2010.","title":"Using terp to augment the system combination for smt","authors":"Way ; Du, Jinhua Du, Andy Way","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feng et al., 2009] Yang Feng, Yang Liu, Haitao Mi, Qun Liu, and Yajuan Lü. Lattice-based system combination for statistical machine translation. In EMNLP, 2009.","title":"Lattice-based system combination for statistical machine translation","authors":"Yang Feng, Yang Liu, Haitao Mi, Qun Liu, Yajuan Lü","year":"2009","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Freitag et al., 2014] Markus Freitag, Matthias Huck, and Hermann Ney. Jane: Open source machine translation sys- tem combination. In EACL, 2014.","title":"Jane: Open source machine translation system combination","authors":"Markus Freitag, Matthias Huck, Hermann Ney","year":"2014","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Freitag et al., 2015] Markus Freitag, Jan-Thorsten Peter, Stephan Peitz, Minwei Feng, and Hermann Ney. Local system voting feature for machine translation system com- bination. In WMT@EMNLP, 2015.","title":"Local system voting feature for machine translation system combination","authors":"Freitag","year":"2015","venue":"WMT@EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ghazvininejad et al., 2019] Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. Mask-predict: Parallel decoding of conditional masked language models. In EMNLP/IJCNLP, 2019.","title":"Mask-predict: Parallel decoding of conditional masked language models","authors":"Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer","year":"2019","venue":"EMNLP/IJCNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2008] Xiaodong He, Mei Yang, Jianfeng Gao, Patrick Nguyen, and Robert Moore. Indirect-HMM-based hypothesis alignment for combining outputs from machine translation systems. In EMNLP, 2008.","title":"Indirect-HMM-based hypothesis alignment for combining outputs from machine translation systems","authors":"Xiaodong He, Mei Yang, Jianfeng Gao, Patrick Nguyen, Robert Moore","year":"2008","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Heafield and Lavie, 2010] Kenneth Heafield and Alon Lavie. Combining machine translation output with open source: The carnegie mellon multi-engine machine translation scheme. In PBML, 2010.","title":"Combining machine translation output with open source: The carnegie mellon multi-engine machine translation scheme","authors":"Lavie Heafield, Kenneth Heafield, Alon Lavie","year":"2010","venue":"PBML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang and Papineni, 2007] Fei Huang and Kishore Pap- ineni. Hierarchical system combination for machine trans- lation. In EMNLP-CoNLL, 2007.","title":"Hierarchical system combination for machine translation","authors":"Fei Huang, Kishore Papineni","year":"2007","venue":"EMNLP-CoNLL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Karakos et al., 2008] Damianos G. Karakos, Jason Eisner, Sanjeev Khudanpur, and Markus Dreyer. Machine trans- lation system combination using itg-based alignments. In ACL, 2008.","title":"Machine translation system combination using itg-based alignments","authors":"G Damianos, Jason Karakos, Sanjeev Eisner, Markus Khudanpur, Dreyer","year":"2008","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Koehn et al., 2003] Philipp Koehn, Franz J. Och, and Daniel Marcu. Statistical phrase-based translation. In NAACL, 2003.","title":"Statistical phrase-based translation","authors":"Koehn","year":"2003","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Koehn, 2004] Philipp Koehn. Statistical significance tests for machine translation evaluation. In EMNLP, 2004.","title":"Statistical significance tests for machine translation evaluation","authors":"Philipp Koehn","year":"2004","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li and Zong, 2008] Maoxi Li and Chengqing Zong. Word reordering alignment for combination of statistical ma- chine translation systems. ISCSLP, 2008.","title":"Maoxi Li and Chengqing Zong. Word reordering alignment for combination of statistical machine translation systems","authors":"Zong Li","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ma and McKeown, 2012] Wei-Yun Ma and Kathleen McK- eown. Phrase-level system combination for machine trans- lation based on target-to-target decoding. 2012.","title":"Phrase-level system combination for machine translation based on target-to-target decoding","authors":"Wei-Yun Ma, Kathleen Mckeown","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ma and McKeown, 2015] Wei-Yun Ma and Kathleen McK- eown. System combination for machine translation through paraphrasing. In EMNLP, 2015.","title":"System combination for machine translation through paraphrasing","authors":"Wei-Yun Ma, Kathleen Mckeown","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Matusov et al., 2006] Evgeny Matusov, Nicola Ueffing, and Hermann Ney. Computing consensus translation for mul- tiple machine translation systems using enhanced hypoth- esis alignment. In EACL, 2006.","title":"Computing consensus translation for multiple machine translation systems using enhanced hypothesis alignment","authors":"Evgeny Matusov, Nicola Ueffing, Hermann Ney","year":"2006","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ott et al., 2018] Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. Scaling neural machine translation. In WMT, 2018.","title":"Scaling neural machine translation","authors":"Myle Ott, Sergey Edunov, David Grangier, Michael Auli","year":"2018","venue":"WMT","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rosti et al., 2007a] Antti-Veikko I. Rosti, Necip Fazil Ayan, Bing Xiang, Spyridon Matsoukas, Richard M. Schwartz, and Bonnie J. Dorr. Combining outputs from multiple ma- chine translation systems. In HLT-NAACL, 2007.","title":"Combining outputs from multiple machine translation systems","authors":"Rosti","year":"2007","venue":"HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rosti et al., 2007b] Antti-Veikko I. Rosti, Spyridon Mat- soukas, and Richard M. Schwartz. Improved word-level system combination for machine translation. In ACL, 2007.","title":"Improved word-level system combination for machine translation","authors":"Rosti","year":"2007","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sennrich et al., 2015] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In ACL, 2015.","title":"Neural machine translation of rare words with subword units","authors":"Sennrich","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tu et al., 2016] Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. Modeling coverage for neu- ral machine translation. In ACL, 2016.","title":"Modeling coverage for neural machine translation","authors":"Tu","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wu et al., 2019] Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, and Michael Auli. Pay less attention with lightweight and dynamic convolutions. ICLR, 2019.","title":"Pay less attention with lightweight and dynamic convolutions","authors":"Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, Michael Auli","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xiao et al., 2013] Tong Xiao, Jingbo Zhu, and T Liu. Bag- ging and boosting statistical machine translation systems. AI, 2013.","title":"Bagging and boosting statistical machine translation systems","authors":"Tong Xiao, Jingbo Zhu, Liu","year":"2013","venue":"AI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2017] Long Zhou, Wenpeng Hu, Jiajun Zhang, and Chengqing Zong. Neural system combination for ma- chine translation. In ACL, 2017.","title":"Neural system combination for machine translation","authors":"Long Zhou, Wenpeng Hu, Jiajun Zhang, Chengqing Zong","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhu et al., 2016] Junguo Zhu, Muyun Yang, Sheng Li, and Tiejun Zhao. Sentence-level paraphrasing for machine translation system combination. In ICYCSEE, 2016.","title":"Sentence-level paraphrasing for machine translation system combination","authors":"Zhu","year":"2016","venue":"ICYCSEE","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zoph and Knight, 2016] Barret Zoph and Kevin Knight. Multi-source neural translation. In HLT-NAACL, 2016.","title":"Multi-source neural translation","authors":"Knight Zoph, Barret Zoph, Kevin Knight","year":"2016","venue":"HLT-NAACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}