{"type":"ScholarlyArticle","title":"Towards Discourse-Aware Document-Level Neural Machine Translation","authors":[{"name":"Xin Tan","slug":"xin-tan","given":"Xin","family":"Tan","affiliation":"School of Computer Science and Technology , Soochow University , China"},{"name":"Longyin Zhang","slug":"longyin-zhang","given":"Longyin","family":"Zhang","affiliation":"School of Computer Science and Technology , Soochow University , China"},{"name":"Fang Kong","slug":"fang-kong","given":"Fang","family":"Kong","affiliation":""},{"name":"Guodong Zhou","slug":"guodong-zhou","given":"Guodong","family":"Zhou","affiliation":"School of Computer Science and Technology , Soochow University , China"}],"abstract":"Current document-level neural machine translation (NMT) systems have achieved remarkable progress with document context. Nevertheless, discourse information that has been proven effective in many NLP tasks is ignored in most previous work. In this work, we aim at incorporating the coherence information hidden within the RST-style discourse structure into machine translation. To achieve it, we propose a document-level NMT system enhanced with the discourse-aware document context, which is named Disco2NMT. Specifically, Disco2NMT models document context based on the discourse dependency structures through a hierarchical architecture. We first convert the RST tree of an article into a dependency structure and then build the graph convolutional network (GCN) upon the segmented EDUs under the guidance of RST dependencies to capture the discourse-aware context for NMT incorporation. We conduct experiments on the document-level English-German and English-Chinese translation tasks with three domains (TED, News, and Europarl). Experimental results show that our Disco2NMT model significantly surpasses both context-agnostic and context-aware baseline systems on multiple evaluation indicators.","doi":"10.24963/ijcai.2022/608","url":"https://ijcai.org/proceedings/2022/608","pdf":"https://ijcai.org/proceedings/2022/0608.pdf","markdown":"https://ijcai.org/proceedings/2022/608.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/608","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":608,"pages":{"first":4383,"last":4389},"keywords":["Natural Language Processing: Machine Translation and Multilinguality","Natural Language Processing: Language Generation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986789","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984932","length_sec":295,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Carlson et al., 2001] Lynn Carlson, Daniel Marcu, and Mary Ellen Okurovsky. Building a discourse-tagged cor- pus in the framework of rhetorical structure theory. In SIG- DIAL, 2001.","title":"Building a discourse-tagged corpus in the framework of rhetorical structure theory","authors":"Lynn Carlson, Daniel Marcu, Mary Ellen Okurovsky","year":"2001","venue":"SIG-DIAL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cettolo et al., 2012] Mauro Cettolo, Christian Girardi, and Marcello Federico. WIT3: Web inventory of transcribed and translated talks. In EAMT, 2012.","title":"WIT3: Web inventory of transcribed and translated talks","authors":"Mauro Cettolo, Christian Girardi, Marcello Federico","year":"2012","venue":"EAMT","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2020] Junxuan Chen, Xiang Li, Jiarui Zhang, et al. Modeling discourse structure for document-level neural machine translation. arXiv:2006.04721, 2020.","title":"Modeling discourse structure for document-level neural machine translation","authors":"Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Denkowski and Lavie, 2014] Michael Denkowski and Alon Lavie. Meteor universal: Language specific translation evaluation for any target language. In WMT, 2014.","title":"Meteor universal: Language specific translation evaluation for any target language","authors":"Lavie Denkowski, Michael Denkowski, Alon Lavie","year":"2014","venue":"WMT","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fernandes et al., 2021] Patrick Fernandes, Kayo Yin, Gra- ham Neubig, and André F. T. Martins. Measuring and in- creasing context usage in context-aware machine transla- tion. In ACL-IJCNLP, 2021.","title":"Measuring and increasing context usage in context-aware machine translation","authors":"Patrick Fernandes, Kayo Yin, Graham Neubig, F T André, Martins","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hirao et al., 2013] Tsutomu Hirao, Yasuhisa Yoshida, Masaaki Nishino, et al. Single-document summarization as a tree knapsack problem. In EMNLP, 2013.","title":"Single-document summarization as a tree knapsack problem","authors":"Tsutomu Hirao, Yasuhisa Yoshida, Masaaki Nishino","year":"2013","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hobbs, 1979] Jerry R Hobbs. Coherence and coreference. Cognitive science, 3(1), 1979.","title":"Coherence and coreference","authors":"Jerry R Hobbs","year":"1979","venue":"Cognitive science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kuang et al., 2018] Shaohui Kuang, Deyi Xiong, Weihua Luo, and Guodong Zhou. Modeling coherence for neu- ral machine translation with dynamic and topic caches. In COLING, 2018.","title":"Modeling coherence for neural machine translation with dynamic and topic caches","authors":"Shaohui Kuang, Deyi Xiong, Weihua Luo, Guodong Zhou","year":"2018","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lapata et al., 2005] Mirella Lapata, Regina Barzilay, et al. Automatic evaluation of text coherence: Models and rep- resentations. In IJCAI, volume 5, 2005.","title":"Automatic evaluation of text coherence: Models and representations","authors":"Mirella Lapata, Regina Barzilay","year":"2005","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2005/30"},{"position":11,"raw":"[Li et al., 2014a] Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. Text-level discourse dependency parsing. In ACL, 2014.","title":"Text-level discourse dependency parsing","authors":"Li","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2014b] Yancui Li, wenhe Feng, jing Sun, Fang Kong, and Guodong Zhou. Building chinese discourse corpus with connective-driven dependency tree structure. In EMNLP, 2014.","title":"Building chinese discourse corpus with connective-driven dependency tree structure","authors":"Li","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2019] Xiang Lin, Shafiq Joty, Prathyusha Jwala- puram, and M Saiful Bari. A unified linear-time frame- work for sentence-level discourse parsing. In ACL, 2019.","title":"A unified linear-time framework for sentence-level discourse parsing","authors":"Xiang Lin, Shafiq Joty, Prathyusha Jwalapuram, M Saiful, Bari","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ma et al., 2020] Shuming Ma, Dongdong Zhang, and Ming Zhou. A simple and effective unified encoder for document-level machine translation. In ACL, 2020.","title":"A simple and effective unified encoder for document-level machine translation","authors":"Shuming Ma, Dongdong Zhang, Ming Zhou","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mann and Thompson, 1988] William C Mann and Sandra A Thompson. Rhetorical structure theory: Toward a func- tional theory of text organization. Text-interdisciplinary Journal for the Study of Discourse, 8(3), 1988.","title":"Rhetorical structure theory: Toward a functional theory of text organization","authors":"C William, Sandra A Mann, Thompson","year":"1988","venue":"Text-interdisciplinary Journal for the Study of Discourse","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Maruf and Haffari, 2018] Sameen Maruf and Gholamreza Haffari. Document context neural machine translation with memory networks. In ACL, 2018.","title":"Document context neural machine translation with memory networks","authors":"Haffari Maruf, Sameen Maruf, Gholamreza Haffari","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Maruf et al., 2019] Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. Selective attention for context-aware neural machine translation. In NAACL, 2019.","title":"Selective attention for context-aware neural machine translation","authors":"Sameen Maruf, F T André, Gholamreza Martins, Haffari","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Miculicich et al., 2018] Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. Document-level neural machine translation with hierarchical attention net- works. In EMNLP, 2018.","title":"Document-level neural machine translation with hierarchical attention networks","authors":"Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, James Henderson","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Miculicich Werlen and Popescu-Belis, 2017] Lesly Miculi- cich Werlen and Andrei Popescu-Belis. Validation of an automatic metric for the accuracy of pronoun translation (APT). In DiscoMT, 2017.","title":"Lesly Miculicich Werlen and Andrei Popescu-Belis. Validation of an automatic metric for the accuracy of pronoun translation (APT)","authors":"Miculicich Werlen, Popescu-Belis","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In ACL, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Snover et al., 2006] Matthew Snover, Bonnie Dorr, Rich Schwartz, Linnea Micciulla, and John Makhoul. A study of translation edit rate with targeted human annotation. In AMTA, 2006.","title":"A study of translation edit rate with targeted human annotation","authors":"Matthew Snover, Bonnie Dorr, Rich Schwartz, Linnea Micciulla, John Makhoul","year":"2006","venue":"AMTA","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Stede and Neumann, 2014] Manfred Stede and Arne Neu- mann. Potsdam commentary corpus 2.0: Annotation for discourse research. In LREC, 2014.","title":"Potsdam commentary corpus 2.0: Annotation for discourse research","authors":"Neumann Stede","year":"2014","venue":"LREC","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sugiyama and Yoshinaga, 2021] Amane Sugiyama and Naoki Yoshinaga. Context-aware decoder for neural machine translation using a target-side document-level language model. In NAACL, 2021.","title":"Context-aware decoder for neural machine translation using a target-side document-level language model","authors":"Yoshinaga Sugiyama, Amane Sugiyama, Naoki Yoshinaga","year":"2021","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tan et al., 2019] Xin Tan, Longyin Zhang, Deyi Xiong, and Guodong Zhou. Hierarchical modeling of global con- text for document-level neural machine translation. In EMNLP, 2019.","title":"Hierarchical modeling of global context for document-level neural machine translation","authors":"Xin Tan, Longyin Zhang, Deyi Xiong, Guodong Zhou","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tan et al., 2021] Xin Tan, Longyin Zhang, and Guodong Zhou. Coupling context modeling with zero pronoun re- covering for document-level natural language generation. In EMNLP, 2021.","title":"Coupling context modeling with zero pronoun recovering for document-level natural language generation","authors":"Xin Tan, Longyin Zhang, Guodong Zhou","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tu et al., 2018] Zhaopeng Tu, Yang Liu, Shuming Shi, and Tong Zhang. Learning to remember translation history with a continuous cache. TACL, 6, 2018.","title":"Learning to remember translation history with a continuous cache","authors":"Tu","year":"2018","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, et al. Attention is all you need. In NIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Voita et al., 2018] Elena Voita, Pavel Serdyukov, Rico Sen- nrich, and Ivan Titov. Context-aware neural machine trans- lation learns anaphora resolution. In ACL, 2018.","title":"Context-aware neural machine translation learns anaphora resolution","authors":"Elena Voita, Pavel Serdyukov, Rico Sennrich, Ivan Titov","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2017] Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. Exploiting cross-sentence context for neural machine translation. In EMNLP, 2017.","title":"Exploiting cross-sentence context for neural machine translation","authors":"Wang","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2021] Hongfei Xu, Deyi Xiong, Josef Van Gen- abith, and Qiuhui Liu. Efficient context-aware neural machine translation with layer-wise weighting and input- aware gating. In IJCAI, 2021.","title":"Efficient context-aware neural machine translation with layer-wise weighting and inputaware gating","authors":"Hongfei Xu, Deyi Xiong, Josef Van Genabith, Qiuhui Liu","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2018] Jiacheng Zhang, Huanbo Luan, Maosong Sun, et al. Improving the transformer translation model with document-level context. In EMNLP, 2018.","title":"Improving the transformer translation model with document-level context","authors":"Zhang","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2021] Longyin Zhang, Fang Kong, and Guodong Zhou. Adversarial learning for discourse rhetor- ical structure parsing. In ACL-IJCNLP, 2021.","title":"Adversarial learning for discourse rhetorical structure parsing","authors":"Zhang","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}