{"type":"ScholarlyArticle","title":"Effective Graph Context Representation for Document-level Machine Translation","authors":[{"name":"Kehai Chen","slug":"kehai-chen","given":"Kehai","family":"Chen","affiliation":"Harbin Institute of Technology , Shenzhen , China"},{"name":"Muyun Yang","slug":"muyun-yang","given":"Muyun","family":"Yang","affiliation":"Harbin Institute of Technology , Harbin , China"},{"name":"Masao Utiyama","slug":"masao-utiyama","given":"Masao","family":"Utiyama","affiliation":"National Institute of Information and Communications Technology , Kyoto , Japan"},{"name":"Eiichiro Sumita","slug":"eiichiro-sumita","given":"Eiichiro","family":"Sumita","affiliation":"National Institute of Information and Communications Technology , Kyoto , Japan"},{"name":"Rui Wang","slug":"rui-wang","given":"Rui","family":"Wang","affiliation":"Shanghai Jiao Tong University , Shanghai , China"},{"name":"Min Zhang","slug":"min-zhang","given":"Min","family":"Zhang","affiliation":"Harbin Institute of Technology , Shenzhen , China"}],"abstract":"Document-level neural machine translation (DocNMT) universally encodes several local sentences or the entire document. Thus, DocNMT does not consider the relevance of document-level contextual information, for example, some context (i.e., content words, logical order, and co-occurrence relation) is more effective than another auxiliary context  (i.e., functional and auxiliary words). To address this issue, we first utilize the word frequency information to recognize content words in the input document, and then use heuristical relations to summarize content words and sentences as a graph structure without relying on external syntactic knowledge. Furthermore, we apply graph attention networks to this graph structure to learn its feature representation, which allows DocNMT to more effectively capture the document-level context. Experimental results on several widely-used document-level benchmarks demonstrated the effectiveness of the proposed approach.","doi":"10.24963/ijcai.2022/566","url":"https://ijcai.org/proceedings/2022/566","pdf":"https://ijcai.org/proceedings/2022/0566.pdf","markdown":"https://ijcai.org/proceedings/2022/566.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/566","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":566,"pages":{"first":4079,"last":4085},"keywords":["Natural Language Processing: Machine Translation and Multilinguality","Natural Language Processing: Language Generation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986675","length_sec":92,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984818","length_sec":348,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bao et al., 2021] Guangsheng Bao, Yue Zhang, Zhiyang Teng, Boxing Chen, and Weihua Luo. G-transformer for document-level machine translation. In Proc. of ACL, 2021.","title":"G-transformer for document-level machine translation","authors":"Guangsheng Bao, Yue Zhang, Zhiyang Teng, Boxing Chen, Weihua Luo","year":"2021","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bawden et al., 2018] Rachel Bawden, Rico Sennrich, Alexandra Birch, and Barry Haddow. Evaluating discourse phenomena in neural machine translation. In Proc. of NAACL, 2018.","title":"Evaluating discourse phenomena in neural machine translation","authors":"Rachel Bawden, Rico Sennrich, Alexandra Birch, Barry Haddow","year":"2018","venue":"Proc. of NAACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ben et al., 2013] Guosheng Ben, Deyi Xiong, Zhiyang Teng, Yajuan Lü, and Qun Liu. Bilingual lexical cohesion trigger model for document-level machine translation. In Proc. of ACL, 2013.","title":"Bilingual lexical cohesion trigger model for document-level machine translation","authors":"Guosheng Ben, Deyi Xiong, Zhiyang Teng, Yajuan Lü, Qun Liu","year":"2013","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Collins et al., 2005] Michael Collins, Philipp Koehn, and Ivona Kučerová. Clause restructuring for statistical machine translation. In Proc. of ACL, 2005.","title":"Clause restructuring for statistical machine translation","authors":"Michael Collins, Philipp Koehn, Ivona Kučerová","year":"2005","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Foltz et al., 1998] Peter W. Foltz, Walter Kintsch, and Thomas K Landauer. The measurement of textual coherence with latent semantic analysis. Discourse Processes, 1998.","title":"The measurement of textual coherence with latent semantic analysis","authors":"Peter W Foltz, Walter Kintsch, Thomas K Landauer","year":"1998","venue":"Discourse Processes","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guo et al., 2019] Zhijiang Guo, Yan Zhang, and Wei Lu. Attention guided graph convolutional networks for relation extraction. In Proc. of ACL, 2019.","title":"Attention guided graph convolutional networks for relation extraction","authors":"Zhijiang Guo, Yan Zhang, Wei Lu","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kang et al., 2020] Xiaomian Kang, Yang Zhao, Jiajun Zhang, and Chengqing Zong. Dynamic context selection for document-level neural machine translation via reinforcement learning. In Proc. of EMNLP, 2020.","title":"Dynamic context selection for document-level neural machine translation via reinforcement learning","authors":"Xiaomian Kang, Yang Zhao, Jiajun Zhang, Chengqing Zong","year":"2020","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kuang et al., 2018] Shaohui Kuang, Deyi Xiong, Weihua Luo, and Guodong Zhou. Modeling coherence for neural machine translation with dynamic and topic caches. In Proc. of COLING, 2018.","title":"Modeling coherence for neural machine translation with dynamic and topic caches","authors":"Kuang","year":"2018","venue":"Proc. of COLING","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lin et al., 2015] Rui Lin, Shujie Liu, Muyun Yang, Mu Li, Ming Zhou, and Sheng Li. Hierarchical recurrent neural network for document modeling. In Proc. of EMNLP, 2015.","title":"Hierarchical recurrent neural network for document modeling","authors":"Rui Lin, Shujie Liu, Muyun Yang, Mu Li, Ming Zhou, Sheng Li","year":"2015","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ma et al., 2020] Shuming Ma, Dongdong Zhang, and Ming Zhou. A simple and effective unified encoder for document- level machine translation. In Proc. of ACL, 2020.","title":"A simple and effective unified encoder for documentlevel machine translation","authors":"Shuming Ma, Dongdong Zhang, Ming Zhou","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Maruf and Haffari, 2018] Sameen Maruf and Gholamreza Haffari. Document context neural machine translation with memory networks. In Proc. of ACL, 2018.","title":"Document context neural machine translation with memory networks","authors":"Haffari Maruf, Sameen Maruf, Gholamreza Haffari","year":"2018","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Maruf et al., 2019] Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. Selective attention for context-aware neural machine translation. In Proc. of NAACL, 2019.","title":"Selective attention for context-aware neural machine translation","authors":"Sameen Maruf, F T André, Gholamreza Martins, Haffari","year":"2019","venue":"Proc. of NAACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Miculicich et al., 2018] Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. Document- level neural machine translation with hierarchical attention networks. In Proc. of EMNLP, 2018.","title":"Documentlevel neural machine translation with hierarchical attention networks","authors":"Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, James Henderson","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Miculicich Werlen and Popescu-Belis, 2017] Lesly Miculi- cich Werlen and Andrei Popescu-Belis. Validation of an automatic metric for the accuracy of pronoun translation. In Workshop, 2017.","title":"Lesly Miculicich Werlen and Andrei Popescu-Belis. Validation of an automatic metric for the accuracy of pronoun translation","authors":"Miculicich Werlen, Popescu-Belis","year":"2017","venue":"Workshop","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sennrich et al., 2016] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proc. of ACL, 2016.","title":"Neural machine translation of rare words with subword units","authors":"Sennrich","year":"2016","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tan et al., 2019] Xin Tan, Longyin Zhang, Deyi Xiong, and Guodong Zhou. Hierarchical modeling of global context for document-level neural machine translation. In Proc. of EMNLP, 2019.","title":"Hierarchical modeling of global context for document-level neural machine translation","authors":"Xin Tan, Longyin Zhang, Deyi Xiong, Guodong Zhou","year":"2019","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tiedemann and Scherrer, 2017] Jörg Tiedemann and Yves Scherrer. Neural machine translation with extended context. In Workshop, 2017.","title":"Neural machine translation with extended context","authors":"Scherrer Tiedemann, Jörg Tiedemann, Yves Scherrer","year":"2017","venue":"Workshop","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tu et al., 2018] Zhaopeng Tu, Yang Liu, Shuming Shi, and Tong Zhang. Learning to remember translation history with a continuous cache. TACL, 2018.","title":"Learning to remember translation history with a continuous cache","authors":"Tu","year":"2018","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proc. of NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł Ukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Voita et al., 2018] Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. Context-aware neural machine translation learns anaphora resolution. In Proc. of ACL, 2018.","title":"Context-aware neural machine translation learns anaphora resolution","authors":"Elena Voita, Pavel Serdyukov, Rico Sennrich, Ivan Titov","year":"2018","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Voita et al., 2019] Elena Voita, Rico Sennrich, and Ivan Titov. When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion. In Proc. of ACL, 2019.","title":"When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion","authors":"Elena Voita, Rico Sennrich, Ivan Titov","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wong and Kit, 2012] Billy T. M. Wong and Chunyu Kit. Extending machine translation evaluation metrics with lexical cohesion to document level. In Proc. of EMNLP, 2012.","title":"Extending machine translation evaluation metrics with lexical cohesion to document level","authors":"T M Billy, Chunyu Wong, Kit","year":"2012","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wong et al., 2020] KayYen Wong, Sameen Maruf, and Gholamreza Haffari. Contextual neural machine translation improves translation of cataphoric pronouns. In Proc. of ACL, 2020.","title":"Contextual neural machine translation improves translation of cataphoric pronouns","authors":"Wong","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xiong and Zhang, 2013] Deyi Xiong and Min Zhang. A topic-based coherence model for statistical machine translation. In Proc. of AAAI, 2013.","title":"A topic-based coherence model for statistical machine translation","authors":"Deyi Xiong, Min Zhang","year":"2013","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xiong et al., 2013a] Deyi Xiong, Guosheng Ben, Min Zhang, Yajuan Lü, and Qun Liu. Modeling lexical cohesion for document-level machine translation. In Proc. of IJCAI, 2013.","title":"Modeling lexical cohesion for document-level machine translation","authors":"Xiong","year":"2013","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xiong et al., 2013b] Deyi Xiong, Yang Ding, Min Zhang, and Chew Lim Tan. Lexical chain based cohesion models for document-level statistical machine translation. In Proc. of EMNLP, 2013.","title":"Lexical chain based cohesion models for document-level statistical machine translation","authors":"Xiong","year":"2013","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2021a] Mingzhou Xu, Liangyou Li, Derek F. Wong, Qun Liu, and Lidia S. Chao. Document graph for neural machine translation. In Proc. of EMNLP, 2021.","title":"Document graph for neural machine translation","authors":"Xu","year":"2021","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu et al., 2021b] Wang Xu, Kehai Chen, and Tiejun Zhao. Document-level relation extraction with reconstruction. In Proc. of AAAI, 2021.","title":"Document-level relation extraction with reconstruction","authors":"Xu","year":"2021","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang et al., 2019] Zhengxin Yang, Jinchao Zhang, Fandong Meng, Shuhao Gu, Yang Feng, and Jie Zhou. Enhancing context modeling with a query-guided capsule network for document-level translation. In Proc. of EMNLP, 2019.","title":"Enhancing context modeling with a query-guided capsule network for document-level translation","authors":"Yang","year":"2019","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2018] Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. Improving the transformer translation model with document-level context. In Proc. of EMNLP, 2018.","title":"Improving the transformer translation model with document-level context","authors":"Zhang","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}