{"type":"ScholarlyArticle","title":"Better AMR-To-Text Generation with Graph Structure Reconstruction","authors":[{"name":"Tianming Wang","slug":"tianming-wang","given":"Tianming","family":"Wang","affiliation":"Wangxuan Institute of Computer Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"},{"name":"Xiaojun Wan","slug":"xiaojun-wan","given":"Xiaojun","family":"Wan","affiliation":"Wangxuan Institute of Computer Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"},{"name":"Shaowei Yao","slug":"shaowei-yao","given":"Shaowei","family":"Yao","affiliation":"Wangxuan Institute of Computer Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University"}],"abstract":"AMR-to-text generation is a challenging task of generating texts from graph-based semantic representations. Recent studies formalize this task a graph-to-sequence learning problem and use various graph neural networks to model graph structure. In this paper, we propose a novel approach that generates texts from AMR graphs while reconstructing the input graph structures. Our model employs graph attention mechanism to aggregate information for encoding the inputs. Moreover, better node representations are learned by optimizing two simple but effective auxiliary reconstruction objectives: link prediction objective which requires predicting the semantic relationship between nodes, and distance prediction objective which requires predicting the distance between nodes. Experimental results on two benchmark datasets show that our proposed model improves considerably over strong baselines and achieves new state-of-the-art.","doi":"10.24963/ijcai.2020/542","url":"https://ijcai.org/proceedings/2020/542","pdf":"https://ijcai.org/proceedings/2020/0542.pdf","markdown":"https://ijcai.org/proceedings/2020/542.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/542","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":542,"pages":{"first":3919,"last":3925},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Natural Language Generation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24041/file_24041.m3u8","length_sec":290,"provider":"hls"}],"references":[{"position":1,"raw":"[Banerjee and Lavie, 2005] Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceed- ings of the acl workshop on intrinsic and extrinsic evalua- tion measures for machine translation and/or summariza- tion, pages 65-72, 2005.","title":"Meteor: An automatic metric for mt evaluation with improved correlation with human judgments","authors":"Lavie Banerjee, Satanjeev Banerjee, Alon Lavie","year":"2005","venue":"Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beck et al., 2018] Daniel Beck, Gholamreza Haffari, and Trevor Cohn. Graph-to-sequence learning using gated graph neural networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 273-283, 2018.","title":"Graph-to-sequence learning using gated graph neural networks","authors":"Daniel Beck, Gholamreza Haffari, Trevor Cohn","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai and Knight, 2013] Shu Cai and Kevin Knight. Smatch: an evaluation metric for semantic feature structures. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 748-752, 2013.","title":"Smatch: an evaluation metric for semantic feature structures","authors":"Knight ; ] Shu Cai, Kevin Cai, Knight","year":"2013","venue":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai and Lam, 2019] Deng Cai and Wai Lam. Graph trans- former for graph-to-sequence learning. arXiv preprint arXiv:1911.07470, 2019.","title":"Graph transformer for graph-to-sequence learning","authors":"Lam ; ] Deng Cai, Wai Cai, Lam","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Damonte and Cohen, 2019] Marco Damonte and Shay B. Cohen. Structural neural encoders for amr-to-text genera- tion. arXiv preprint arXiv:1903.11410v1, 2019.","title":"Structural neural encoders for amr-to-text generation","authors":"Marco Damonte, Shay B Cohen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Damonte et al., 2016] Marco Damonte, Shay B Cohen, and Giorgio Satta. An incremental parser for abstract meaning representation. arXiv preprint arXiv:1608.06111, 2016.","title":"An incremental parser for abstract meaning representation","authors":"Marco Damonte, Giorgio Shay B Cohen, Satta","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Flanigan et al., 2016] Jeffrey Flanigan, Chris Dyer, Noah A Smith, and Jaime Carbonell. Generation from abstract meaning representation using tree transducers. In Proceed- ings of the 2016 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 731-739, 2016.","title":"Generation from abstract meaning representation using tree transducers","authors":"Jeffrey Flanigan, Chris Dyer, Noah A Smith, Jaime Carbonell","year":"2016","venue":"Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: Human language technologies","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2019] Zhijiang Guo, Yan Zhang, Zhiyang Teng, and Wei Lu. Densely connected graph convolutional net- works for graph-to-sequence learning. Transactions of the Association for Computational Linguistics, 7:297-312, 2019.","title":"Densely connected graph convolutional networks for graph-to-sequence learning","authors":"Zhijiang Guo, Yan Zhang, Zhiyang Teng, Wei Lu","year":"2019","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. interna- tional conference on learning representations, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"international conference on learning representations","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Konstas et al., 2017] Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi, and Luke Zettlemoyer. Neural amr: Sequence-to-sequence models for parsing and generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 146-157, 2017.","title":"Neural amr: Sequence-to-sequence models for parsing and generation","authors":"Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi, Luke Zettlemoyer","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for auto- matic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311-318. Association for Computational Linguistics, 2002.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of the 40th annual meeting on association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Popović, 2017] Maja Popović. chrf++: words helping char- acter n-grams. In Proceedings of the second conference on machine translation, pages 612-618, 2017.","title":"chrf++: words helping character n-grams","authors":"Maja Popović, Popović","year":"2017","venue":"Proceedings of the second conference on machine translation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pourdamghani et al., 2016] Nima Pourdamghani, Kevin Knight, and Ulf Hermjakob. Generating english from abstract meaning representations. In Proceedings of the 9th international natural language generation conference, pages 21-25, 2016.","title":"Generating english from abstract meaning representations","authors":"Pourdamghani","year":"2016","venue":"Proceedings of the 9th international natural language generation conference","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ribeiro et al., 2019] Leonardo FR Ribeiro, Claire Gardent, and Iryna Gurevych. Enhancing amr-to-text generation with dual graph representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP-IJCNLP), pages 3174-3185, 2019.","title":"Enhancing amr-to-text generation with dual graph representations","authors":"Ribeiro","year":"2019","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Song et al., 2017] Linfeng Song, Xiaochang Peng, Yue Zhang, Zhiguo Wang, and Daniel Gildea. Amr-to-text generation with synchronous node replacement grammar. meeting of the association for computational linguistics, 2:7-13, 2017.","title":"Amr-to-text generation with synchronous node replacement grammar","authors":"Song","year":"2017","venue":"meeting of the association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Song et al., 2018] Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. A graph-to-sequence model for amr-to- text generation. In Proceedings of the 56th Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 1: Long Papers), volume 1, pages 1616-1626, 2018.","title":"A graph-to-sequence model for amr-totext generation","authors":"Song","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang et al., 2019] Sheng Zhang, Xutai Ma, Kevin Duh, and Benjamin Van Durme. Amr parsing as sequence-to- graph transduction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 80-94, 2019.","title":"Amr parsing as sequence-tograph transduction","authors":"Sheng Zhang, Xutai Ma, Kevin Duh, Benjamin Van Durme","year":"2019","venue":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhu et al., 2019] Jie Zhu, Junhui Li, Muhua Zhu, Longhua Qian, Min Zhang, and Guodong Zhou. Modeling graph structure in transformer for better amr-to-text generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural Language Pro- cessing (EMNLP-IJCNLP), pages 5462-5471, 2019.","title":"Modeling graph structure in transformer for better amr-to-text generation","authors":"Jie Zhu, Junhui Li, Muhua Zhu, Longhua Qian, Min Zhang, Guodong Zhou","year":"2019","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}