{"type":"ScholarlyArticle","title":"Graph-based Neural Sentence Ordering","authors":[{"name":"Yongjing Yin","slug":"yongjing-yin","given":"Yongjing","family":"Yin","affiliation":"Xiamen University , Xiamen , China"},{"name":"Linfeng Song","slug":"linfeng-song","given":"Linfeng","family":"Song","affiliation":"University of Rochester , Rochester , NY , U.S.; Tencent AI Lab , Bellevue , WA , U.S."},{"name":"Jinsong Su","slug":"jinsong-su","given":"Jinsong","family":"Su","affiliation":"Xiamen University , Xiamen , China"},{"name":"Jiali Zeng","slug":"jiali-zeng","given":"Jiali","family":"Zeng","affiliation":"Xiamen University , Xiamen , China"},{"name":"Chulun Zhou","slug":"chulun-zhou","given":"Chulun","family":"Zhou","affiliation":"Xiamen University , Xiamen , China"},{"name":"Jiebo Luo","slug":"jiebo-luo","given":"Jiebo","family":"Luo","affiliation":"University of Rochester , Rochester , NY , U.S."}],"abstract":"Sentence ordering is to restore the original paragraph from a set of sentences. It involves capturing global dependencies among sentences regardless of their input order. In this paper, we propose a novel and flexible graph-based neural sentence ordering model, which adopts graph recurrent network \\citep{Zhang:acl18} to accurately learn semantic representations of the sentences. Instead of assuming connections between all pairs of input sentences, we use entities that are shared among multiple sentences to make more expressive graph representations with less noise. Experimental results show that our proposed model outperforms the existing state-of-the-art systems on several benchmark datasets, demonstrating the effectiveness of our model. We also conduct a thorough analysis on how entities help the performance. Our code is available at https://github.com/DeepLearnXMU/NSEG.git.","doi":"10.24963/ijcai.2019/748","url":"https://ijcai.org/proceedings/2019/748","pdf":"https://ijcai.org/proceedings/2019/0748.pdf","markdown":"https://ijcai.org/proceedings/2019/748.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/748","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":748,"pages":{"first":5387,"last":5393},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Discourse"],"videos":[],"references":[{"position":1,"raw":"[Barzilay and Lapata, 2005] Regina Barzilay and Mirella Lapata. Modeling local coherence: An entity-based ap- proach. In ACL, pages 141-148, 2005.","title":"Modeling local coherence: An entity-based approach","authors":"Lapata Barzilay, Regina Barzilay, Mirella Lapata","year":"2005","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barzilay and Lapata, 2008] Regina Barzilay and Mirella Lapata. Modeling local coherence: An entity-based ap- proach. Computational Linguistics, 34(1):1-34, 2008.","title":"Modeling local coherence: An entity-based approach","authors":"Lapata Barzilay, Regina Barzilay, Mirella Lapata","year":"2008","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Barzilay and Lee, 2004] Regina Barzilay and Lillian Lee. Catching the drift: Probabilistic content models, with ap- plications to generation and summarization. In NAACL, pages 113-120, 2004.","title":"Catching the drift: Probabilistic content models, with applications to generation and summarization","authors":"Lee ; Barzilay, Regina Barzilay, Lillian Lee","year":"2004","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Barzilay et al., 2002] Regina Barzilay, Noemie Elhadad, and Kathleen R. McKeown. Inferring strategies for sentence ordering in multidocument news summariza- tion. Journal of Artificial Intelligence Research, 17:35-55, 2002.","title":"Inferring strategies for sentence ordering in multidocument news summarization","authors":"Regina Barzilay, Noemie Elhadad, Kathleen R Mckeown","year":"2002","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bastings et al., 2017] Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Sima'an. Graph convolutional encoders for syntax-aware neural machine translation. In EMNLP, pages 1957-1967, 2017.","title":"Graph convolutional encoders for syntax-aware neural machine translation","authors":"Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, Khalil Sima'an","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Beck et al., 2018] Daniel Beck, Gholamreza Haffari, and Trevor Cohn. Graph-to-sequence learning using gated graph neural networks. In ACL, pages 273-283, 2018.","title":"Graph-to-sequence learning using gated graph neural networks","authors":"Daniel Beck, Gholamreza Haffari, Trevor Cohn","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2016] Xinchi Chen, Xipeng Qiu, and Xuanjing Huang. Neural sentence ordering. CoRR, abs/1607.06952, 2016.","title":"Neural sentence ordering","authors":"Chen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cui et al., 2018] Baiyun Cui, Yingming Li, Ming Chen, and Zhongfei Zhang. Deep attentive sentence ordering net- work. In EMNLP, pages 4340-4349, 2018.","title":"Deep attentive sentence ordering network","authors":"Baiyun Cui, Yingming Li, Ming Chen, Zhongfei Zhang","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Elsner and Charniak, 2011] Micha Elsner and Eugene Charniak. Extending the entity grid with entity-specific features. In ACL, pages 125-129, 2011.","title":"Extending the entity grid with entity-specific features","authors":"Charniak Elsner, Micha Elsner, Eugene Charniak","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Galanis et al., 2012] Dimitrios Galanis, Gerasimos Lam- pouras, and Ion Androutsopoulos. Extractive multi- document summarization with integer linear programming and support vector regression. In COLING, pages 911- 926, 2012.","title":"Extractive multidocument summarization with integer linear programming and support vector regression","authors":"Dimitrios Galanis, Gerasimos Lampouras, Ion Androutsopoulos","year":"2012","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gong et al., 2016] Jingjing Gong, Xinchi Chen, Xipeng Qiu, and Xuanjing Huang. End-to-end neural sentence ordering using pointer network. CoRR, abs/1611.04953, 2016.","title":"End-to-end neural sentence ordering using pointer network","authors":"Jingjing Gong, Xinchi Chen, Xipeng Qiu, Xuanjing Huang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guinaudeau and Strube, 2013] Camille Guinaudeau and Michael Strube. Graph-based local coherence modeling. In ACL, pages 93-103, 2013.","title":"Graph-based local coherence modeling","authors":"Strube Guinaudeau, Camille Guinaudeau, Michael Strube","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Koehn, 2004] Philipp Koehn. Statistical significance tests for machine translation evaluation. In EMNLP, pages 388- 395, 2004.","title":"Statistical significance tests for machine translation evaluation","authors":"Philipp Koehn, Koehn","year":"2004","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lapata, 2003] Mirella Lapata. Probabilistic text structuring: Experiments with sentence ordering. In ACL, 2003.","title":"Probabilistic text structuring: Experiments with sentence ordering","authors":"Mirella Lapata, Lapata","year":"2003","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li and Hovy, 2014] Jiwei Li and Eduard H. Hovy. A model of coherence based on distributed sentence representation. In EMNLP, pages 2039-2048, 2014.","title":"A model of coherence based on distributed sentence representation","authors":"Hovy ; ] Jiwei Li, Eduard H Li, Hovy","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Logeswaran et al., 2018] Lajanugen Logeswaran, Honglak Lee, and Dragomir R. Radev. Sentence ordering and co- herence modeling using recurrent neural networks. In AAAI Conference on Artificial Intelligence, pages 5285- 5292, 2018.","title":"Sentence ordering and coherence modeling using recurrent neural networks","authors":"Logeswaran","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Marcheggiani and Titov, 2017] Diego Marcheggiani and Ivan Titov. Encoding sentences with graph convolutional networks for semantic role labeling. In EMNLP, pages 1506-1515, 2017.","title":"Diego Marcheggiani and Ivan Titov. Encoding sentences with graph convolutional networks for semantic role labeling","authors":"Titov Marcheggiani","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Radev et al., 2016] Dragomir R. Radev, Mark Thomas Joseph, Bryan R. Gibson, and Pradeep Muthukrishnan. A bibliometric and network analysis of the field of computa- tional linguistics. JASIST, 67(3):683-706, 2016.","title":"A bibliometric and network analysis of the field of computational linguistics","authors":"R Dragomir, Mark Thomas Radev, Bryan R Joseph, Pradeep Gibson, Muthukrishnan","year":"2016","venue":"JASIST","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Scarselli et al., 2009] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Trans. Neural Networks, 20(1):61-80, 2009.","title":"The graph neural network model","authors":"Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, Gabriele Monfardini","year":"2009","venue":"IEEE Trans. Neural Networks","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Song et al., 2018a] Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. A graph-to-sequence model for amr-to-text generation. In ACL, pages 1616-1626, 2018.","title":"A graph-to-sequence model for amr-to-text generation","authors":"Song","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2018b] Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. N-ary relation extraction using graph-state lstm. In EMNLP, pages 2226-2235, 2018.","title":"N-ary relation extraction using graph-state lstm","authors":"Song","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Song et al., 2019] Linfeng Song, Daniel Gildea, Yue Zhang, Zhiguo Wang, and Jinsong Su. Semantic neural machine translation using amr. CoRR, abs/1902.07282, 2019.","title":"Semantic neural machine translation using amr","authors":"Song","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 6000-6010, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vinyals et al., 2015a] Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. Order matters: Sequence to sequence for sets. In ICLR, 2015.","title":"Order matters: Sequence to sequence for sets","authors":"Vinyals","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vinyals et al., 2015b] Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In NIPS, pages 2692- 2700, 2015.","title":"Pointer networks","authors":"Vinyals","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2018] Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. Cross-lingual knowledge graph align- ment via graph convolutional networks. In EMNLP, pages 349-357, 2018.","title":"Cross-lingual knowledge graph alignment via graph convolutional networks","authors":"Zhichun Wang, Qingsong Lv, Xiaohan Lan, Yu Zhang","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2018] Kun Xu, Lingfei Wu, Zhiguo Wang, Yan- song Feng, Michael Witbrock, and Vadim Sheinin. Graph2seq: Graph to sequence learning with attention- based neural networks. CoRR, abs/1804.00823, 2018.","title":"Graph2seq: Graph to sequence learning with attentionbased neural networks","authors":"Kun Xu, Lingfei Wu, Zhiguo Wang, Yansong Feng, Michael Witbrock, Vadim Sheinin","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yu et al., 2018] Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le. Qanet: Combining local convolution with global self-attention for reading comprehension. In ICLR, 2018.","title":"Qanet: Combining local convolution with global self-attention for reading comprehension","authors":"Adams Wei, Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, V Quoc, Le","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zeiler, 2012] Matthew D. Zeiler. ADADELTA: an adaptive learning rate method. CoRR, abs/1212.5701, 2012.","title":"ADADELTA: an adaptive learning rate method","authors":"D Matthew, Zeiler","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2018] Yue Zhang, Qi Liu, and Linfeng Song. Sentence-state lstm for text representation. In ACL, pages 317-327, 2018.","title":"Sentence-state lstm for text representation","authors":"Yue Zhang, Qi Liu, Linfeng Song","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}