{"type":"ScholarlyArticle","title":"SegBot: A Generic Neural Text Segmentation Model with Pointer Network","authors":[{"name":"Jing Li","slug":"jing-li","given":"Jing","family":"Li","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Aixin Sun","slug":"aixin-sun","given":"Aixin","family":"Sun","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Shafiq Joty","slug":"shafiq-joty","given":"Shafiq","family":"Joty","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"}],"abstract":"Text segmentation is a fundamental task in natural language processing that comes in two levels of granularity: (i) segmenting a document into a sequence of topical segments (topic segmentation), and (ii) segmenting a sentence into a sequence of elementary discourse units (EDU segmentation). Traditional solutions to the two tasks heavily rely on carefully designed features. The recently proposed neural models do not need manual feature engineering, but they either suffer from sparse boundary tags or they cannot well handle  the issue of variable size output vocabulary.  We propose a generic end-to-end segmentation model called SegBot. SegBot uses a bidirectional recurrent neural network to encode input text sequence. The model then uses another recurrent neural network together with a pointer network to select text boundaries in the input sequence. In this way, SegBot does not require hand-crafted features. More importantly, our model inherently handles the issue of variable size output vocabulary and the issue of sparse boundary tags. In our experiments, SegBot outperforms  state-of-the-art models on both topic and EDU segmentation tasks.","doi":"10.24963/ijcai.2018/579","url":"https://ijcai.org/proceedings/2018/579","pdf":"https://ijcai.org/proceedings/2018/0579.pdf","markdown":"https://ijcai.org/proceedings/2018/579.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/579","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":579,"pages":{"first":4166,"last":4172},"keywords":["Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Discourse","Natural Language Processing: Tagging, chunking, and parsing"],"videos":[],"references":[{"position":1,"raw":"[Arora et al., 2017] Sanjeev Arora, Yingyu Liang, and Tengyu Ma. A simple but tough-to-beat baseline for sentence embeddings. In ICLR, 2017.","title":"A simple but tough-to-beat baseline for sentence embeddings","authors":"Sanjeev Arora, Yingyu Liang, Tengyu Ma","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beeferman et al., 1999] Doug Beeferman, Adam Berger, and John Lafferty. Statistical models for text segmentation. Machine lear- ning, 34(1):177-210, 1999.","title":"Statistical models for text segmentation","authors":"Beeferman","year":"1999","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brants et al., 2002] Thorsten Brants, Francine Chen, and Ioannis Tsochantaridis. Topic-based document segmentation with proba- bilistic latent semantic analysis. In CIKM, pages 211-218, 2002.","title":"Topic-based document segmentation with probabilistic latent semantic analysis","authors":"Thorsten Brants, Francine Chen, Ioannis Tsochantaridis","year":"2002","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carlson et al., 2002] Lynn Carlson, Mary Ellen Okurowski, and Daniel Marcu. RST discourse treebank. Linguistic Data Con- sortium, University of Pennsylvania, 2002.","title":"RST discourse treebank","authors":"Lynn Carlson, Mary Ellen Okurowski, Daniel Marcu","year":"2002","venue":"Linguistic Data Consortium","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural ma- chine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259, 2014.","title":"On the properties of neural ma-chine translation: Encoder-decoder approaches","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi, 2000] Freddy YY Choi. Advances in domain independent linear text segmentation. In NAACL, pages 26-33, 2000.","title":"Advances in domain independent linear text segmentation","authors":"Freddy Yy Choi, Choi","year":"2000","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Du et al., 2013] Lan Du, Wray L Buntine, and Mark Johnson. To- pic segmentation with a structured topic model. In HLT-NAACL, pages 190-200, 2013.","title":"Topic segmentation with a structured topic model","authors":"Lan Du, Mark Wray L Buntine, Johnson","year":"2013","venue":"HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fisher and Roark, 2007] Seeger Fisher and Brian Roark. The uti- lity of parse-derived features for automatic discourse segmenta- tion. In ACL, volume 45, page 488, 2007.","title":"The utility of parse-derived features for automatic discourse segmentation","authors":"Roark Fisher, Seeger Fisher, Brian Roark","year":"2007","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fragkou et al., 2004] Pavlina Fragkou, Vassilios Petridis, and Ath Kehagias. A dynamic programming algorithm for linear text seg- mentation. JIIS, 23(2):179-197, 2004.","title":"Pavlina Fragkou, Vassilios Petridis, and Ath Kehagias. A dynamic programming algorithm for linear text segmentation","authors":"Fragkou","year":"2004","venue":"JIIS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Galley et al., 2003] Michel Galley, Kathleen McKeown, Eric Fosler-Lussier, and Hongyan Jing. Discourse segmentation of multi-party conversation. In ACL, pages 562-569, 2003.","title":"Discourse segmentation of multi-party conversation","authors":"Michel Galley, Kathleen Mckeown, Eric Fosler-Lussier, Hongyan Jing","year":"2003","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Glavaš et al., 2016] Goran Glavaš, Federico Nanni, and Si- mone Paolo Ponzetto. Unsupervised text segmentation using se- mantic relatedness graphs. In SEM@ACL, 2016.","title":"Unsupervised text segmentation using semantic relatedness graphs","authors":"Goran Glavaš, Federico Nanni, Simone Paolo, Ponzetto","year":"2016","venue":"SEM@ACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Goldberg, 2017] Yoav Goldberg. Neural Network Methods in Na- tural Language Processing. Morgan & Claypool Publishers, 2017.","title":"Neural Network Methods in Natural Language Processing","authors":"Yoav Goldberg","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Marti A Hearst. Texttiling: Segmenting text into multi-paragraph subtopic passages. Computational linguistics, 23(1):33-64, 1997.","title":"Texttiling: Segmenting text into multi-paragraph subtopic passages","authors":"A Marti, Hearst","year":"1997","venue":"Computational linguistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hernault et al., 2010] Hugo Hernault, Danushka Bollegala, and Mitsuru Ishizuka. A sequential model for discourse segmenta- tion. In CICLing, pages 315-326, 2010.","title":"A sequential model for discourse segmentation","authors":"Hernault","year":"2010","venue":"CICLing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hsueh et al., 2006] Pei-Yun Hsueh, Johanna D Moore, and Steve Renals. Automatic segmentation of multiparty dialogue. In EACL, 2006.","title":"Automatic segmentation of multiparty dialogue","authors":"Pei-Yun Hsueh, Johanna D Moore, Steve Renals","year":"2006","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ji and Zha, 2003] Xiang Ji and Hongyuan Zha. Domain- independent text segmentation using anisotropic diffusion and dynamic programming. In SIGIR, pages 322-329, 2003.","title":"Domainindependent text segmentation using anisotropic diffusion and dynamic programming","authors":"Xiang Ji, Hongyuan Zha","year":"2003","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Joty et al., 2015] Shafiq Joty, Giuseppe Carenini, and Raymond T Ng. Codra: A novel discriminative framework for rhetorical ana- lysis. Computational Linguistics, 41:385-435, 2015.","title":"Codra: A novel discriminative framework for rhetorical analysis","authors":"Shafiq Joty, Giuseppe Carenini, Raymond T Ng","year":"2015","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lamb et al., 2016] Alex M Lamb, Anirudh GOYAL, Goyal, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Ben- gio. Professor forcing: A new algorithm for training recurrent networks. In NIPS, pages 4601-4609, 2016.","title":"Professor forcing: A new algorithm for training recurrent networks","authors":"Alex M Lamb, Goyal Anirudh, Ying Goyal, Saizheng Zhang, Aaron C Zhang, Yoshua Courville, Bengio","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lample et al., 2016] Guillaume Lample, Miguel Ballesteros, San- deep Subramanian, Kazuya Kawakami, and Chris Dyer. Neu- ral architectures for named entity recognition. arXiv preprint arXiv:1603.01360, 2016.","title":"Neural architectures for named entity recognition","authors":"Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, Chris Dyer","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2011] Xiaohua Liu, Shaodian Zhang, Furu Wei, and Ming Zhou. Recognizing named entities in tweets. In ACL, pages 359-367, 2011.","title":"Recognizing named entities in tweets","authors":"Xiaohua Liu, Shaodian Zhang, Furu Wei, Ming Zhou","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Marcu, 2000] Daniel Marcu. The Theory and Practice of Dis- course Parsing and Summarization. MIT Press, 2000.","title":"The Theory and Practice of Discourse Parsing and Summarization","authors":"Daniel Marcu","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Misra et al., 2009] Hemant Misra, François Yvon, Joemon M Jose, and Olivier Cappe. Text segmentation via topic modeling: an analytical study. In CIKM, pages 1553-1556, 2009.","title":"Text segmentation via topic modeling: an analytical study","authors":"Hemant Misra, François Yvon, Joemon M Jose, Olivier Cappe","year":"2009","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word represen- tation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Qiao et al., 2015] Maoying Qiao, Wei Bian, Richard Xu, and Da- cheng Tao. Diversified hidden markov models for sequential la- beling. TKDE, 27(11):2947-2960, 2015.","title":"Diversified hidden markov models for sequential labeling","authors":"Maoying Qiao, Wei Bian, Richard Xu, Dacheng Tao","year":"2015","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Riedl and Biemann, 2012] Martin Riedl and Chris Biemann. To- pictiling: a text segmentation algorithm based on lda. In ACL, pages 37-42, 2012.","title":"Topictiling: a text segmentation algorithm based on lda","authors":"Biemann Riedl, Martin Riedl, Chris Biemann","year":"2012","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Soricut and Marcu, 2003] Radu Soricut and Daniel Marcu. Sen- tence level discourse parsing using syntactic and lexical informa- tion. In NAACL, pages 149-156, 2003.","title":"Sentence level discourse parsing using syntactic and lexical information","authors":"Radu Soricut, Daniel Marcu","year":"2003","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Utiyama and Isahara, 2001] Masao Utiyama and Hitoshi Isahara. A statistical model for domain-independent text segmentation. In ACL, pages 499-506, 2001.","title":"A statistical model for domain-independent text segmentation","authors":"Masao Utiyama, Hitoshi Isahara","year":"2001","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Vaswani et al., 2016] Ashish Vaswani, Yonatan Bisk, Kenji Sagae, and Ryan Musa. Supertagging with lstms. In HLT-NAACL, pages 232-237, 2016.","title":"Supertagging with lstms","authors":"Ashish Vaswani, Yonatan Bisk, Kenji Sagae, Ryan Musa","year":"2016","venue":"HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Vinyals et al., 2015] Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In NIPS, pages 2692-2700, 2015.","title":"Pointer networks","authors":"Oriol Vinyals, Meire Fortunato, Navdeep Jaitly","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[William and Thompson, 1988] Mann William and Sandra Thomp- son. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243-281, 1988.","title":"Mann William and Sandra Thompson. Rhetorical structure theory: Toward a functional theory of text organization","authors":"Thompson William","year":"1988","venue":"Text","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}