{"type":"ScholarlyArticle","title":"Automatic Assessment of Absolute Sentence Complexity","authors":[{"name":"Sanja Stajner","slug":"sanja-stajner","given":"Sanja","family":"Stajner","affiliation":"Data and Web Science Group , University of Mannheim , Germany"},{"name":"Simone Paolo Ponzetto","slug":"simone-paolo-ponzetto","given":"Simone Paolo","family":"Ponzetto","affiliation":"Data and Web Science Group , University of Mannheim , Germany"},{"name":"Heiner Stuckenschmidt","slug":"heiner-stuckenschmidt","given":"Heiner","family":"Stuckenschmidt","affiliation":"Data and Web Science Group , University of Mannheim , Germany"}],"abstract":"Lexically and syntactically simpler sentences result in shorter reading time and better understanding in many people. However, no reliable systems for automatic assessment of absolute sentence complexity have been proposed so far. Instead, the assessment is usually done manually, requiring expert human annotators. To address this problem, we first define the sentence complexity assessment as a five-level classification task, and build a ‘gold standard’ dataset. Next, we propose robust systems for sentence complexity assessment, using a novel set of features based on leveraging lexical properties of freely available corpora, and investigate the impact of the feature type and corpus size on the classification performance.","doi":"10.24963/ijcai.2017/572","url":"https://ijcai.org/proceedings/2017/572","pdf":"https://ijcai.org/proceedings/2017/0572.pdf","markdown":"https://ijcai.org/proceedings/2017/572.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/572","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":572,"pages":{"first":4096,"last":4102},"keywords":["Natural Language Processing: Text Classification","Natural Language Processing: Resources and Evaluation"],"videos":[],"references":[{"position":1,"raw":"[Aluísio and Gasperin, 2010] Sandra Maria Aluísio and Caro- line Gasperin. Fostering Digital Inclusion and Accessibility: The PorSimples Project for Simplification of Portuguese Texts. In Proceedings of YIWCALA, pages 46-53, 2010.","title":"Fostering Digital Inclusion and Accessibility: The PorSimples Project for Simplification of Portuguese Texts","authors":"Sandra Maria, Aluísio, Caroline Gasperin","year":"2010","venue":"Proceedings of YIWCALA","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ambati et al., 2016] Bharat Ram Ambati, Siva Reddy, and Mark Steedman. Assessing Relative Sentence Complex- ity using an Incremental CCG Parser. In Proceedings of NAACL-HLT, pages 1051-1057, 2016.","title":"Assessing Relative Sentence Complexity using an Incremental CCG Parser","authors":"Ram Bharat, Siva Ambati, Mark Reddy, Steedman","year":"2016","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Angrosh et al., 2014] Mandya Angrosh, Tadashi Nomoto, and Advaith Siddharthan. Lexico-syntactic text simpli- fication and compression with typed dependencies. In Pro- cedings of COLING 2014, pages 1996-2006, 2014.","title":"Lexico-syntactic text simplification and compression with typed dependencies","authors":"Angrosh","year":"2014","venue":"Procedings of COLING 2014","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Breiman, 2001] Leo Breiman. Random Forests. Machine Learning, 45(1):5-32, 2001.","title":"Leo Breiman. Random Forests","authors":"Breiman","year":"2001","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Carroll et al., 1999] John Carroll, Guido Minnen, Darren Pearce, Yvonne Canning, Siobhan Devlin, and John Tait. Simplifying text for language-impaired readers. In Pro- ceedings of EACL 1999, pages 269-270, 1999.","title":"Simplifying text for language-impaired readers","authors":"John Carroll, Guido Minnen, Darren Pearce, Yvonne Canning, Siobhan Devlin, John Tait","year":"1999","venue":"Proceedings of EACL 1999","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cohen, 1995] William W. Cohen. Fast Effective Rule Induc- tion. In Proceedings of the ICML, pages 115-123, 1995.","title":"Fast Effective Rule Induction","authors":"William W Cohen","year":"1995","venue":"Proceedings of the ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[De Belder and Moens, 2010] Jan De Belder and Marie- Francine Moens. Text simplification for children. In Pro- ceedings of the SIGIR workshop on accessible search sys- tems, pages 19-26, 2010.","title":"Text simplification for children","authors":"De Belder, Jan De Moens, Marie-Francine Belder, Moens","year":"2010","venue":"Proceedings of the SIGIR workshop on accessible search systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Drndarevic et al., 2012] Biljana Drndarevic, Sanja Štajner, and Horacio Saggion. Reporting Simply: A Lexical Sim- plification Strategy for Enhancing Text Accessibility. In Proceedings of Easy-to-read on the Web Symposium, 2012.","title":"Reporting Simply: A Lexical Simplification Strategy for Enhancing Text Accessibility","authors":"Biljana Drndarevic, Sanja Štajner, Horacio Saggion","year":"2012","venue":"Proceedings of Easy-to-read on the Web Symposium","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Flesch, 1949] Rudolf Flesch. The art of readable writing. Harper, New York, 1949.","title":"Rudolf Flesch. The art of readable writing","authors":"Flesch","year":"1949","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Glavaš and Štajner, 2015] Goran Glavaš and Sanja Štajner. Simplifying Lexical Simplification: Do We Need Simpli- fied Corpora? In Proceedings of ACL&IJCNLP, pages 63-68, 2015.","title":"Simplifying Lexical Simplification: Do We Need Simplified Corpora?","authors":"Štajner Glavaš, Goran Glavaš, Sanja Štajner","year":"2015","venue":"Proceedings of ACL&IJCNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gunning, 1952] Robert Gunning. The technique of clear writing. McGraw-Hill, New York, 1952.","title":"The technique of clear writing","authors":"Robert Gunning","year":"1952","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hall and Smith, 1998] Mark A. Hall and Lloyd A. Smith. Practical feature subset selection for machine learning. In C. McDonald, editor, Proceedings of the 21st Australasian Computer Science Conference (ACSC), pages 181-191. Berlin: Springer, 1998.","title":"Practical feature subset selection for machine learning","authors":"Mark A Hall, Lloyd A Smith","year":"1998","venue":"Proceedings of the 21st Australasian Computer Science Conference (ACSC)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hall et al., 2009] Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Wit- ten. The weka data mining software: an update. SIGKDD Explor. Newsl., 11:10-18, 2009.","title":"Witten. The weka data mining software: an update","authors":"Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, Ian H","year":"2009","venue":"SIGKDD Explor. Newsl","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Horn et al., 2014] Colby Horn, Cathryn Manduca, and David Kauchak. Learning a lexical simplifier using wikipedia. In Proceedings of ACL 2014 (Short Papers), pages 458-463, 2014.","title":"Learning a lexical simplifier using wikipedia","authors":"Colby Horn, Cathryn Manduca, David Kauchak","year":"2014","venue":"Proceedings of ACL 2014 (Short Papers)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[le Cessie and van Houwelingen, 1992] Saskia le Cessie and Johannes C. van Houwelingen. Ridge Estimators in Logis- tic Regression. Applied Statistics, 41(1):191-201, 1992.","title":"Ridge Estimators in Logistic Regression","authors":"; Le Cessie And Van Houwelingen, Johannes C Cessie, Van Houwelingen","year":"1992","venue":"Applied Statistics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Paetzold and Specia, 2015] Gustavo Henrique Paetzold and Lucia Specia. LEXenstein: A Framework for Lexical Sim- plification. In Proceedings of ACL-IJCNLP 2015 System Demonstrations, pages 85-90, 2015.","title":"LEXenstein: A Framework for Lexical Simplification","authors":"Gustavo Henrique, Paetzold, Lucia Specia","year":"2015","venue":"Proceedings of ACL-IJCNLP 2015 System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Paetzold and Specia, 2016] Gustavo Henrique Paetzold and Lucia Specia. Unsupervised lexical simplification for non- native speakers. In Proceedings of the AAAI, 2016.","title":"Unsupervised lexical simplification for nonnative speakers","authors":"Gustavo Henrique, Paetzold, Lucia Specia","year":"2016","venue":"Proceedings of the AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Petersen and Ostendorf, 2007] Sarah E. Petersen and Mari Ostendorf. Text Simplification for Language Learners: A Corpus Analysis. In Proceedings of the SLaTE Workshop, pages 69-72, 2007.","title":"Text Simplification for Language Learners: A Corpus Analysis","authors":"Sarah E Petersen, Mari Ostendorf","year":"2007","venue":"Proceedings of the SLaTE Workshop","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Platt, 1998] John C. Platt. Fast Training of Support Vec- tor Machines using Sequential Minimal Optimization. In B. Schoelkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods Support Vector Learning, chapter 12, pages 41-65. MIT Press Cambridge, 1998.","title":"Fast Training of Support Vector Machines using Sequential Minimal Optimization","authors":"John C Platt","year":"1998","venue":"Advances in Kernel Methods Support Vector Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Porter et al., 2011] Andrew Porter, Jennifer McMaken, Jun Hwang, and Rui Yang. Common Core Standards: The New U.S. Intended Curriculum. Educational Researcher, 40(3):103-116, 2011.","title":"Common Core Standards: The New U.S. Intended Curriculum","authors":"Andrew Porter, Jennifer Mcmaken, Jun Hwang, Rui Yang","year":"2011","venue":"Educational Researcher","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Quinlan, 1993] Ross Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, CA, 1993.","title":"C4.5: Programs for Machine Learning","authors":"Ross Quinlan","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Saggion et al., 2015] Horacio Saggion, Sanja Štajner, Ste- fan Bott, Simon Mille, Luz Rello, and Biljana Drndarevic. Making It Simplext: Implementation and Evaluation of a Text Simplification System for Spanish. ACM Transactions on Accessible Computing, 6(4):14:1-14:36, 2015.","title":"Making It Simplext: Implementation and Evaluation of a Text Simplification System for Spanish","authors":"Horacio Saggion, Sanja Štajner, Stefan Bott, Simon Mille, Luz Rello, Biljana Drndarevic","year":"2015","venue":"ACM Transactions on Accessible Computing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vajjala and Meurers, 2014] Sowmya Vajjala and Detmar Meurers. Assessing the relative reading level of sentence pairs for text simplification. In Proceedings of the EACL 2014, pages 288-297, 2014.","title":"Assessing the relative reading level of sentence pairs for text simplification","authors":"Meurers Vajjala, Sowmya Vajjala, Detmar Meurers","year":"2014","venue":"Proceedings of the EACL 2014","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vajjala and Meurers, 2015] Sowmya Vajjala and Detmar Meurers. Readability-based Sentence Ranking for Eval- uating Text Simplification. Unpublished technical report, arXiv:1603.06009 [cs.CL], 2015.","title":"Readability-based Sentence Ranking for Evaluating Text Simplification","authors":"Meurers Vajjala, Sowmya Vajjala, Detmar Meurers","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Štajner and Glavaš, 2017] Sanja Štajner and Goran Glavaš. Leveraging event-based semantics for automated text sim- plification. Expert Systems with Applications, 82:383 -395, 2017.","title":"Leveraging event-based semantics for automated text simplification","authors":"Glavaš Štajner, Sanja Štajner, Goran Glavaš","year":"2017","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Štajner and Saggion, 2015] Sanja Štajner and Horacio Sag- gion. Translating from Original to Simplified Sentences using Moses: When does it Actually Work? In Proceedings of RANLP 2015, pages 611-617, 2015.","title":"Sanja Štajner and Horacio Saggion. Translating from Original to Simplified Sentences using Moses: When does it Actually Work","authors":"Saggion Štajner","year":"2015","venue":"Proceedings of RANLP 2015","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Štajner et al., 2013] Sanja Štajner, Biljana Drndarević, and Horacio Saggion. Corpus-based Sentence Deletion and Split Decisions for Spanish Text Simplification. Com- putación y Systemas, 17(2):251-262, 2013.","title":"Corpus-based Sentence Deletion and Split Decisions for Spanish Text Simplification","authors":"Sanja Štajner, Biljana Drndarević, Horacio Saggion","year":"2013","venue":"Computación y Systemas","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Štajner et al., 2016a] Sanja Štajner, Maja Popović, and Hanna Béchara. Quality Estimation for Text Simplification. In Proceedings of the QATS Workshop, pages 15-21, 2016.","title":"Quality Estimation for Text Simplification","authors":"Štajner","year":"2016","venue":"Proceedings of the QATS Workshop","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Štajner et al., 2016b] Sanja. Štajner, Maja Popović, Horacio Saggion, Lucia Specia, and Mark Fishel. Shared Task on Quality Assessment for Text Simplification. In Proceedings of the QATS Workshop, pages 22-31, 2016.","title":"Shared Task on Quality Assessment for Text Simplification","authors":"Štajner","year":"2016","venue":"Proceedings of the QATS Workshop","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Woodsend and Lapata, 2011] Kristian Woodsend and Mirella Lapata. Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming. In Proceedings of the EMNLP, pages 409-420, 2011.","title":"Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming","authors":"Lapata Woodsend, Kristian Woodsend, Mirella Lapata","year":"2011","venue":"Proceedings of the EMNLP","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xia et al., 2016] Menglin Xia, Ekaterina Kochmar, and Ted Briscoe. Text Readability Assessment for Second Language Learners. In Proceedings of the BEA Workshop, pages 12- 22, 2016.","title":"Text Readability Assessment for Second Language Learners","authors":"Xia","year":"2016","venue":"Proceedings of the BEA Workshop","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xu et al., 2015] Wei Xu, Chris Callison-Burch, and Court- ney Napoles. Problems in Current Text Simplification Re- search: New Data Can Help. TACL, 3:283-297, 2015.","title":"Problems in Current Text Simplification Research: New Data Can Help","authors":"Wei Xu, Chris Callison-Burch, Courtney Napoles","year":"2015","venue":"TACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}