{"type":"ScholarlyArticle","title":"Hierarchical Multi-task Learning for Organization Evaluation of Argumentative Student Essays","authors":[{"name":"Wei Song","slug":"wei-song","given":"Wei","family":"Song","affiliation":"College of Information Engineering and Academy for Multidisciplinary Studies , Capital Normal University , Beijing , China"},{"name":"Ziyao Song","slug":"ziyao-song","given":"Ziyao","family":"Song","affiliation":"College of Information Engineering and Academy for Multidisciplinary Studies , Capital Normal University , Beijing , China"},{"name":"Lizhen Liu","slug":"lizhen-liu","given":"Lizhen","family":"Liu","affiliation":"College of Information Engineering and Academy for Multidisciplinary Studies , Capital Normal University , Beijing , China"},{"name":"Ruiji Fu","slug":"ruiji-fu","given":"Ruiji","family":"Fu","affiliation":"State Key Laboratory of Cognitive Intelligence , iFLYTEK Research , China; iFLYTEK AI Research (Hebei) , Langfang , China"}],"abstract":"Organization evaluation is an important dimension of automated essay scoring. \r\nThis paper focuses on discourse element (i.e., functions of sentences and paragraphs) based organization evaluation.\r\nExisting approaches mostly separate discourse element identification and organization evaluation.\r\nIn contrast,  we propose a neural hierarchical multi-task learning approach for jointly optimizing sentence and paragraph level discourse element identification and organization evaluation.\r\nWe represent the organization as a grid to simulate the visual layout of an essay and integrate discourse elements at multiple linguistic levels.\r\nExperimental results show that the multi-task learning based organization evaluation can achieve significant improvements compared with existing work and pipeline baselines. \r\nMultiple level discourse element identification also benefits from multi-task learning through mutual enhancement.","doi":"10.24963/ijcai.2020/536","url":"https://ijcai.org/proceedings/2020/536","pdf":"https://ijcai.org/proceedings/2020/0536.pdf","markdown":"https://ijcai.org/proceedings/2020/536.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/536","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":536,"pages":{"first":3875,"last":3881},"keywords":["Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Discourse"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24702/file_24702.m3u8","length_sec":265,"provider":"hls"}],"references":[{"position":1,"raw":"[Attali and Burstein, 2006] Yigal Attali and Jill Burstein. Automated essay scoring with e-rater v. 2. The Journal of Technology, Learning and Assessment, 4(3), 2006.","title":"Automated essay scoring with e-rater v. 2","authors":"Burstein Attali, Yigal Attali, Jill Burstein","year":"2006","venue":"The Journal of Technology, Learning and Assessment","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Burstein et al., 2003] Jill Burstein, Daniel Marcu, and Kevin Knight. Finding the write stuff: Automatic iden- tification of discourse structure in student essays. IEEE Intelligent Systems, 18(1):32-39, 2003.","title":"Finding the write stuff: Automatic identification of discourse structure in student essays","authors":"Jill Burstein, Daniel Marcu, Kevin Knight","year":"2003","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Collobert and Weston, 2008] Ronan Collobert and Jason Weston. A unified architecture for natural language pro- cessing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Ma- chine learning, pages 160-167, 2008.","title":"A unified architecture for natural language processing: Deep neural networks with multitask learning","authors":"Weston ; Collobert, Ronan Collobert, Jason Weston","year":"2008","venue":"Proceedings of the 25th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dong et al., 2017] Fei Dong, Yue Zhang, and Jie Yang. Attention-based recurrent convolutional neural network for automatic essay scoring. In Proceedings of the 21st Conference on Computational Natural Language Learn- ing (CoNLL 2017), pages 153-162, 2017.","title":"Attention-based recurrent convolutional neural network for automatic essay scoring","authors":"Fei Dong, Yue Zhang, Jie Yang","year":"2017","venue":"Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Farag and Yannakoudakis, 2019] Youmna Farag and Helen Yannakoudakis. Multi-task learning for coherence model- ing. In Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, pages 629-639, 2019.","title":"Youmna Farag and Helen Yannakoudakis. Multi-task learning for coherence modeling","authors":"Yannakoudakis Farag","year":"2019","venue":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hashimoto et al., 2017] Kazuma Hashimoto, Yoshimasa Tsuruoka, Richard Socher, et al. A joint many-task model: Growing a neural network for multiple nlp tasks. In Pro- ceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1923-1933, 2017.","title":"A joint many-task model: Growing a neural network for multiple nlp tasks","authors":"Kazuma Hashimoto, Yoshimasa Tsuruoka, Richard Socher","year":"2017","venue":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"Marti A Hearst. Texttiling: Segmenting text into multi-paragraph subtopic passages. Computational linguistics, 23(1):33-64, 1997.","title":"Texttiling: Segmenting text into multi-paragraph subtopic passages","authors":"A Marti, Hearst","year":"1997","venue":"Computational linguistics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Higgins et al., 2004] Derrick Higgins, Jill Burstein, Daniel Marcu, and Claudia Gentile. Evaluating multiple aspects of coherence in student essays. In Proceedings of the Hu- man Language Technology Conference of the North Amer- ican Chapter of the Association for Computational Lin- guistics: HLT-NAACL 2004, pages 185-192, 2004.","title":"Evaluating multiple aspects of coherence in student essays","authors":"Derrick Higgins, Jill Burstein, Daniel Marcu, Claudia Gentile","year":"2004","venue":"Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ke et al., 2018] Zixuan Ke, Winston Carlile, Nishant Gurra- padi, and Vincent Ng. Learning to give feedback: Model- ing attributes affecting argument persuasiveness in student essays. In IJCAI, pages 4130-4136, 2018.","title":"Learning to give feedback: Modeling attributes affecting argument persuasiveness in student essays","authors":"Ke","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/574"},{"position":11,"raw":"[Mann and Thompson, 1988] William C Mann and Sandra A Thompson. Rhetorical structure theory: Toward a func- tional theory of text organization. Text-interdisciplinary Journal for the Study of Discourse, 8(3):243-281, 1988.","title":"Rhetorical structure theory: Toward a functional theory of text organization","authors":"C William, Sandra A Mann, Thompson","year":"1988","venue":"Text-interdisciplinary Journal for the Study of Discourse","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"Ellis B Page. The imminence of... grading es- says by computer. The Phi Delta Kappan, 47(5):238-243, 1966.","title":"The imminence of... grading essays by computer","authors":"Ellis B Page","year":"1966","venue":"The Phi Delta Kappan","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Persing et al., 2010] Isaac Persing, Alan Davis, and Vincent Ng. Modeling organization in student essays. In Proceed- ings of the 2010 Conference on Empirical Methods in Nat- ural Language Processing, pages 229-239, 2010.","title":"Modeling organization in student essays","authors":"Persing","year":"2010","venue":"Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sanh et al., 2019] Victor Sanh, Thomas Wolf, and Sebastian Ruder. A hierarchical multi-task approach for learning em- beddings from semantic tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6949-6956, 2019.","title":"A hierarchical multi-task approach for learning embeddings from semantic tasks","authors":"Sanh","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Smith, 2003] Carlota S Smith. Modes of discourse: The lo- cal structure of texts, volume 103. Cambridge University Press, 2003.","title":"Modes of discourse: The local structure of texts","authors":"Carlota S Smith","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Søgaard and Goldberg, 2016] Anders Søgaard and Yoav Goldberg. Deep multi-task learning with low level tasks supervised at lower layers. In Proceedings of the 54th An- nual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers), pages 231-235, 2016.","title":"Deep multi-task learning with low level tasks supervised at lower layers","authors":"Goldberg Søgaard, Anders Søgaard, Yoav Goldberg","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Song et al., 2015] Wei Song, Ruiji Fu, Lizhen Liu, and Ting Liu. Discourse element identification in student essays based on global and local cohesion. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing, pages 2255-2261, 2015.","title":"Discourse element identification in student essays based on global and local cohesion","authors":"Wei Song, Ruiji Fu, Lizhen Liu, Ting Liu","year":"2015","venue":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Song et al., 2017] Wei Song, Dong Wang, Ruiji Fu, Lizhen Liu, Ting Liu, and Guoping Hu. Discourse mode iden- tification in essays. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 112-122, 2017.","title":"Discourse mode identification in essays","authors":"Wei Song, Dong Wang, Ruiji Fu, Lizhen Liu, Ting Liu, Guoping Hu","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Song et al., 2018] Yan Song, Shuming Shi, Jing Li, and Haisong Zhang. Directional skip-gram: Explicitly distin- guishing left and right context for word embeddings. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 2 (Short Papers), pages 175-180, 2018.","title":"Directional skip-gram: Explicitly distinguishing left and right context for word embeddings","authors":"Yan Song, Shuming Shi, Jing Li, Haisong Zhang","year":"2018","venue":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Stab and Gurevych, 2017a] Christian Stab and Iryna Gurevych. Parsing argumentation structures in persuasive essays. Computational Linguistics, 43(3):619-659, 2017.","title":"Christian Stab and Iryna Gurevych. Parsing argumentation structures in persuasive essays","authors":"Gurevych Stab","year":"2017","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Stab and Gurevych, 2017b] Christian Stab and Iryna Gurevych. Recognizing insufficiently supported argu- ments in argumentative essays. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 980-990, 2017.","title":"Christian Stab and Iryna Gurevych. Recognizing insufficiently supported arguments in argumentative essays","authors":"Gurevych Stab","year":"2017","venue":"Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wachsmuth et al., 2016] Henning Wachsmuth, Khalid Al Khatib, and Benno Stein. Using argument mining to assess the argumentation quality of essays. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1680-1691, 2016.","title":"Using argument mining to assess the argumentation quality of essays","authors":"Khalid Henning Wachsmuth, Al Khatib, Benno Stein","year":"2016","venue":"Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2016] Zichao Yang, Diyi Yang, Chris Dyer, Xi- aodong He, Alex Smola, and Eduard Hovy. Hierarchical attention networks for document classification. In Pro- ceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pages 1480-1489, 2016.","title":"Hierarchical attention networks for document classification","authors":"Yang","year":"2016","venue":"Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}