{"type":"ScholarlyArticle","title":"Hierarchical Inter-Attention Network for Document Classification with Multi-Task Learning","authors":[{"name":"Bing Tian","slug":"bing-tian","given":"Bing","family":"Tian","affiliation":"RIIT , TNList , Dept. of Computer Science and Technology , Tsinghua University , Beijing , China."},{"name":"Yong Zhang","slug":"yong-zhang","given":"Yong","family":"Zhang","affiliation":"RIIT , TNList , Dept. of Computer Science and Technology , Tsinghua University , Beijing , China."},{"name":"Jin Wang","slug":"jin-wang","given":"Jin","family":"Wang","affiliation":"Computer Science Department , University of California , Los Angeles"},{"name":"Chunxiao Xing","slug":"chunxiao-xing","given":"Chunxiao","family":"Xing","affiliation":"RIIT , TNList , Dept. of Computer Science and Technology , Tsinghua University , Beijing , China."}],"abstract":"Document classification is an essential task in many real world applications. Existing approaches adopt both text semantics and document structure to obtain the document representation. However, these models usually require a large collection of annotated training instances, which are not always feasible, especially in low-resource settings. In this paper, we propose a multi-task learning framework to jointly train multiple related document classification tasks. We devise a hierarchical architecture to make use of the shared knowledge from all tasks to enhance the document representation of each task. We further propose an inter-attention approach to improve the task-specific modeling of documents with global information. Experimental results on 15 public datasets demonstrate the benefits of our proposed model.","doi":"10.24963/ijcai.2019/495","url":"https://ijcai.org/proceedings/2019/495","pdf":"https://ijcai.org/proceedings/2019/0495.pdf","markdown":"https://ijcai.org/proceedings/2019/495.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/495","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":495,"pages":{"first":3569,"last":3575},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Natural Language Processing: Text Classification","Natural Language Processing: Sentiment Analysis and Text Mining"],"videos":[],"references":[{"position":1,"raw":"[Blitzer et al., 2007] John Blitzer, Mark Dredze, and Fer- nando Pereira. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In ACL, 2007.","title":"Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification","authors":"John Blitzer, Mark Dredze, Fernando Pereira","year":"2007","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caruana, 1997] Rich Caruana. Multitask learning. Machine Learning, 28(1):41-75, 1997.","title":"Multitask learning","authors":"Caruana","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Junkun Chen, Xipeng Qiu, Pengfei Liu, and Xuanjing Huang. Meta multi-task learning for se- quence modeling. In AAAI, 2018.","title":"Meta multi-task learning for sequence modeling","authors":"Chen","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Collobert and Weston, 2008] Ronan Collobert and Jason Weston. A unified architecture for natural language pro- cessing: deep neural networks with multitask learning. In ICML, pages 160-167, 2008.","title":"A unified architecture for natural language processing: deep neural networks with multitask learning","authors":"Weston ; Collobert, Ronan Collobert, Jason Weston","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Firat et al., 2016] Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. Multi-way, multilingual neural machine translation with a shared attention mechanism. In NAACL- HLT, pages 866-875, 2016.","title":"Multi-way, multilingual neural machine translation with a shared attention mechanism","authors":"Orhan Firat, Kyunghyun Cho, Yoshua Bengio","year":"2016","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ha et al., 2016] David Ha, Andrew M. Dai, and Quoc V. Le. Hypernetworks. CoRR, abs/1609.09106, 2016.","title":null,"authors":"David Ha, Andrew M Dai, V Quoc, Le, Hypernetworks","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Johnson and Zhang, 2016] Rie Johnson and Tong Zhang. Supervised and semi-supervised text categorization using LSTM for region embeddings. In ICML, pages 526-534, 2016.","title":"Supervised and semi-supervised text categorization using LSTM for region embeddings","authors":"Rie Johnson, Tong Zhang","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Józefowicz et al., 2015] Rafal Józefowicz, Wojciech Zaremba, and Ilya Sutskever. An empirical exploration of recurrent network architectures. In ICML, pages 2342-2350, 2015.","title":"An empirical exploration of recurrent network architectures","authors":"Rafal Józefowicz, Wojciech Zaremba, Ilya Sutskever","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2015] Pengfei Liu, Xipeng Qiu, Xinchi Chen, Shiyu Wu, and Xuanjing Huang. Multi-timescale long short-term memory neural network for modelling sen- tences and documents. In EMNLP, pages 2326-2335, 2015.","title":"Multi-timescale long short-term memory neural network for modelling sentences and documents","authors":"Pengfei Liu, Xipeng Qiu, Xinchi Chen, Shiyu Wu, Xuanjing Huang","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2016a] Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. Deep multi-task learning with shared memory for text classification. In EMNLP, pages 118-127, 2016.","title":"Deep multi-task learning with shared memory for text classification","authors":"Liu","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2016b] Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. Recurrent neural network for text classification with multi-task learning. In IJCAI, pages 2873-2879, 2016.","title":"Recurrent neural network for text classification with multi-task learning","authors":"Liu","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/408"},{"position":13,"raw":"[Liu et al., 2017] Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. Adversarial multi-task learning for text classifi- cation. In ACL, pages 1-10, 2017.","title":"Adversarial multi-task learning for text classification","authors":"Pengfei Liu, Xipeng Qiu, Xuanjing Huang","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2019] Pengfei Liu, Jie Fu, Yue Dong, Xipeng Qiu, and Jackie Chi Kit Cheung. Multi-task learning over graph structures. In AAAI, 2019.","title":"Multi-task learning over graph structures","authors":"Pengfei Liu, Jie Fu, Yue Dong, Xipeng Qiu, Jackie Chi, Kit Cheung","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Luo et al., 2018] Ling Luo, Xiang Ao, Feiyang Pan, Jin Wang, Tong Zhao, Ningzi Yu, and Qing He. Beyond polar- ity: Interpretable financial sentiment analysis with hierar- chical query-driven attention. In IJCAI, pages 4244-4250, 2018.","title":"Beyond polarity: Interpretable financial sentiment analysis with hierarchical query-driven attention","authors":"Ling Luo, Xiang Ao, Feiyang Pan, Jin Wang, Tong Zhao, Ningzi Yu, Qing He","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/590"},{"position":16,"raw":"[Maas et al., 2011] Andrew L. Maas, Raymond E. Daly, Pe- ter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In ACL, pages 142-150, 2011.","title":"Learning word vectors for sentiment analysis","authors":"Andrew L Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, Christopher Potts","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[McCann et al., 2017] Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. Learned in transla- tion: Contextualized word vectors. In NIPS, pages 6297- 6308, 2017.","title":"Learned in translation: Contextualized word vectors","authors":"Bryan Mccann, James Bradbury, Caiming Xiong, Richard Socher","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rao et al., 2018] Jinfeng Rao, Ferhan Türe, and Jimmy Lin. Multi-task learning with neural networks for voice query understanding on an entertainment platform. In KDD, pages 636-645, 2018.","title":"Multi-task learning with neural networks for voice query understanding on an entertainment platform","authors":"Rao","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Socher et al., 2012] Richard Socher, Brody Huval, Christo- pher D. Manning, and Andrew Y. Ng. Semantic com- positionality through recursive matrix-vector spaces. In EMNLP, pages 1201-1211, 2012.","title":"Semantic compositionality through recursive matrix-vector spaces","authors":"Richard Socher, Brody Huval, Christopher D Manning, Andrew Y Ng","year":"2012","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Subramanian et al., 2018] Sandeep Subramanian, Adam Trischler, Yoshua Bengio, and Christopher J. Pal. Learn- ing general purpose distributed sentence representations via large scale multi-task learning. ICLR, 2018.","title":"Learning general purpose distributed sentence representations via large scale multi-task learning","authors":"Sandeep Subramanian, Adam Trischler, Yoshua Bengio, Christopher J Pal","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tang et al., 2015] Duyu Tang, Bing Qin, and Ting Liu. Learning semantic representations of users and products for document level sentiment classification. In ACL, pages 1014-1023, 2015.","title":"Learning semantic representations of users and products for document level sentiment classification","authors":"Tang","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang and Manning, 2012] Sida I. Wang and Christopher D. Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In ACL, pages 90-94, 2012.","title":"Baselines and bigrams: Simple, good sentiment and topic classification","authors":"Wang, Manning ; ] Sida, I Wang, Christopher D Manning","year":"2012","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2017] Jin Wang, Zhongyuan Wang, Dawei Zhang, and Jun Yan. Combining knowledge with deep convolutional neural networks for short text classification. In IJCAI, pages 2915-2921, 2017.","title":"Combining knowledge with deep convolutional neural networks for short text classification","authors":"Jin Wang, Zhongyuan Wang, Dawei Zhang, Jun Yan","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/406"},{"position":26,"raw":"[Yang et al., 2016] Zichao Yang, Diyi Yang, Chris Dyer, Xi- aodong He, Alexander J. Smola, and Eduard H. Hovy. Hi- erarchical attention networks for document classification. In NAACL-HLT, pages 1480-1489, 2016.","title":"Hierarchical attention networks for document classification","authors":"Yang","year":"2016","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang and Yang, 2017] Yu Zhang and Qiang Yang. A sur- vey on multi-task learning. CoRR, abs/1707.08114, 2017.","title":"A survey on multi-task learning","authors":"Yu Zhang, Qiang Yang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2012] Tianzhu Zhang, Bernard Ghanem, Si Liu, and Narendra Ahuja. Robust visual tracking via multi-task sparse learning. In CVPR, pages 2042-2049, 2012.","title":"Robust visual tracking via multi-task sparse learning","authors":"Tianzhu Zhang, Bernard Ghanem, Si Liu, Narendra Ahuja","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2015] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS, pages 649-657, 2015.","title":"Character-level convolutional networks for text classification","authors":"Xiang Zhang, Junbo Zhao, Yann Lecun","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zheng et al., 2018] Renjie Zheng, Junkun Chen, and Xipeng Qiu. Same representation, different attentions: Shareable sentence representation learning from multiple tasks. In IJCAI, pages 4616-4622, 2018.","title":"Same representation, different attentions: Shareable sentence representation learning from multiple tasks","authors":"Renjie Zheng, Junkun Chen, Xipeng Qiu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/642"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}