{"type":"ScholarlyArticle","title":"Translations as Additional Contexts for Sentence Classification","authors":[{"name":"Reinald Kim Amplayo","slug":"reinald-kim-amplayo","given":"Reinald Kim","family":"Amplayo","affiliation":"† Yonsei University , Seoul , South Korea"},{"name":"Kyungjae Lee","slug":"kyungjae-lee","given":"Kyungjae","family":"Lee","affiliation":"† Yonsei University , Seoul , South Korea"},{"name":"Jinyoung Yeo","slug":"jinyoung-yeo","given":"Jinyoung","family":"Yeo","affiliation":"‡ Pohang University of Science and Technology , Pohang , South Korea"},{"name":"Seung-won Hwang","slug":"seung-won-hwang","given":"Seung-won","family":"Hwang","affiliation":"† Yonsei University , Seoul , South Korea"}],"abstract":"In sentence classification tasks, additional contexts, such as the neighboring sentences, may improve the accuracy of the classifier. However, such contexts are domain-dependent and thus cannot be used for another classification task with an inappropriate domain. In contrast, we propose the use of translated sentences as domain-free context that is always available regardless of the domain. We find that naive feature expansion of translations gains only marginal improvements and may decrease the performance of the classifier, due to possible inaccurate translations thus producing noisy sentence vectors. To this end, we present multiple context fixing attachment (MCFA), a series of modules attached to multiple sentence vectors to fix the noise in the vectors using the other sentence vectors as context. We show that our method performs competitively compared to previous models, achieving best classification performance on multiple data sets. We are the first to use translations as domain-free contexts for sentence classification.","doi":"10.24963/ijcai.2018/550","url":"https://ijcai.org/proceedings/2018/550","pdf":"https://ijcai.org/proceedings/2018/0550.pdf","markdown":"https://ijcai.org/proceedings/2018/550.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/550","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":550,"pages":{"first":3955,"last":3961},"keywords":["Natural Language Processing: Text Classification","Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Sentiment Analysis and Text Mining"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bojanowski et al., 2016] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606, 2016.","title":"Enriching word vectors with subword information","authors":"Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug):2493-2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fu et al., 2014] Guohong Fu, Yu He, Jiaying Song, and Chaoyue Wang. Improving chinese sentence polarity clas- sification via opinion paraphrasing. CLP 2014, page 35, 2014.","title":"Improving chinese sentence polarity classification via opinion paraphrasing","authors":"Guohong Fu, Yu He, Jiaying Song, Chaoyue Wang","year":"2014","venue":"CLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hill et al., 2016] Felix Hill, Kyunghyun Cho, and Anna Ko- rhonen. Learning distributed representations of sentences from unlabelled data. arXiv preprint arXiv:1602.03483, 2016.","title":"Learning distributed representations of sentences from unlabelled data","authors":"Felix Hill, Kyunghyun Cho, Anna Korhonen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hill et al., 2017] Felix Hill, Kyunghyun Cho, Sébastien Jean, and Yoshua Bengio. The representational geometry of word meanings acquired by neural machine translation models. Machine Translation, pages 1-16, 2017.","title":"The representational geometry of word meanings acquired by neural machine translation models","authors":"Felix Hill, Kyunghyun Cho, Sébastien Jean, Yoshua Bengio","year":"2017","venue":"Machine Translation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu and Liu, 2004] Minqing Hu and Bing Liu. Mining and summarizing customer reviews. In SIGKDD, pages 168- 177. ACM, 2004.","title":"Mining and summarizing customer reviews","authors":"Liu Hu, Minqing Hu, Bing Liu","year":"2004","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2012] Eric H Huang, Richard Socher, Christo- pher D Manning, and Andrew Y Ng. Improving word rep- resentations via global context and multiple word proto- types. In ACL, pages 873-882. Association for Computa- tional Linguistics, 2012.","title":"Improving word representations via global context and multiple word prototypes","authors":"Eric H Huang, Richard Socher, Christopher D Manning, Andrew Y Ng","year":"2012","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Johnson et al., 2016] Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. Google's multilingual neural machine translation system: enabling zero-shot translation. arXiv preprint arXiv:1611.04558, 2016.","title":"Google's multilingual neural machine translation system: enabling zero-shot translation","authors":"Melvin Johnson, Mike Schuster, V Quoc, Maxim Le, Yonghui Krikun, Zhifeng Wu, Nikhil Chen, Fernanda Thorat, Martin Viégas, Greg Wattenberg, Corrado","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joulin et al., 2017] Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for effi- cient text classification. In EACL, 2017.","title":"Bag of tricks for efficient text classification","authors":"Armand Joulin, Edouard Grave, Piotr Bojanowski, Tomas Mikolov","year":"2017","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. In EMNLP, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li and Roth, 2002] Xin Li and Dan Roth. Learning ques- tion classifiers. In COLING, pages 1-7. Association for Computational Linguistics, 2002.","title":"Learning question classifiers","authors":"Roth ; Li, Xin Li, Dan Roth","year":"2002","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2015] Rui Lin, Shujie Liu, Muyun Yang, Mu Li, Ming Zhou, and Sheng Li. Hierarchical recurrent neural network for document modeling. In EMNLP, pages 899- 907, 2015.","title":"Hierarchical recurrent neural network for document modeling","authors":"Rui Lin, Shujie Liu, Muyun Yang, Mu Li, Ming Zhou, Sheng Li","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2015] Mingbo Ma, Liang Huang, Bing Xiang, and Bowen Zhou. Dependency-based convolutional neu- ral networks for sentence embedding. arXiv preprint arXiv:1507.01839, 2015.","title":"Dependency-based convolutional neural networks for sentence embedding","authors":"Mingbo Ma, Liang Huang, Bing Xiang, Bowen Zhou","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[McCann et al., 2017] Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. Learned in trans- lation: Contextualized word vectors. arXiv preprint arXiv:1708.00107, 2017.","title":"Learned in translation: Contextualized word vectors","authors":"Bryan Mccann, James Bradbury, Caiming Xiong, Richard Socher","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mimno et al., 2011] David Mimno, Hanna M Wallach, Ed- mund Talley, Miriam Leenders, and Andrew McCallum. Optimizing semantic coherence in topic models. In EMNLP, pages 262-272. Association for Computational Linguistics, 2011.","title":"Optimizing semantic coherence in topic models","authors":"David Mimno, Edmund Hanna M Wallach, Miriam Talley, Andrew Leenders, Mccallum","year":"2011","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pang and Lee, 2004] Bo Pang and Lillian Lee. A sentimen- tal education: Sentiment analysis using subjectivity sum- marization based on minimum cuts. In ACL, page 271. Association for Computational Linguistics, 2004.","title":"A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2004","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pang and Lee, 2005] Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In ACL, pages 115-124. As- sociation for Computational Linguistics, 2005.","title":"Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2005","venue":"Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pang and Lee, 2007] Bo Pang and Lillian Lee. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2:1-135, 2007.","title":"Opinion mining and sentiment analysis","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2007","venue":"Foundations and Trends in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hin- ton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2015] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS, pages 649-657, 2015.","title":"Character-level convolutional networks for text classification","authors":"Xiang Zhang, Junbo Zhao, Yann Lecun","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2016] Rui Zhang, Honglak Lee, and Dragomir Radev. Dependency sensitive convolutional neural net- works for modeling sentences and documents. arXiv preprint arXiv:1611.02361, 2016.","title":"Dependency sensitive convolutional neural networks for modeling sentences and documents","authors":"Rui Zhang, Honglak Lee, Dragomir Radev","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2015] Han Zhao, Zhengdong Lu, and Pascal Poupart. Self-adaptive hierarchical sentence model. In IJCAI, pages 4069-4076, 2015.","title":"Self-adaptive hierarchical sentence model","authors":"Han Zhao, Zhengdong Lu, Pascal Poupart","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/571"},{"position":25,"raw":"[Zhao et al., 2017] Rui Zhao, Kezhi Mao, Rui Zhao, and Kezhi Mao. Topic-aware deep compositional models for sentence classification. IEEE/ACM Transactions on Audio, Speech and Language Processing, 25(2):248-260, 2017.","title":"Topic-aware deep compositional models for sentence classification","authors":"Rui Zhao, Kezhi Mao, Rui Zhao, Kezhi Mao","year":"2017","venue":"IEEE/ACM Transactions on Audio, Speech and Language Processing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}