{"type":"ScholarlyArticle","title":"Multi-Domain Sentiment Classification Based on Domain-Aware Embedding and Attention","authors":[{"name":"Yitao Cai","slug":"yitao-cai","given":"Yitao","family":"Cai","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University; Center for Data Science , Peking University"},{"name":"Xiaojun Wan","slug":"xiaojun-wan","given":"Xiaojun","family":"Wan","affiliation":"Institute of Computer Science and Technology , Peking University The MOE Key Laboratory of Computational Linguistics , Peking University; Center for Data Science , Peking University"}],"abstract":"Sentiment classification is a fundamental task in NLP. However, as revealed by many researches, sentiment classification models are highly domain-dependent. It is worth investigating to leverage data from different domains to improve the classification performance in each domain. In this work, we propose a novel completely-shared multi-domain neural sentiment classification model to learn domain-aware word embeddings and make use of domain-aware attention mechanism. Our model first utilizes BiLSTM for domain classification and extracts domain-specific features for words, which are then combined with general word embeddings to form domain-aware word embeddings. Domain-aware word embeddings are fed into another BiLSTM to extract sentence features. The domain-aware attention mechanism is used for selecting significant features, by using the domain-aware sentence representation as the query vector. Evaluation results on public datasets with 16 different domains demonstrate the efficacy of our proposed model. Further experiments show the generalization ability and the transferability of our model.","doi":"10.24963/ijcai.2019/681","url":"https://ijcai.org/proceedings/2019/681","pdf":"https://ijcai.org/proceedings/2019/0681.pdf","markdown":"https://ijcai.org/proceedings/2019/681.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/681","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":681,"pages":{"first":4904,"last":4910},"keywords":["Natural Language Processing: Text Classification","Natural Language Processing: Sentiment Analysis and Text Mining"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2003] Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilis- tic language model. Journal of machine learning research, 3(Feb):1137-1155, 2003.","title":"A neural probabilistic language model","authors":"Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Jauvin","year":"2003","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Caruana, 1997] Rich Caruana. Multitask learning. Machine learning, 28(1):41-75, 1997.","title":"Multitask learning","authors":"Caruana","year":"1997","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen and Cardie, 2018] Xilun Chen and Claire Cardie. Multinomial adversarial networks for multi-domain text classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume 1 (Long Papers), volume 1, pages 1226-1240, 2018.","title":"Multinomial adversarial networks for multi-domain text classification","authors":"Chen, Cardie ;, Xilun Chen, Claire Cardie","year":"2018","venue":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choi and Cardie, 2008] Yejin Choi and Claire Cardie. Learning with compositional semantics as structural infer- ence for subsentential sentiment analysis. In Proceedings of the 2008 Conference on Empirical Methods in Natu- ral Language Processing, pages 793-801. Association for Computational Linguistics, 2008.","title":"Learning with compositional semantics as structural inference for subsentential sentiment analysis","authors":"Cardie ; ] Yejin Choi, Claire Choi, Cardie","year":"2008","venue":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2017] Zheng Li, Yu Zhang, Ying Wei, Yuxiang Wu, and Qiang Yang. End-to-end adversarial memory net- work for cross-domain sentiment classification. In Pro- ceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2017), 2017.","title":"End-to-end adversarial memory network for cross-domain sentiment classification","authors":"Zheng Li, Yu Zhang, Ying Wei, Yuxiang Wu, Qiang Yang","year":"2017","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2017)","doi":null,"url":null,"internal_url":"/proceedings/2017/311"},{"position":10,"raw":"[Liu et al., 2016a] Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. Deep multi-task learning with shared memory. arXiv preprint arXiv:1609.07222, 2016.","title":"Deep multi-task learning with shared memory","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2016b] Pengfei Liu, Xipeng Qiu, and Xuan- jing Huang. Recurrent neural network for text clas- sification with multi-task learning. arXiv preprint arXiv:1605.05101, 2016.","title":"Recurrent neural network for text classification with multi-task learning","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2016/408"},{"position":12,"raw":"[Liu et al., 2017] Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. Adversarial multi-task learning for text classifica- tion. In Proceedings of the 55th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 1-10. Association for Computational Lin- guistics, 2017.","title":"Adversarial multi-task learning for text classification","authors":"Pengfei Liu, Xipeng Qiu, Xuanjing Huang","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2018] Qi Liu, Yue Zhang, and Jiangming Liu. Learning domain representation for multi-domain senti- ment classification. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume 1 (Long Papers), volume 1, pages 541-550, 2018.","title":"Learning domain representation for multi-domain senti-ment classification","authors":"Qi Liu, Yue Zhang, Jiangming Liu","year":"2018","venue":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luong et al., 2015] Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025, 2015.","title":"Effective approaches to attention-based neural machine translation","authors":"Minh-Thang Luong, Hieu Pham, Christopher D Manning","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[McCallum et al., 1998] Andrew McCallum, Kamal Nigam, et al. A comparison of event models for naive bayes text classification. In AAAI-98 workshop on learning for text categorization, volume 752, pages 41-48. Citeseer, 1998.","title":"A comparison of event models for naive bayes text classification","authors":"Andrew Mccallum, Kamal Nigam","year":"1998","venue":"AAAI-98 workshop on learning for text categorization","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Peters et al., 2018] Matthew E Peters, Mark Neumann, Mo- hit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word repre- sentations. arXiv preprint arXiv:1802.05365, 2018.","title":"Deep contextualized word representations","authors":"Mark Matthew E Peters, Mohit Neumann, Matt Iyyer, Christopher Gardner, Kenton Clark, Luke Lee, Zettlemoyer","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Radford et al., 2018] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. URL https://s3- us-west-2. amazonaws. com/openai-assets/research- covers/languageunsupervised/language understanding paper. pdf, 2018.","title":"Improving language understanding by generative pre-training","authors":"Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever","year":"2018","venue":null,"doi":null,"url":"https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/languageunsupervised/languageunderstandingpaper.pdf","internal_url":null},{"position":19,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yang et al., 2016] Zichao Yang, Diyi Yang, Chris Dyer, Xi- aodong He, Alex Smola, and Eduard Hovy. Hierarchical attention networks for document classification. In Pro- ceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480-1489, 2016.","title":"Hierarchical attention networks for document classification","authors":"Yang","year":"2016","venue":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yuan et al., 2018] Zhigang Yuan, Sixing Wu, Fangzhao Wu, Junxin Liu, and Yongfeng Huang. Domain attention model for multi-domain sentiment classification. Knowledge- Based Systems, 155:1-10, 2018.","title":"Domain attention model for multi-domain sentiment classification","authors":"Zhigang Yuan, Sixing Wu, Fangzhao Wu, Junxin Liu, Yongfeng Huang","year":"2018","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2014] Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial land- mark detection by deep multi-task learning. In European conference on computer vision, pages 94-108. Springer, 2014.","title":"Facial landmark detection by deep multi-task learning","authors":"Zhang","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2018] Lei Zhang, Shuai Wang, and Bing Liu. Deep learning for sentiment analysis : A survey. CoRR, abs/1801.07883, 2018.","title":"Deep learning for sentiment analysis : A survey","authors":"Lei Zhang, Shuai Wang, Bing Liu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zheng et al., 2018] Renjie Zheng, Junkun Chen, and Xipeng Qiu. Same representation, different attentions: Shareable sentence representation learning from multiple tasks. arXiv preprint arXiv:1804.08139, 2018.","title":"Same representation, different attentions: Shareable sentence representation learning from multiple tasks","authors":"Renjie Zheng, Junkun Chen, Xipeng Qiu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2018/642"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}