{"type":"ScholarlyArticle","title":"Earlier Attention? Aspect-Aware LSTM for Aspect-Based Sentiment Analysis","authors":[{"name":"Bowen Xing","slug":"bowen-xing","given":"Bowen","family":"Xing","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"},{"name":"Lejian Liao","slug":"lejian-liao","given":"Lejian","family":"Liao","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"},{"name":"Dandan Song","slug":"dandan-song","given":"Dandan","family":"Song","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"},{"name":"Jingang Wang","slug":"jingang-wang","given":"Jingang","family":"Wang","affiliation":"Meituan-Dianping Group"},{"name":"Fuzheng Zhang","slug":"fuzheng-zhang","given":"Fuzheng","family":"Zhang","affiliation":"Meituan-Dianping Group"},{"name":"Zhongyuan Wang","slug":"zhongyuan-wang","given":"Zhongyuan","family":"Wang","affiliation":"Meituan-Dianping Group"},{"name":"Heyan Huang","slug":"heyan-huang","given":"Heyan","family":"Huang","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"}],"abstract":"Aspect-based sentiment analysis (ABSA) aims to predict fine-grained sentiments of comments with respect to given aspect terms or categories. In previous ABSA methods, the importance of aspect has been realized and verified. Most existing LSTM-based models take aspect into account via the attention mechanism, where the attention weights are calculated after the context is modeled in the form of contextual vectors. However, aspect-related information may be already discarded and aspect-irrelevant information may be retained in classic LSTM cells in the context modeling process, which can be improved to generate more effective context representations. This paper proposes a novel variant of LSTM, termed as aspect-aware LSTM (AA-LSTM), which incorporates aspect information into LSTM cells in the context modeling stage before the attention mechanism. Therefore, our AA-LSTM can dynamically produce aspect-aware contextual representations. We experiment with several representative LSTM-based models by replacing the classic LSTM cells with the AA-LSTM cells. Experimental results on SemEval-2014 Datasets demonstrate the effectiveness of AA-LSTM.","doi":"10.24963/ijcai.2019/738","url":"https://ijcai.org/proceedings/2019/738","pdf":"https://ijcai.org/proceedings/2019/0738.pdf","markdown":"https://ijcai.org/proceedings/2019/738.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/738","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":738,"pages":{"first":5313,"last":5319},"keywords":["Natural Language Processing: Sentiment Analysis and Text Mining","Natural Language Processing: Text Classification"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. Computer Science, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2017] Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. Recurrent attention network on mem- ory for aspect sentiment analysis. In Empirical Methods in Natural Language Processing, pages 452-461, 2017.","title":"Recurrent attention network on memory for aspect sentiment analysis","authors":"Peng Chen, Zhongqian Sun, Lidong Bing, Wei Yang","year":"2017","venue":"Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Devamanyu et al., 2018] Hazarika Devamanyu, Poria Sou- janya, Vij Prateek, Krishnamurthy Gangeshwar, Cambria Erik, and Zimmermann Roger. Modeling inter-aspect de- pendencies for aspect-based sentiment analysis. In Con- ference of the North American Chapter of the Association for Computational Linguistics, pages 266-270, 2018.","title":"Modeling inter-aspect dependencies for aspect-based sentiment analysis","authors":"Hazarika Devamanyu, Poria Soujanya, Vij Prateek, Krishnamurthy Gangeshwar, Cambria Erik, Zimmermann Roger","year":"2018","venue":"Conference of the North American Chapter of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Diederik and Jimmy, 2014] Kingma Diederik and Ba Jim- my. Adam: A method for stochastic optimization. CoRR abs/1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Kingma Diederik, Ba Jimmy","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jeffrey et al., 2014] Pennington Jeffrey, Socher Richard, and Manning Christopher. Glove: Global vectors for word representation. In Empirical Methods in Natural Lan- guage Processing, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Pennington Jeffrey, Socher Richard, Manning Christopher","year":"2014","venue":"Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kiritchenko et al., 2014] Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherrt, and Saif Mohammad and. Nrc-canada- 2014: Detecting aspects and sentiment in customer re- views. In International Workshop on Semantic Evaluation, pages 437-442, 2014.","title":"Nrc-canada-2014: Detecting aspects and sentiment in customer reviews","authors":"Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherrt, Saif Mohammad","year":"2014","venue":"International Workshop on Semantic Evaluation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu and Zhang, 2017] Jiangming Liu and Yue Zhang. At- tention modeling for targeted sentiment. In Proceedings of the 15th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, page 572577, 2017.","title":"Attention modeling for targeted sentiment","authors":"Zhang Liu, Jiangming Liu, Yue Zhang","year":"2017","venue":"Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ma et al., 2017] Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. Interactive attention networks for aspect-level sentiment classification. In International Joint Conference on Artificial Intelligence, pages 4068- 4074, 2017.","title":"Interactive attention networks for aspect-level sentiment classification","authors":"Dehong Ma, Sujian Li, Xiaodong Zhang, Houfeng Wang","year":"2017","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/568"},{"position":10,"raw":"[Ma et al., 2018] Yukun Ma, Haiyun Peng, and Erik Cam- bria. Targeted aspect-based sentiment analysis via embed- ding commonsense knowledge into an attentive lstm. In AAAI Conference on Artificial Intelligence, pages 5876- 5883, 2018.","title":"Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm","authors":"Yukun Ma, Haiyun Peng, Erik Cambria","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Pontiki et al., 2014] Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Haris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. Semeval-2014 task 4: Aspect based sentiment analysis. In International Workshop on Semantic Evaluation, pages 27-35, 2014.","title":"Semeval-2014 task 4: Aspect based sentiment analysis","authors":"Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar","year":"2014","venue":"International Workshop on Semantic Evaluation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Tang et al., 2016a] Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. Effective lstms for target-dependent senti- ment classification. In International Conference on Com- putational Linguistics, pages 3298-3307, 2016.","title":"Effective lstms for target-dependent sentiment classification","authors":"Tang","year":"2016","venue":"International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Tang et al., 2016b] Duyu Tang, Bing Qin, and Ting Liu. As- pect level sentiment classification with deep memory net- work. In Empirical Methods in Natural Language Pro- cessing, pages 214-224, 2016.","title":"Aspect level sentiment classification with deep memory network","authors":"Tang","year":"2016","venue":"Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tay et al., 2017] Yi Tay, Tuan Luu Anh, and Hui Siu Che- ung. Dyadic memory networks for aspect-based sentimen- t analysis. In ACM on Conference on Information and Knowledge Management, CIKM, pages 107-116, 2017.","title":"Dyadic memory networks for aspect-based sentiment analysis","authors":"Yi Tay, Tuan Luu, Anh, Hui Siu Cheung","year":"2017","venue":"ACM on Conference on Information and Knowledge Management, CIKM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tay et al., 2018] Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In AAAI Conference on Artificial Intelligence, pages 5956-5963, 2018.","title":"Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis","authors":"Yi Tay, Anh Luu, Siu Tuan, Hui Cheung","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wagner et al., 2014] Joachim Wagner, Piyush Arora, San- tiago Cortes, Utsab Barman, Dasha Bogdanova, Jennifer Foster, and Lamia Tounsi. Dcu: Aspect-based polarity classification for semeval task 4. In International Work- shop on Semantic Evaluation, pages 223-229, 2014.","title":"Dcu: Aspect-based polarity classification for semeval task 4","authors":"Joachim Wagner, Piyush Arora, Santiago Cortes, Utsab Barman, Dasha Bogdanova, Jennifer Foster, Lamia Tounsi","year":"2014","venue":"International Workshop on Semantic Evaluation","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2016] Yequan Wang, Minlie Huang, and Li Zhao. Attention-based lstm for aspect-level sentiment classification. In Empirical Methods in Natural Language Processing, pages 606-615, 2016.","title":"Attention-based lstm for aspect-level sentiment classification","authors":"Wang","year":"2016","venue":"Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2018] Shuai Wang, Sahisnu Mazumder, Bing Liu, Mianwei Zhou, and Yi Chang. Target-sensitive mem- ory networks for aspect sentiment classification. In Annual Meeting of the Association for Computational Linguistics,, pages 957-967, 2018.","title":"Target-sensitive memory networks for aspect sentiment classification","authors":"Shuai Wang, Sahisnu Mazumder, Bing Liu, Mianwei Zhou, Yi Chang","year":"2018","venue":"Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xin et al., 2018] Li Xin, Bing Lidong, Lam Wai, and Shi Bei. Transformation networks for target-oriented senti- ment classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 946-956, 2018.","title":"Transformation networks for target-oriented sentiment classification","authors":"Li Xin, Bing Lidong, Lam Wai, Shi Bei","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2017] Jiacheng Xu, Xipeng Qiu, Kan Chen, and Xuanjing Huang. Knowledge graph representation with jointly structural and textual encoding. In International Joint Conference on Artificial Intelligence, pages 1318- 1324, 2017.","title":"Knowledge graph representation with jointly structural and textual encoding","authors":"Xu","year":"2017","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/183"},{"position":21,"raw":"[Xue and Li, 2018] Wei Xue and Tao Li. Aspect based senti- ment analysis with gated convolutional networks. In Annu- al Meeting of the Association for Computational Linguis- tics,, pages 2514-2523, 2018.","title":"Aspect based sentiment analysis with gated convolutional networks","authors":"Wei Xue, Tao Li","year":"2018","venue":"Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang and Mitchell, 2017] Bishan Yang and Tom M. Mitchell. Leveraging knowledge bases in lstms for improving machine reading. In Annual Meeting of the Association for Computational Linguistics, pages 1436-1446, 2017.","title":"Leveraging knowledge bases in lstms for improving machine reading","authors":"Yang, Mitchell ; ] Bishan Yang, Tom M Mitchell","year":"2017","venue":"Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2017] Min Yang, Wenting Tu, Jingxuan Wang, Fei Xu, and Xiaojun Chen. Attention based lstm for target dependent sentiment classification. In AAAI Conference on Artificial Intelligence, page 50135014, 2017.","title":"Attention based lstm for target dependent sentiment classification","authors":"Yang","year":"2017","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhen et al., 2017] Xu Zhen, Liu Bingquan, Wang Baox- un, Sun Chengjie, and Wang Xiaolong. Incorporating loose-structured knowledge into conversation modeling via recall-gate lstm. In International Joint Conference on Neural Networks, pages 3506-3513, 2017.","title":"Incorporating loose-structured knowledge into conversation modeling via recall-gate lstm","authors":"Liu Xu Zhen, Wang Bingquan, Sun Baoxun, Wang Chengjie, Xiaolong","year":"2017","venue":"International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}