{"type":"ScholarlyArticle","title":"Modeling both Context- and Speaker-Sensitive Dependence for Emotion Detection in Multi-speaker Conversations","authors":[{"name":"Dong Zhang","slug":"dong-zhang","given":"Dong","family":"Zhang","affiliation":"School of Computer Science and Technology , Soochow University , China"},{"name":"Liangqing Wu","slug":"liangqing-wu","given":"Liangqing","family":"Wu","affiliation":"School of Computer Science and Technology , Soochow University , China"},{"name":"Changlong Sun","slug":"changlong-sun","given":"Changlong","family":"Sun","affiliation":"Alibaba Group , China"},{"name":"Shoushan Li","slug":"shoushan-li","given":"Shoushan","family":"Li","affiliation":"School of Computer Science and Technology , Soochow University , China"},{"name":"Qiaoming Zhu","slug":"qiaoming-zhu","given":"Qiaoming","family":"Zhu","affiliation":"School of Computer Science and Technology , Soochow University , China"},{"name":"Guodong Zhou","slug":"guodong-zhou","given":"Guodong","family":"Zhou","affiliation":"School of Computer Science and Technology , Soochow University , China"}],"abstract":"Recently, emotion detection in conversations becomes a hot research topic in the Natural Language Processing community. In this paper, we focus on emotion detection in multi-speaker conversations instead of traditional two-speaker conversations in existing studies. Different from non-conversation text, emotion detection in conversation text has one specific challenge in modeling the context-sensitive dependence. Besides, emotion detection in multi-speaker conversations endorses another specific challenge in modeling the speaker-sensitive dependence. To address above two challenges, we propose a conversational graph-based convolutional neural network. On the one hand, our approach represents each utterance and each speaker as a node. On the other hand, the context-sensitive dependence is represented by an undirected edge between two utterances nodes from the same conversation and the speaker-sensitive dependence is represented by an undirected edge between an utterance node and its speaker node. In this way, the entire conversational corpus can be symbolized as a large heterogeneous graph and the emotion detection task can be recast as a classification problem of the utterance nodes in the graph. The experimental results on a multi-modal and multi-speaker conversation corpus demonstrate the great effectiveness of the proposed approach.","doi":"10.24963/ijcai.2019/752","url":"https://ijcai.org/proceedings/2019/752","pdf":"https://ijcai.org/proceedings/2019/0752.pdf","markdown":"https://ijcai.org/proceedings/2019/752.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/752","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":752,"pages":{"first":5415,"last":5421},"keywords":["Natural Language Processing: Sentiment Analysis and Text Mining","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Alm et al., 2005] Cecilia Ovesdotter Alm, Dan Roth, and Richard Sproat. Emotions from text: Machine learning for text-based emotion prediction. In HLT/EMNLP, pages 579-586, 2005.","title":"Emotions from text: Machine learning for text-based emotion prediction","authors":"Cecilia Ovesdotter Alm, Dan Roth, Richard Sproat","year":"2005","venue":"HLT/EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bastings et al., 2017] Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Sima'an. Graph convolutional encoders for syntax-aware neural machine translation. In EMNLP, pages 1957-1967, 2017.","title":"Graph convolutional encoders for syntax-aware neural machine translation","authors":"Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, Khalil Sima'an","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Busso et al., 2008] Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jean- nette N. Chang, Sungbok Lee, and Shrikanth Narayanan. IEMOCAP: interactive emotional dyadic motion cap- ture database. Language Resources and Evaluation, 42(4):335-359, 2008.","title":"IEMOCAP: interactive emotional dyadic motion capture database","authors":"Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, Shrikanth Narayanan","year":"2008","venue":"Language Resources and Evaluation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2018] HongYun Cai, Vincent W. Zheng, and Kevin Chen-Chuan Chang. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Trans. Knowl. Data Eng., 30(9):1616-1637, 2018.","title":"A comprehensive survey of graph embedding: Problems, techniques, and applications","authors":"Hongyun Cai, Vincent W Zheng, Kevin Chen, -Chuan Chang","year":"2018","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cambria et al., 2017] Erik Cambria, Soujanya Poria, Alexander F. Gelbukh, and Mike Thelwall. Sentiment analysis is a big suitcase. IEEE Intelligent Systems, 32(6):74-80, 2017.","title":"Sentiment analysis is a big suitcase","authors":"Cambria","year":"2017","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Felbo et al., 2017] Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. Using mil- lions of emoji occurrences to learn any-domain represen- tations for detecting sentiment, emotion and sarcasm. In EMNLP, pages 1615-1625, 2017.","title":"Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm","authors":"Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, Sune Lehmann","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gu et al., 2018] Yue Gu, Kangning Yang, Shiyu Fu, Shuhong Chen, Xinyu Li, and Ivan Marsic. Hybrid atten- tion based multimodal network for spoken language clas- sification. In COLING, pages 2379-2390, 2018.","title":"Hybrid attention based multimodal network for spoken language classification","authors":"Yue Gu, Kangning Yang, Shiyu Fu, Shuhong Chen, Xinyu Li, Ivan Marsic","year":"2018","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hazarika et al., 2018a] Devamanyu Hazarika, Soujanya Po- ria, Rada Mihalcea, Erik Cambria, and Roger Zimmer- mann. ICON: interactive conversational memory network for multimodal emotion detection. In EMNLP, pages 2594-2604, 2018.","title":"ICON: interactive conversational memory network for multimodal emotion detection","authors":"Hazarika","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hazarika et al., 2018b] Devamanyu Hazarika, Soujanya Po- ria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmermann. Conversational memory net- work for emotion recognition in dyadic dialogue videos. In NAACL-HLT, pages 2122-2132, 2018.","title":"Conversational memory network for emotion recognition in dyadic dialogue videos","authors":"Hazarika","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hsu et al., 2018] Chao-Chun Hsu, Sheng-Yeh Chen, Chuan-Chun Kuo, Ting-Hao K. Huang, and Lun-Wei Ku. Emotionlines: An emotion corpus of multi-party conversations. In LREC, 2018.","title":"Emotionlines: An emotion corpus of multi-party conversations","authors":"Chao-Chun, Sheng-Yeh Hsu, Chuan-Chun Chen, Kuo, K Ting-Hao, Lun-Wei Huang, Ku","year":"2018","venue":"LREC","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2015] Shoushan Li, Lei Huang, Rong Wang, and Guodong Zhou. Sentence-level emotion classification with label and context dependence. In ACL, pages 1045-1053, 2015.","title":"Sentence-level emotion classification with label and context dependence","authors":"Shoushan Li, Lei Huang, Rong Wang, Guodong Zhou","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Majumder et al., 2019] Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander F. Gel- bukh, and Erik Cambria. Dialoguernn: An attentive RNN for emotion detection in conversations. In AAAI, 2019.","title":"Dialoguernn: An attentive RNN for emotion detection in conversations","authors":"Majumder","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Marcheggiani and Titov, 2017] Diego Marcheggiani and Ivan Titov. Encoding sentences with graph convolutional networks for semantic role labeling. In EMNLP, pages 1506-1515, 2017.","title":"Diego Marcheggiani and Ivan Titov. Encoding sentences with graph convolutional networks for semantic role labeling","authors":"Titov Marcheggiani","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[McKeown et al., 2012] Gary McKeown, Michel Franc ¸ois Valstar, Roddy Cowie, Maja Pantic, and Marc Schröder. The SEMAINE database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent. IEEE Trans. Affective Computing, 3(1):5-17, 2012.","title":"The SEMAINE database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent","authors":"Gary Mckeown, Michel Franc ¸ois, Roddy Valstar, Maja Cowie, Marc Pantic, Schröder","year":"2012","venue":"IEEE Trans. Affective Computing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Picard, 2010] Rosalind W. Picard. Affective computing: From laughter to ieee. IEEE Transactions on Affective Computing, 1(1):11-17, 2010.","title":"Affective computing: From laughter to ieee","authors":"Rosalind W Picard","year":"2010","venue":"IEEE Transactions on Affective Computing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Poria et al., 2017] Soujanya Poria, Erik Cambria, Deva- manyu Hazarika, Navonil Majumder, Amir Zadeh, and Louis-Philippe Morency. Context-dependent sentiment analysis in user-generated videos. In ACL, pages 873-883, 2017.","title":"Context-dependent sentiment analysis in user-generated videos","authors":"Poria","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Poria et al., 2019] Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. MELD: A multimodal multi-party dataset for emotion recognition in conversations. In ACL, 2019.","title":"MELD: A multimodal multi-party dataset for emotion recognition in conversations","authors":"Poria","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Richards, 2010] Jane M. Richards. The cognitive conse- quences of concealing feelings. Current Directions in Psy- chological Science, 13(4):131-134, 2010.","title":"The cognitive consequences of concealing feelings","authors":"Jane M Richards","year":"2010","venue":"Current Directions in Psychological Science","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Skianis et al., 2018] Konstantinos Skianis, Fragkiskos D. Malliaros, and Michalis Vazirgiannis. Fusing docu- ment, collection and label graph-based representations with word embeddings for text classification. In NAACL- HLT, pages 49-58, 2018.","title":"Fusing document, collection and label graph-based representations with word embeddings for text classification","authors":"Konstantinos Skianis, Fragkiskos D Malliaros, Michalis Vazirgiannis","year":"2018","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2016] Jin Wang, Liang-Chih Yu, K. Robert Lai, and Xue-Jie Zhang. Dimensional sentiment analysis using a regional CNN-LSTM model. In ACL, 2016.","title":"Dimensional sentiment analysis using a regional CNN-LSTM model","authors":"Jin Wang, Liang-Chih Yu, K Robert Lai, Xue-Jie Zhang","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yao et al., 2019] Liang Yao, Chengsheng Mao, and Yuan Luo. Graph convolutional networks for text classification. In AAAI, 2019.","title":"Graph convolutional networks for text classification","authors":"Liang Yao, Chengsheng Mao, Yuan Luo","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Young et al., 2018] Tom Young, Erik Cambria, Iti Chaturvedi, Hao Zhou, Subham Biswas, and Minlie Huang. Augmenting end-to-end dialogue systems with commonsense knowledge. In AAAI, pages 4970-4977, 2018.","title":"Augmenting end-to-end dialogue systems with commonsense knowledge","authors":"Tom Young, Erik Cambria, Iti Chaturvedi, Hao Zhou, Subham Biswas, Minlie Huang","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zadeh et al., 2018] Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, and Louis- Philippe Morency. Memory fusion network for multi-view sequential learning. In AAAI, pages 5634-5641, 2018.","title":"Memory fusion network for multi-view sequential learning","authors":"Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, Louis-Philippe Morency","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}