{"type":"ScholarlyArticle","title":"Combining Eye Movements and EEG to Enhance Emotion Recognition","authors":[{"name":"Yifei Lu","slug":"yifei-lu","given":"Yifei","family":"Lu","affiliation":""},{"name":"Wei-Long Zheng","slug":"wei-long-zheng","given":"Wei-Long","family":"Zheng","affiliation":""},{"name":"Binbin Li","slug":"binbin-li","given":"Binbin","family":"Li","affiliation":""},{"name":"Bao-Liang Lu","slug":"bao-liang-lu","given":"Bao-Liang","family":"Lu","affiliation":""}],"abstract":"In this paper, we adopt a multimodal emotion recognition framework by combining eye movements and electroencephalography (EEG) to enhance emotion recognition. The main contributions of this paper are twofold. a) We investigate sixteen eye movements related to emotions and identify the intrinsic patterns of these eye movements for three emotional states: positive, neutral and negative. b) We examine various modality fusion strategies for integrating users external subconscious behaviors and internal cognitive states and reveal that the characteristics of eye movements and EEG are complementary to emotion recognition. Experiment results demonstrate that modality fusion could significantly improve emotion recognition accuracy in comparison with single modality. The best accuracy achieved by fuzzy integral fusion strategy is 87.59%, whereas the accuracies of solely using eye movements and EEG data are 77.80% and 78.51%, respectively.","doi":null,"url":"https://ijcai.org/proceedings/2015/169","pdf":"https://ijcai.org/proceedings/2015/0169.pdf","markdown":"https://ijcai.org/proceedings/2015/169.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/169","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":169,"pages":{"first":1170,"last":1175},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}