Combining Eye Movements and EEG to Enhance Emotion Recognition

Lu, Yifei; Zheng, Wei-Long; Li, Binbin; Lu, Bao-Liang

Combining Eye Movements and EEG to Enhance Emotion Recognition

Yifei Lu, Wei-Long Zheng, Binbin Li, Bao-Liang Lu

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 1170-1175.

Abstract

In this paper, we adopt a multimodal emotion recognition framework by combining eye movements and electroencephalography (EEG) to enhance emotion recognition. The main contributions of this paper are twofold. a) We investigate sixteen eye movements related to emotions and identify the intrinsic patterns of these eye movements for three emotional states: positive, neutral and negative. b) We examine various modality fusion strategies for integrating users external subconscious behaviors and internal cognitive states and reveal that the characteristics of eye movements and EEG are complementary to emotion recognition. Experiment results demonstrate that modality fusion could significantly improve emotion recognition accuracy in comparison with single modality. The best accuracy achieved by fuzzy integral fusion strategy is 87.59%, whereas the accuracies of solely using eye movements and EEG data are 77.80% and 78.51%, respectively.

Yifei Lu, Wei-Long Zheng, Binbin Li, and Bao-Liang Lu. 2015. Combining Eye Movements and EEG to Enhance Emotion Recognition. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 1170–1175. https://ijcai.org/proceedings/2015/169

@inproceedings{ijcai2015p169,
  title     = {Combining Eye Movements and EEG to Enhance Emotion Recognition},
  author    = {Lu, Yifei and Zheng, Wei-Long and Li, Binbin and Lu, Bao-Liang},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {1170--1175},
  year      = {2015},
  month     = {7},
  note      = {Main Track},
  url       = {https://ijcai.org/proceedings/2015/169},
}

TY  - CPAPER
TI  - Combining Eye Movements and EEG to Enhance Emotion Recognition
AU  - Lu, Yifei
AU  - Zheng, Wei-Long
AU  - Li, Binbin
AU  - Lu, Bao-Liang
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 1170
EP  - 1175
UR  - https://ijcai.org/proceedings/2015/169
ER  -