{"type":"ScholarlyArticle","title":"Online ECG Emotion Recognition for Unknown Subjects via Hypergraph-Based Transfer Learning","authors":[{"name":"Yalan Ye","slug":"yalan-ye","given":"Yalan","family":"Ye","affiliation":"University of Electronic Science and Technology of China"},{"name":"Tongjie Pan","slug":"tongjie-pan","given":"Tongjie","family":"Pan","affiliation":"University of Electronic Science and Technology of China"},{"name":"Qianhe Meng","slug":"qianhe-meng","given":"Qianhe","family":"Meng","affiliation":"University of Electronic Science and Technology of China"},{"name":"Jingjing Li","slug":"jingjing-li","given":"Jingjing","family":"Li","affiliation":"University of Electronic Science and Technology of China"},{"name":"Li Lu","slug":"li-lu","given":"Li","family":"Lu","affiliation":"University of Electronic Science and Technology of China"}],"abstract":"Electrocardiogram (ECG) signal based cross-subject emotion recognition methods reduce the influence of individual differences using domain adaptation (DA) techniques. These methods generally assume that the entire unlabeled data of unknown target subjects are available in training phase.  However, this assumption does not hold in some practical scenarios where the data of target subjects arrive one by one in an online manner instead of being acquired at a time. Thus, existing DA methods cannot be directly applied in this case since the unknown target data is inaccessible in training phase. To tackle the problem, we propose a novel online cross-subject ECG emotion recognition method leveraging hypergraph-based online transfer learning (HOTL). Specifically, the proposed hypergraph structure is capable of learning the high-order correlation among data, such that the recognition model trained on source subjects can be more effectively generalized to target subjects. Meanwhile, the structure can be easily updated by adding a hyperedge which connects a newly coming sample with the current hypergraph, resulting in further reduce the individual differences in online manner without re-training the model. Consequently, HOTL can effectively deal with the online cross-subject scenario where unknown target ECG data arrive one by one and varying overtime. Extensive experiments conducted on the Amigos dataset validate the superiority of the proposed method.","doi":"10.24963/ijcai.2022/509","url":"https://ijcai.org/proceedings/2022/509","pdf":"https://ijcai.org/proceedings/2022/0509.pdf","markdown":"https://ijcai.org/proceedings/2022/509.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/509","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":509,"pages":{"first":3666,"last":3672},"keywords":["Machine Learning: Online Learning","Humans and AI: Personalization and User Modeling","Machine Learning: Multi-task and Transfer Learning","Machine Learning: Feature Extraction, Selection and Dimensionality Reduction"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986811","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984954","length_sec":333,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Agrafioti et al., 2011] Foteini Agrafioti, Dimitris Hatzi- nakos, and Adam K Anderson. Ecg pattern analysis for emotion detection. IEEE TAC, 3(1):102-115, 2011.","title":"Foteini Agrafioti, Dimitris Hatzinakos, and Adam K Anderson. Ecg pattern analysis for emotion detection","authors":"Agrafioti","year":"2011","venue":"IEEE TAC","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Correa et al., 2018] Juan Abdon Miranda Correa, Mo- jtaba Khomami Abadi, Niculae Sebe, and Ioannis Patras. Amigos: A dataset for affect, personality and mood re- search on individuals and groups. IEEE TAC, 2018.","title":"Amigos: A dataset for affect, personality and mood research on individuals and groups","authors":"Correa","year":"2018","venue":"IEEE TAC","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Crammer et al., 2006] Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. Online passive aggressive algorithms. Journal of Machine Learn- ing Research, 2006.","title":"Online passive aggressive algorithms","authors":"Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, Yoram Singer","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Du et al., 2020] Xiaobing Du, Cuixia Ma, Guanhua Zhang, Jinyao Li, Yu-Kun Lai, Guozhen Zhao, Xiaoming Deng, Yong-Jin Liu, and Hongan Wang. An efficient lstm net- work for emotion recognition from multichannel eeg sig- nals. IEEE TAC, 2020.","title":"An efficient lstm network for emotion recognition from multichannel eeg signals","authors":"Xiaobing Du, Cuixia Ma, Guanhua Zhang, Jinyao Li, Yu-Kun Lai, Guozhen Zhao, Xiaoming Deng, Yong-Jin Liu, Hongan Wang","year":"2020","venue":"IEEE TAC","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gao et al., 2020] Yue Gao, Zizhao Zhang, Haojie Lin, Xibin Zhao, Shaoyi Du, and Changqing Zou. Hypergraph learning: Methods and practices. IEEE TPAMI, 2020.","title":"Hypergraph learning: Methods and practices","authors":"Yue Gao, Zizhao Zhang, Haojie Lin, Xibin Zhao, Shaoyi Du, Changqing Zou","year":"2020","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2021] Wenwen He, Yalan Ye, Yunxia Li, Tongjie Pan, and Li Lu. Online cross-subject emotion recognition from ecg via unsupervised domain adaptation. In EMBC, pages 1001-1005. IEEE, 2021.","title":"Online cross-subject emotion recognition from ecg via unsupervised domain adaptation","authors":"Wenwen He, Yalan Ye, Yunxia Li, Tongjie Pan, Li Lu","year":"2021","venue":"EMBC","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hsu et al., 2017] Yu-Liang Hsu, Jeen-Shing Wang, Wei- Chun Chiang, and Chien-Han Hung. Automatic ecg- based emotion recognition in music listening. IEEE TAC, 11(1):85-99, 2017.","title":"Automatic ecgbased emotion recognition in music listening","authors":"Yu-Liang Hsu, Jeen-Shing Wang, Wei-Chun Chiang, Chien-Han Hung","year":"2017","venue":"IEEE TAC","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jiang et al., 2019] Jianwen Jiang, Yuxuan Wei, Yifan Feng, Jingxuan Cao, and Yue Gao. Dynamic hypergraph neural networks. In IJCAI, pages 2635-2641, 2019.","title":"Dynamic hypergraph neural networks","authors":"Jianwen Jiang, Yuxuan Wei, Yifan Feng, Jingxuan Cao, Yue Gao","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/366"},{"position":10,"raw":"[Kim and André, 2008] Jonghwa Kim and Elisabeth André. Emotion recognition based on physiological changes in music listening. IEEE TPAMI, 30(12):2067-2083, 2008.","title":"Emotion recognition based on physiological changes in music listening","authors":"André ; Kim, Jonghwa Kim, Elisabeth André","year":"2008","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lan et al., 2018] Zirui Lan, Olga Sourina, Lipo Wang, Reinhold Scherer, and Gernot R Müller-Putz. Domain adaptation techniques for eeg-based emotion recognition: a comparative study on two public datasets. IEEE TCDS, 11(1):85-94, 2018.","title":"Domain adaptation techniques for eeg-based emotion recognition: a comparative study on two public datasets","authors":"Lan","year":"2018","venue":"IEEE TCDS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Li, 2014] Ming Li and Xin Li. Verification based ecg biometrics with cardiac irregular conditions using heartbeat level and segment level information fusion. In ICASSP, pages 3769-3773. IEEE, 2014.","title":"Verification based ecg biometrics with cardiac irregular conditions using heartbeat level and segment level information fusion","authors":"Li ; Li, Ming Li, Xin Li","year":"2014","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2020] Chao Li, Zhongtian Bao, Linhao Li, and Ziping Zhao. Exploring temporal representations by lever- aging attention-based bidirectional lstm-rnns for multi- modal emotion recognition. Information Processing & Management, 57(3):102185, 2020.","title":"Exploring temporal representations by leveraging attention-based bidirectional lstm-rnns for multimodal emotion recognition","authors":"Chao Li, Zhongtian Bao, Linhao Li, Ziping Zhao","year":"2020","venue":"Information Processing & Management","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pan and Yang, 2009] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE TKDE, 22(10):1345- 1359, 2009.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2009","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pan et al., 2010] Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE TNN, 22(2):199-210, 2010.","title":"Domain adaptation via transfer component analysis","authors":"Ivor W Sinno Jialin Pan, James T Tsang, Qiang Kwok, Yang","year":"2010","venue":"IEEE TNN","doi":null,"url":null,"internal_url":"/proceedings/2009/200"},{"position":16,"raw":"[Sepúlveda et al., 2021] Axel Sepúlveda, Francisco Castillo, Carlos Palma, and Maria Rodriguez-Fernandez. Emotion recognition from ecg signals using wavelet scattering and machine learning. Applied Sciences, 11(11):4945, 2021.","title":"Emotion recognition from ecg signals using wavelet scattering and machine learning","authors":"Axel Sepúlveda, Francisco Castillo, Carlos Palma, Maria Rodriguez-Fernandez","year":"2021","venue":"Applied Sciences","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tung et al., 2018] Kuan Tung, Po-Kang Liu, Yu-Chuan Chuang, Sheng-Hui Wang, and An-Yeu Andy Wu. Entropy-assisted multi-modal emotion recognition frame- work based on physiological signals. In IECBES, pages 22-26. IEEE, 2018.","title":"Entropy-assisted multi-modal emotion recognition framework based on physiological signals","authors":"Kuan Tung, Po-Kang Liu, Yu-Chuan Chuang, Sheng-Hui Wang, An-Yeu Andy Wu","year":"2018","venue":"IECBES","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wu and Ng, 2022] Hanrui Wu and Michael K Ng. Hyper- graph convolution on nodes-hyperedges network for semi- supervised node classification. TKDD, 16(4):1-19, 2022.","title":"Hypergraph convolution on nodes-hyperedges network for semisupervised node classification","authors":"Ng ; ] Hanrui Wu, Michael K Wu, Ng","year":"2022","venue":"TKDD","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wu et al., 2017a] Qingyao Wu, Hanrui Wu, Xiaoming Zhou, Mingkui Tan, Yonghui Xu, Yuguang Yan, and Tianyong Hao. Online transfer learning with multiple homogeneous or heterogeneous sources. IEEE TKDE, 29(7):1494-1507, 2017.","title":"Online transfer learning with multiple homogeneous or heterogeneous sources","authors":"Wu","year":"2017","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2017b] Qingyao Wu, Xiaoming Zhou, Yuguang Yan, Hanrui Wu, and Huaqing Min. Online transfer learning by leveraging multiple source domains. KIS, 52(3):687-707, 2017.","title":"Online transfer learning by leveraging multiple source domains","authors":"Wu","year":"2017","venue":"KIS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu et al., 2019] Hanrui Wu, Yuguang Yan, Yuzhong Ye, Huaqing Min, Michael K Ng, and Qingyao Wu. Online heterogeneous transfer learning by knowledge transition. TIST, 10(3):1-19, 2019.","title":"Online heterogeneous transfer learning by knowledge transition","authors":"Hanrui Wu, Yuguang Yan, Yuzhong Ye, Huaqing Min, Michael K Ng, Qingyao Wu","year":"2019","venue":"TIST","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2018] Zizhao Zhang, Haojie Lin, Xibin Zhao, Rongrong Ji, and Yue Gao. Inductive multi-hypergraph learning and its application on view-based 3d object clas- sification. IEEE TIP, 27(12):5957-5968, 2018.","title":"Inductive multi-hypergraph learning and its application on view-based 3d object classification","authors":"Zhang","year":"2018","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[ZHAO and HOI, 2010] Peilin ZHAO and Steven CH HOI. Otl: A framework of online transfer learning.(2010). In ICML, pages 219-1, 2010.","title":"Otl: A framework of online transfer learning","authors":"Peilin Zhao, C H Steven, Hoi","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2014] Peilin Zhao, Steven CH Hoi, Jialei Wang, and Bin Li. Online transfer learning. Artificial Intelli- gence, 216:76-102, 2014.","title":"Online transfer learning","authors":"Peilin Zhao, C H Steven, Jialei Hoi, Bin Wang, Li","year":"2014","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2018] Sicheng Zhao, Guiguang Ding, Jungong Han, and Yue Gao. Personality-aware personalized emo- tion recognition from physiological signals. In IJCAI, pages 1660-1667, 2018.","title":"Personality-aware personalized emotion recognition from physiological signals","authors":"Sicheng Zhao, Guiguang Ding, Jungong Han, Yue Gao","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/230"},{"position":26,"raw":"[Zheng and Lu, 2016] Wei-Long Zheng and Bao-Liang Lu. Personalizing eeg-based affective models with transfer learning. In Proceedings of the twenty-fifth international joint conference on artificial intelligence, pages 2732- 2738, 2016.","title":"Personalizing eeg-based affective models with transfer learning","authors":"Wei-Long Zheng, Bao-Liang Lu","year":"2016","venue":"Proceedings of the twenty-fifth international joint conference on artificial intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/388"},{"position":27,"raw":"[Zhou et al., 2006] Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Cluster- ing, classification, and embedding. NIPS, 19:1601-1608, 2006.","title":"Learning with hypergraphs: Clustering, classification, and embedding","authors":"Zhou","year":"2006","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2020] Hui Zhou, Kun Wang, and Jie Tian. On- line transfer learning for differential diagnosis of benign and malignant thyroid nodules with ultrasound images. IEEE TBME, 67(10):2773-2780, 2020.","title":"Online transfer learning for differential diagnosis of benign and malignant thyroid nodules with ultrasound images","authors":"Hui Zhou, Kun Wang, Jie Tian","year":"2020","venue":"IEEE TBME","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}