{"type":"ScholarlyArticle","title":"IntEr-HRI Competition: Intrinsic Error Evaluation during Human - Robot Interaction","authors":[{"name":"Kartik Chari","slug":"kartik-chari","given":"Kartik","family":"Chari","affiliation":"Robotics Innovation Centre , German Research Centre for Artificial Intelligence (DFKI) , Bremen , Germany."},{"name":"Niklas Kueper","slug":"niklas-kueper","given":"Niklas","family":"Kueper","affiliation":"Robotics Innovation Centre , German Research Centre for Artificial Intelligence (DFKI) , Bremen , Germany."},{"name":"Su Kyoung Kim","slug":"su-kyoung-kim","given":"Su Kyoung","family":"Kim","affiliation":""},{"name":"Frank Kirchner","slug":"frank-kirchner","given":"Frank","family":"Kirchner","affiliation":"Robotics Innovation Centre , German Research Centre for Artificial Intelligence (DFKI) , Bremen , Germany.; Robotics Lab , University of Bremen , Bremen , Germany."},{"name":"Elsa Andrea Kirchner","slug":"elsa-andrea-kirchner","given":"Elsa Andrea","family":"Kirchner","affiliation":"Robotics Innovation Centre , German Research Centre for Artificial Intelligence (DFKI) , Bremen , Germany.; Institute of Medical Technology Systems , University of Duisburg-Essen , Duisburg , Germany."}],"abstract":"Reliable detection of human intentions from electroencephalogram (EEG) to improve human-robot interaction (HRI) has recently gained significant importance. To ensure safe and satisfactory interactions, implicit detection of erroneous behavior of robotic systems, particularly assistive devices, is essential. This can be achieved by detecting error-related potentials (ErrPs) in EEG, evoked by visual, tactile, or visuo-tactile stimuli. Of these, the ErrPs evoked tactilely with the help of a robot remains unexplored and has been the main focus of this competition. The task for participating teams was to develop robust AI models for continuous real-time classification of erroneous behavior of assistive robotic devices from the human EEG. Even though the competition results prove its feasibility, a performance gap (balanced accuracy and computation time) of more than 10% was observed between the offline and online classification of errors in real-world scenarios. In addition to the competitive AI models developed by the participating teams, this competition also contributed towards a one-of-its-kind open-access EEG and EMG dataset, a lossless live streaming solution for EEG data, and a novel quantitative metric for benchmarking online asynchronous EEG detection solutions.","doi":"10.24963/ijcai.2024/994","url":"https://ijcai.org/proceedings/2024/994","pdf":"https://ijcai.org/proceedings/2024/0994.pdf","markdown":"https://ijcai.org/proceedings/2024/994.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/994","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Demo Track","track":"Demo Track","number":994,"pages":{"first":8623,"last":8626},"keywords":["Humans and AI: HAI: Brain sciences","Robotics: ROB: Human robot interaction"],"videos":[],"references":[{"position":1,"raw":"[Ahkami and Ghassemi, 2021] Bahareh Ahkami and Farnaz Ghassemi. Adding tactile feedback and changing isi to im- prove bci systems' robustness: An error-related potential study. Brain topography, 34(4):467-477, 2021.","title":"Adding tactile feedback and changing isi to improve bci systems' robustness: An error-related potential study","authors":"Ghassemi Ahkami, Bahareh Ahkami, Farnaz Ghassemi","year":"2021","venue":"Brain topography","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blankertz, 2004] Benjamin Blankertz. https://www.bbci.de/ competition/iii/, 2004. Accessed: 2024-02-19.","title":null,"authors":"Benjamin Blankertz","year":"2024","venue":null,"doi":null,"url":"https://www.bbci.de/competition/iii/,2004","internal_url":null},{"position":3,"raw":"[Blankertz, 2008] Benjamin Blankertz. https://www.bbci.de/ competition/iv/, 2008. Accessed: 2024-02-19.","title":null,"authors":"Benjamin Blankertz","year":"2024","venue":null,"doi":null,"url":"https://www.bbci.de/competition/iv/,2008","internal_url":null},{"position":4,"raw":"[Chavarriaga et al., 2012] Ricardo Chavarriaga, Xavier Per- rin, Roland Siegwart, and José del R Millán. Anticipation- and error-related eeg signals during realistic human- machine interaction: A study on visual and tactile feed- back. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 6723-6726. Ieee, 2012.","title":"Anticipationand error-related eeg signals during realistic humanmachine interaction: A study on visual and tactile feedback","authors":"Ricardo Chavarriaga, Xavier Perrin, Roland Siegwart, José Del R Millán","year":"2012","venue":"Annual International Conference of the IEEE Engineering in Medicine and Biology Society","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chavarriaga et al., 2014] Ricardo Chavarriaga, Aleksander Sobolewski, and José del R Millán. Errare machinale est: the use of error-related potentials in brain-machine inter- faces. Frontiers in neuroscience, page 208, 2014.","title":"Errare machinale est: the use of error-related potentials in brain-machine interfaces","authors":"Ricardo Chavarriaga, Aleksander Sobolewski, José Del R Millán","year":"2014","venue":"Frontiers in neuroscience","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ehrlich and Cheng, 2019] Stefan K Ehrlich and Gordon Cheng. A feasibility study for validating robot actions us- ing eeg-based error-related potentials. International Jour- nal of Social Robotics, 11:271-283, 2019.","title":"A feasibility study for validating robot actions using eeg-based error-related potentials","authors":"Cheng Ehrlich, Stefan K Ehrlich, Gordon Cheng","year":"2019","venue":"International Journal of Social Robotics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Iturrate et al., 2010] Inaki Iturrate, Luis Montesano, and Javier Minguez. Single trial recognition of error-related potentials during observation of robot operation. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, pages 4181-4184. IEEE, 2010.","title":"Single trial recognition of error-related potentials during observation of robot operation","authors":"Inaki Iturrate, Luis Montesano, Javier Minguez","year":"2010","venue":"Annual International Conference of the IEEE Engineering in Medicine and Biology","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kim and Kirchner, 2013] Su Kyoung Kim and Elsa Andrea Kirchner. Classifier transferability in the detection of er- ror related potentials from observation to interaction. In 2013 IEEE international conference on systems, man, and cybernetics, pages 3360-3365. IEEE, 2013.","title":"Classifier transferability in the detection of error related potentials from observation to interaction","authors":"Kirchner ; ] Kim, Kyoung Su, Elsa Andrea Kim, Kirchner","year":"2013","venue":"2013 IEEE international conference on systems, man, and cybernetics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim and Kirchner, 2015] Su Kyoung Kim and Elsa Andrea Kirchner. Handling few training data: classifier transfer between different types of error-related potentials. IEEE Transactions on Neural Systems and Rehabilitation Engi- neering, 24(3):320-332, 2015.","title":"Handling few training data: classifier transfer between different types of error-related potentials","authors":"Kirchner ; ] Kim, Kyoung Su, Elsa Andrea Kim, Kirchner","year":"2015","venue":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2017] Su Kyoung Kim, Elsa Andrea Kirchner, Arne Stefes, and Frank Kirchner. Intrinsic interactive rein- forcement learning-using error-related potentials for real world human-robot interaction. Scientific reports, 7(1):1- 16, 2017.","title":"Intrinsic interactive reinforcement learning-using error-related potentials for real world human-robot interaction","authors":"Su Kyoung, Kim, Elsa Andrea Kirchner, Arne Stefes, Frank Kirchner","year":"2017","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kim et al., 2020] Su Kyoung Kim, Elsa Andrea Kirchner, and Frank Kirchner. Flexible online adaptation of learn- ing strategy using eeg-based reinforcement signals in real- world robotic applications. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 4885-4891. IEEE, 2020.","title":"Flexible online adaptation of learning strategy using eeg-based reinforcement signals in realworld robotic applications","authors":"Su Kyoung, Kim, Elsa Andrea Kirchner, Frank Kirchner","year":"2020","venue":"IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kothe et al., 2018] Christian Kothe, David Medine, Chad- wick Boulay, Matthew Grivich, and Tristan Stenner. Lab streaming layer (lsl). https://github.com/sccn/ labstreaminglayer, 2018. Accessed: 2023-08-09.","title":"Lab streaming layer (lsl)","authors":"Kothe","year":"2018","venue":null,"doi":null,"url":"https://github.com/sccn/labstreaminglayer,2018","internal_url":null},{"position":13,"raw":"[Kueper et al., 2024] Niklas Kueper, Kartik Chari, Judith Bütefür, Julia Habenicht, Tobias Rossol, Su Kyoung Kim, Marc Tabie, Frank Kirchner, and Elsa Andrea Kirchner. Eeg and emg dataset for the detection of errors introduced by an active orthosis device. Frontiers in Human Neuro- science, 18, 2024.","title":"Eeg and emg dataset for the detection of errors introduced by an active orthosis device","authors":"Niklas Kueper, Kartik Chari, Judith Bütefür, Julia Habenicht, Tobias Rossol, Su Kyoung Kim, Marc Tabie, Frank Kirchner, Elsa Andrea Kirchner","year":"2024","venue":"Frontiers in Human Neuroscience","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Perrin et al., 2008] Xavier Perrin, Ricardo Chavarriaga, Céline Ray, Roland Siegwart, and José del R Millán. A comparative psychophysical and eeg study of different feedback modalities for hri. In Proceedings of the 3rd ACM/IEEE international conference on Human robot in- teraction, pages 41-48, 2008.","title":"A comparative psychophysical and eeg study of different feedback modalities for hri","authors":"Xavier Perrin, Ricardo Chavarriaga, Céline Ray, Roland Siegwart, José Del R Millán","year":"2008","venue":"Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Roy et al., 2022] Raphaëlle Roy, Marcel Hinss, Ludovic Darmet, Simon Ladouce, Emilie Jahanpour, Bertille Somon, Xiaoqi Xu, Nicolas Drougard, Frederic Dehais, and Fabien Lotte. Retrospective on the first passive brain- computer interface competition on cross-session workload estimation. Frontiers in Neuroergonomics, 3, 2022.","title":"Retrospective on the first passive braincomputer interface competition on cross-session workload estimation","authors":"Raphaëlle Roy, Marcel Hinss, Ludovic Darmet, Simon Ladouce, Emilie Jahanpour, Bertille Somon, Xiaoqi Xu, Nicolas Drougard, Frederic Dehais, Fabien Lotte","year":"2022","venue":"Frontiers in Neuroergonomics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Schiatti et al., 2019] Lucia Schiatti, Giacinto Barresi, Ja- copo Tessadori, Louis Charles King, and Leonardo S Mat- tos. The effect of vibrotactile feedback on errp-based adap- tive classification of motor imagery. In 2019 41st An- nual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 6750-6753. IEEE, 2019.","title":"The effect of vibrotactile feedback on errp-based adaptive classification of motor imagery","authors":"Lucia Schiatti, Giacinto Barresi, Jacopo Tessadori, Louis Charles King, Leonardo S Mattos","year":"2019","venue":"41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tessadori et al., 2017] Jacopo Tessadori, Lucia Schiatti, Gi- acinto Barresi, and Leonardo S Mattos. Does tactile feed- back enhance single-trial detection of error-related eeg potentials? In 2017 IEEE international conference on systems, man, and cybernetics (SMC), pages 1417-1422. IEEE, 2017.","title":"Jacopo Tessadori, Lucia Schiatti, Giacinto Barresi, and Leonardo S Mattos. Does tactile feedback enhance single-trial detection of error-related eeg potentials?","authors":"Tessadori","year":"2017","venue":"2017 IEEE international conference on systems, man, and cybernetics (SMC)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wu et al., 2022] Dongrui Wu, Yifan Xu, and Bao-Liang Lu. Transfer learning for eeg-based brain-computer interfaces: A review of progress made since 2016. IEEE Transac- tions on Cognitive and Developmental Systems, 14(1):4- 19, 2022.","title":"Transfer learning for eeg-based brain-computer interfaces: A review of progress made since","authors":"Dongrui Wu, Yifan Xu, Bao-Liang Lu","year":"2016","venue":"IEEE Transactions on Cognitive and Developmental Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}