{"type":"ScholarlyArticle","title":"A Simple Convolutional Neural Network for Accurate P300 Detection and Character Spelling in Brain Computer Interface","authors":[{"name":"Hongchang Shan","slug":"hongchang-shan","given":"Hongchang","family":"Shan","affiliation":"Leiden Institute of Advanced Computer Science , Leiden University , Leiden , The Netherlands"},{"name":"Yu Liu","slug":"yu-liu","given":"Yu","family":"Liu","affiliation":"Leiden Institute of Advanced Computer Science , Leiden University , Leiden , The Netherlands"},{"name":"Todor Stefanov","slug":"todor-stefanov","given":"Todor","family":"Stefanov","affiliation":"Leiden Institute of Advanced Computer Science , Leiden University , Leiden , The Netherlands"}],"abstract":"A Brain Computer Interface (BCI) character speller allows human-beings to directly spell characters using eye-gazes, thereby building communication between the human brain and a computer. Convolutional Neural Networks (CNNs) have shown better performance than traditional machine learning methods for BCI signal recognition and its application to the character speller. However, current CNN architectures limit further accuracy improvements of signal detection and character spelling and also need high complexity to achieve competitive accuracy, thereby preventing the use of CNNs in portable BCIs. To address these issues, we propose a novel and simple CNN which effectively learns feature representations from both raw temporal information and raw spatial information. The complexity of the proposed CNN is significantly reduced compared with state-of-the-art CNNs for BCI signal detection. We perform experiments on three benchmark datasets and compare our results with those in previous research works which report the best results. The comparison shows that our proposed CNN can increase the signal detection accuracy by up to 15.61% and the character spelling accuracy by up to 19.35%.","doi":"10.24963/ijcai.2018/222","url":"https://ijcai.org/proceedings/2018/222","pdf":"https://ijcai.org/proceedings/2018/0222.pdf","markdown":"https://ijcai.org/proceedings/2018/222.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/222","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Humans and AI","track":"Main track","number":222,"pages":{"first":1604,"last":1610},"keywords":["Machine Learning Applications: Applications of Supervised Learning","Humans and AI: Brain Sciences","Humans and AI: Human-Computer Interaction"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Ashish Agarwal, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016.","title":"Tensorflow: Large-scale machine learning on heterogeneous distributed systems","authors":"Abadi","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blankertz, 2003] B Blankertz. BCI competition II webpage. http://www.bbci.de/competition/ii/, 2003.","title":"BCI competition II webpage","authors":"Blankertz","year":null,"venue":null,"doi":null,"url":"http://www.bbci.de/competition/ii/,2003","internal_url":null},{"position":3,"raw":"[Blankertz, 2008] B Blankertz. BCI competition III web- page. http://www.bbci.de/competition/iii/, 2008.","title":"BCI competition III webpage","authors":"Blankertz","year":null,"venue":null,"doi":null,"url":"http://www.bbci.de/competition/iii/,2008","internal_url":null},{"position":4,"raw":"[Bostanov, 2004] Vladimir Bostanov. BCI competition 2003-data sets Ib and IIb: feature extraction from event- related brain potentials with the continuous wavelet trans- form and the t-value scalogram. IEEE Transactions on Biomedical Engineering, 51(6):1057-1061, 2004.","title":"BCI competition 2003-data sets Ib and IIb: feature extraction from eventrelated brain potentials with the continuous wavelet transform and the t-value scalogram","authors":"Vladimir Bostanov","year":"2004","venue":"IEEE Transactions on Biomedical Engineering","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cecotti and Graser, 2011] Hubert Cecotti and Axel Graser. Convolutional neural networks for P300 detection with ap- plication to brain-computer interfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(3):433- 445, 2011.","title":"Convolutional neural networks for P300 detection with application to brain-computer interfaces","authors":"Graser Cecotti, Hubert Cecotti, Axel Graser","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cecotti et al., 2011] Hubert Cecotti, Bertrand Rivet, et al. A robust sensor-selection method for P300 brain-computer interfaces. Journal of Neural Engineering, 8(1):016001, 2011.","title":"A robust sensor-selection method for P300 brain-computer interfaces","authors":"Hubert Cecotti, Bertrand Rivet","year":"2011","venue":"Journal of Neural Engineering","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2016] Tsan-Yu Chen, Chih-Wei Feng, and Wai- Chi Fang. Development of a reliable SSVEP-based BCI mobile dialing system. In Consumer Electronics (ICCE), 2016 IEEE International Conference on, pages 269-272. IEEE, 2016.","title":"Development of a reliable SSVEP-based BCI mobile dialing system","authors":"Tsan-Yu Chen, Chih-Wei Feng, Wai-Chi Fang","year":"2016","venue":"Consumer Electronics (ICCE), 2016 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chollet and others, 2015] Franc ¸ois Chollet et al. Keras. https://github.com/keras-team/keras, 2015.","title":"Franc ¸ois Chollet et al. Keras","authors":"Others Chollet","year":"2015","venue":null,"doi":null,"url":"https://github.com/keras-team/keras,2015","internal_url":null},{"position":9,"raw":"[Farwell and Donchin, 1988] Lawrence Ashley Farwell and Emanuel Donchin. Talking off the top of your head: to- ward a mental prosthesis utilizing event-related brain po- tentials. Electroencephalography and Clinical Neurophys- iology, 70(6):510-523, 1988.","title":"Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials","authors":"Donchin ; Farwell, Ashley Lawrence, Emanuel Farwell, Donchin","year":"1988","venue":"Electroencephalography and Clinical Neurophysiology","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gao et al., 2010] Junfeng Gao, Chongxun Zheng, and Pei Wang. Online removal of muscle artifact from electroen- cephalogram signals based on canonical correlation anal- ysis. Clinical EEG and Neuroscience, 41(1):53-59, 2010.","title":"Online removal of muscle artifact from electroencephalogram signals based on canonical correlation analysis","authors":"Junfeng Gao, Chongxun Zheng, Pei Wang","year":"2010","venue":"Clinical EEG and Neuroscience","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gilroy et al., 2013] Stephen William Gilroy, Julie Porteous, et al. A brain-computer interface to a plan-based narra- tive. In International Joint Conference on Artificial Intel- ligence, pages 1997-2005, 2013.","title":"A brain-computer interface to a plan-based narrative","authors":"Stephen William Gilroy, Julie Porteous","year":"2013","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gwin et al., 2010] Joseph T Gwin, Klaus Gramann, et al. Removal of movement artifact from high-density EEG recorded during walking and running. Journal of Neuro- physiology, 103(6):3526-3534, 2010.","title":"Removal of movement artifact from high-density EEG recorded during walking and running","authors":"Klaus Joseph T Gwin, Gramann","year":"2010","venue":"Journal of Neurophysiology","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448-456, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural In- formation Processing Systems, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Krizhevsky","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2013] Chin-Teng Lin, Shu-Fang Tsai, and Li-Wei Ko. EEG-based learning system for online motion sick- ness level estimation in a dynamic vehicle environment. IEEE Transactions on Neural Networks and Learning Sys- tems, 24(10):1689-1700, 2013.","title":"EEG-based learning system for online motion sickness level estimation in a dynamic vehicle environment","authors":"Chin-Teng Lin, Shu-Fang Tsai, Li-Wei Ko","year":"2013","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2014] Chin-Teng Lin, Bor-Shyh Lin, et al. Brain computer interface-based smart living environmental auto- adjustment control system in UPnP home networking. IEEE Systems Journal, 8(2):363-370, 2014.","title":"Brain computer interface-based smart living environmental autoadjustment control system in UPnP home networking","authors":"Chin-Teng Lin, Bor-Shyh Lin","year":"2014","venue":"IEEE Systems Journal","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2017] Mingfei Liu, Wei Wu, Zhenghui Gu, Zhu- liang Yu, FeiFei Qi, and Yuanqing Li. Deep learning based on Batch Normalization for P300 signal detection. Neuro- computing, 2017.","title":"Deep learning based on Batch Normalization for P300 signal detection","authors":"Mingfei Liu, Wei Wu, Zhenghui Gu, Zhuliang Yu, Feifei Qi, Yuanqing Li","year":"2017","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Manor and Geva, 2015] Ran Manor and Amir B Geva. Con- volutional neural network for multi-category rapid serial visual presentation BCI. Frontiers in Computational Neu- roscience, 9, 2015.","title":"Convolutional neural network for multi-category rapid serial visual presentation BCI","authors":"Geva ; ] Ran Manor, Manor, B Amir, Geva","year":"2015","venue":"Frontiers in Computational Neuroscience","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mennes et al., 2010] Maarten Mennes, Heidi Wouters, Bart Vanrumste, Lieven Lagae, and Peter Stiers. Validation of ICA as a tool to remove eye movement artifacts from EEG/ERP. Psychophysiology, 47(6):1142-1150, 2010.","title":"Validation of ICA as a tool to remove eye movement artifacts from EEG/ERP","authors":"Mennes","year":"2010","venue":"Psychophysiology","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Polich, 2007] John Polich. Updating P300: an integra- tive theory of P3a and P3b. Clinical Neurophysiology, 118(10):2128-2148, 2007.","title":"Updating P300: an integrative theory of P3a and P3b","authors":"John Polich, Polich","year":"2007","venue":"Clinical Neurophysiology","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rakotomamonjy and Guigue, 2008] Alain Rakotomamonjy and Vincent Guigue. BCI competition III: dataset II- ensemble of SVMs for BCI P300 speller. IEEE Transac- tions on Biomedical Engineering, 55(3):1147-1154, 2008.","title":"Alain Rakotomamonjy and Vincent Guigue. BCI competition III: dataset IIensemble of SVMs for BCI P300 speller","authors":"Guigue Rakotomamonjy","year":"2008","venue":"IEEE Transactions on Biomedical Engineering","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rivet et al., 2009] Bertrand Rivet, Antoine Souloumiac, et al. xDAWN algorithm to enhance evoked potentials: ap- plication to brain-computer interface. IEEE Transactions on Biomedical Engineering, 56(8):2035-2043, 2009.","title":"xDAWN algorithm to enhance evoked potentials: application to brain-computer interface","authors":"Bertrand Rivet, Antoine Souloumiac","year":"2009","venue":"IEEE Transactions on Biomedical Engineering","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sellers and Donchin, 2006] Eric W Sellers and Emanuel Donchin. A P300-based brain-computer interface: ini- tial tests by ALS patients. Clinical Neurophysiology, 117(3):538-548, 2006.","title":"A P300-based brain-computer interface: initial tests by ALS patients","authors":"Donchin Sellers, Eric W Sellers, Emanuel Donchin","year":"2006","venue":"Clinical Neurophysiology","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hin- ton, et al. Dropout: a simple way to prevent neural net- works from overfitting. Journal of Machine Learning Re- search, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton","year":"2014","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sutton et al., 1967] Samuel Sutton, Patricia Tueting, et al. Information delivery and the sensory evoked potential. Science, 155(3768):1436-1439, 1967.","title":"Information delivery and the sensory evoked potential","authors":"Sutton","year":"1967","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2011] Yu-Te Wang, Yijun Wang, and Tzyy- Ping Jung. A cell-phone-based brain-computer interface for communication in daily life. Journal of Neural Engi- neering, 8(2):025018, 2011.","title":"A cell-phone-based brain-computer interface for communication in daily life","authors":"Yu-Te Wang, Yijun Wang, Tzyy-Ping Jung","year":"2011","venue":"Journal of Neural Engineering","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wolpaw and Wolpaw, 2012] Jonathan Wolpaw and Eliza- beth Winter Wolpaw. Brain-computer interfaces: princi- ples and practice. OUP USA, 2012.","title":"Brain-computer interfaces: principles and practice","authors":"Wolpaw ; Winter Wolpaw, Wolpaw","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}