{"type":"ScholarlyArticle","title":"Teacher Assistant-Based Knowledge Distillation Extracting Multi-level Features on Single Channel Sleep EEG","authors":[{"name":"Heng Liang","slug":"heng-liang","given":"Heng","family":"Liang","affiliation":"Brainnetome Center , Institute of Automation , Chinese Academy of Sciences , Beijing , China"},{"name":"Yucheng Liu","slug":"yucheng-liu","given":"Yucheng","family":"Liu","affiliation":"Brainnetome Center , Institute of Automation , Chinese Academy of Sciences , Beijing , China"},{"name":"Haichao Wang","slug":"haichao-wang","given":"Haichao","family":"Wang","affiliation":"Tsinghua-Berkeley Shenzhen Institute , Shenzhen , China"},{"name":"Ziyu Jia","slug":"ziyu-jia","given":"Ziyu","family":"Jia","affiliation":"Brainnetome Center , Institute of Automation , Chinese Academy of Sciences , Beijing , China"}],"abstract":"Sleep stage classification is of great significance to the diagnosis of sleep disorders. However, existing sleep stage classification models based on deep learning are usually relatively large in size (wider and deeper), which makes them hard to be deployed on wearable devices. Therefore, it is a challenge to lighten the existing sleep stage classification models. In this paper, we propose a novel general knowledge distillation framework for sleep stage classification tasks called SleepKD. Our SleepKD, composed of the multi-level module, teacher assistant module, and other knowledge distillation modules, aims to lighten large-scale sleep stage classification models. Specifically, the multi-level module is able to transfer the multi-level knowledge extracted from sleep signals by the teacher model (large-scale model) to the student model (lightweight model). Moreover, the teacher assistant module bridges the large gap between the teacher and student network, and further improves the distillation. We evaluate our method on two public sleep datasets (Sleep-EDF and ISRUC-III). Compared to the baseline methods, the results show that our knowledge distillation framework achieves state-of-the-art performance. SleepKD can significantly lighten the sleep model while maintaining its classification performance. The source code is available at https://github.com/HychaoWang/SleepKD.","doi":"10.24963/ijcai.2023/439","url":"https://ijcai.org/proceedings/2023/439","pdf":"https://ijcai.org/proceedings/2023/0439.pdf","markdown":"https://ijcai.org/proceedings/2023/439.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/439","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":439,"pages":{"first":3948,"last":3956},"keywords":["Humans and AI: HAI: Brain sciences","Multidisciplinary Topics and Applications: MDA: Health and medicine","Machine Learning: ML: Classification","Machine Learning: ML: Applications"],"videos":[],"references":[{"position":1,"raw":"[Aguilar et al., 2020] Gustavo Aguilar, Yuan Ling, Yu Zhang, Benjamin Yao, Xing Fan, and Chenlei Guo. Knowledge distillation from internal representa- tions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7350-7357, 2020.","title":"Knowledge distillation from internal representations","authors":"Gustavo Aguilar, Yuan Ling, Yu Zhang, Benjamin Yao, Xing Fan, Chenlei Guo","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Back et al., 2019] Seunghyeok Back, Seongju Lee, Ho- geon Seo, Deokhwan Park, Tae Kim, and Kyoobin Lee. Intra-and inter-epoch temporal context network (iit- net) for automatic sleep stage scoring. arXiv preprint arXiv:1902.06562, 2019.","title":"Seunghyeok Back, Seongju Lee","authors":"Back","year":"2019","venue":"Intra-and inter-epoch temporal context network (iitnet) for automatic sleep stage scoring","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Basha et al., 2021] A Jameer Basha, B Saravana Balaji, S Poornima, M Prathilothamai, and K Venkatachalam. Support vector machine and simple recurrent network based automatic sleep stage classification of fuzzy kernel. Journal of ambient intelligence and humanized computing, 12(6):6189-6197, 2021.","title":"Support vector machine and simple recurrent network based automatic sleep stage classification of fuzzy kernel","authors":"Basha","year":"2021","venue":"Journal of ambient intelligence and humanized computing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Berry et al., 2012] Richard B Berry, Rita Brooks, Char- lene E Gamaldo, Susan M Harding, C Marcus, Bradley V Vaughn, et al. The aasm manual for the scoring of sleep and associated events. Rules, Terminology and Techni- cal Specifications, Darien, Illinois, American Academy of Sleep Medicine, 176:2012, 2012.","title":"The aasm manual for the scoring of sleep and associated events. Rules, Terminology and Technical Specifications","authors":"Rita Richard B Berry, Charlene E Brooks, Susan M Gamaldo, C Harding, Bradley V Marcus, Vaughn","year":"2012","venue":"American Academy of Sleep Medicine","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chambon et al., 2018] Stanislas Chambon, Mathieu N Galtier, Pierrick J Arnal, Gilles Wainrib, and Alexandre Gramfort. A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Transactions on Neural Systems and Rehabil- itation Engineering, 26(4):758-769, 2018.","title":"A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series","authors":"Stanislas Chambon, N Mathieu, Galtier, J Pierrick, Gilles Arnal, Alexandre Wainrib, Gramfort","year":"2018","venue":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2021] Pengguang Chen, Shu Liu, Hengshuang Zhao, and Jiaya Jia. Distilling knowledge via knowledge review. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5008- 5017, 2021.","title":"Distilling knowledge via knowledge review","authors":"Chen","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cho and Hariharan, 2019] Jang Hyun Cho and Bharath Har- iharan. On the efficacy of knowledge distillation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4794-4802, 2019.","title":"On the efficacy of knowledge distillation","authors":"Hariharan ; Cho, Hyun Jang, Bharath Cho, Hariharan","year":"2019","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cui et al., 2018] Zhihong Cui, Xiangwei Zheng, Xuexiao Shao, and Lizhen Cui. Automatic sleep stage classification based on convolutional neural network and fine-grained segments. Complexity, 2018, 2018.","title":"Automatic sleep stage classification based on convolutional neural network and fine-grained segments","authors":"Zhihong Cui, Xiangwei Zheng, Xuexiao Shao, Lizhen Cui","year":"2018","venue":"Complexity","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fan et al., 2021] Jiahao Fan, Chenglu Sun, Meng Long, Chen Chen, and Wei Chen. Eognet: A novel deep learn- ing model for sleep stage classification based on single- channel eog signal. Frontiers in Neuroscience, 15, 2021.","title":"Eognet: A novel deep learning model for sleep stage classification based on singlechannel eog signal","authors":"Jiahao Fan, Chenglu Sun, Meng Long, Chen Chen, Wei Chen","year":"2021","venue":"Frontiers in Neuroscience","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Furlanello et al., 2018] Tommaso Furlanello, Zachary Lip- ton, Michael Tschannen, Laurent Itti, and Anima Anand- kumar. Born again neural networks. In International Con- ference on Machine Learning, pages 1607-1616. PMLR, 2018.","title":"Born again neural networks","authors":"Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Harada et al., 2017] Tomohiro Harada, Takahiro Kawashima, Morito Morishima, and Keiki Takadama. Improving accuracy of real-time sleep stage estimation by considering personal sleep feature and rapid change of sleep behavior. In 2017 AAAI Spring Symposium Series, 2017.","title":"Improving accuracy of real-time sleep stage estimation by considering personal sleep feature and rapid change of sleep behavior","authors":"Tomohiro Harada, Takahiro Kawashima, Morito Morishima, Keiki Takadama","year":"2017","venue":"AAAI Spring Symposium Series","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Heo et al., 2019] Byeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park, Nojun Kwak, and Jin Young Choi. A comprehensive overhaul of feature distillation. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 1921-1930, 2019.","title":"A comprehensive overhaul of feature distillation","authors":"Heo","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2(7), 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang and Wang, 2017] Zehao Huang and Naiyan Wang. Like what you like: Knowledge distill via neuron selec- tivity transfer. arXiv preprint arXiv:1707.01219, 2017.","title":"Like what you like: Knowledge distill via neuron selectivity transfer","authors":"Wang ; Huang, Zehao Huang, Naiyan Wang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jia et al., 2020] Ziyu Jia, Youfang Lin, Jing Wang, Rong- hao Zhou, Xiaojun Ning, Yuanlai He, and Yaoshuai Zhao. Graphsleepnet: Adaptive spatial-temporal graph convolu- tional networks for sleep stage classification. In IJCAI, pages 1324-1330, 2020.","title":"Graphsleepnet: Adaptive spatial-temporal graph convolutional networks for sleep stage classification","authors":"Ziyu Jia, Youfang Lin, Jing Wang, Ronghao Zhou, Xiaojun Ning, Yuanlai He, Yaoshuai Zhao","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/184"},{"position":16,"raw":"[Jia et al., 2021a] Ziyu Jia, Youfang Lin, Jing Wang, Xiao- jun Ning, Yuanlai He, Ronghao Zhou, Yuhan Zhou, and H Lehman Li-wei. Multi-view spatial-temporal graph con- volutional networks with domain generalization for sleep stage classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 29:1977-1986, 2021.","title":"Multi-view spatial-temporal graph convolutional networks with domain generalization for sleep stage classification","authors":"Jia","year":"2021","venue":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jia et al., 2021b] Ziyu Jia, Youfang Lin, Jing Wang, Xue- hui Wang, Peiyi Xie, and Yingbin Zhang. Salientsleepnet: Multimodal salient wave detection network for sleep stag- ing. arXiv preprint arXiv:2105.13864, 2021.","title":"Salientsleepnet: Multimodal salient wave detection network for sleep staging","authors":"Jia","year":"2021","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2021/360"},{"position":18,"raw":"[Jia et al., 2022a] Ziyu Jia, Xiyang Cai, and Zehui Jiao. Multi-modal physiological signals based squeeze-and- excitation network with domain adversarial learning for sleep staging. IEEE Sensors Journal, 22(4):3464-3471, 2022.","title":"Multi-modal physiological signals based squeeze-andexcitation network with domain adversarial learning for sleep staging","authors":"Jia","year":"2022","venue":"IEEE Sensors Journal","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jia et al., 2022b] Ziyu Jia, Junyu Ji, Xinliang Zhou, and Yuhan Zhou. Hybrid spiking neural network for sleep elec- troencephalogram signals. Science China Information Sci- ences, 65(4):140403, 2022.","title":"Hybrid spiking neural network for sleep electroencephalogram signals","authors":"Jia","year":"2022","venue":"Science China Information Sciences","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Joshi et al., 2021] Vaibhav Joshi, Sricharan Vijayarangan, Preejith SP, and Mohanasankar Sivaprakasam. A deep knowledge distillation framework for eeg assisted en- hancement of single-lead ecg based sleep staging. arXiv preprint arXiv:2112.07252, 2021.","title":"A deep knowledge distillation framework for eeg assisted enhancement of single-lead ecg based sleep staging","authors":"Vaibhav Joshi, Sricharan Vijayarangan, S P Preejith, Mohanasankar Sivaprakasam","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kemp et al., 2018] Bob Kemp, A Zwinderman, B Tuk, H Kamphuisen, and J Oberyé. Sleep-edf database ex- panded. physionet. org, 2018.","title":"Sleep-edf database expanded","authors":"Bob Kemp, B Zwinderman, H Tuk, Kamphuisen, Oberyé","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Khalighi et al., 2016] Sirvan Khalighi, Teresa Sousa, José Moutinho Santos, and Urbano Nunes. Isruc-sleep: A comprehensive public dataset for sleep researchers. Computer methods and programs in biomedicine, 124:180-192, 2016.","title":"Isruc-sleep: A comprehensive public dataset for sleep researchers","authors":"Sirvan Khalighi, Teresa Sousa, José Moutinho Santos, Urbano Nunes","year":"2016","venue":"Computer methods and programs in biomedicine","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kim et al., 2018] Jangho Kim, SeongUk Park, and Nojun Kwak. Paraphrasing complex network: Network compres- sion via factor transfer. Advances in neural information processing systems, 31, 2018.","title":"Paraphrasing complex network: Network compression via factor transfer","authors":"Kim","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu and Jia, 2023] Yuchen Liu and Ziyu Jia. Bstt: A bayesian spatial-temporal transformer for sleep staging. In The Eleventh International Conference on Learning Rep- resentations, 2023.","title":"Bstt: A bayesian spatial-temporal transformer for sleep staging","authors":"Yuchen Liu, Ziyu Jia","year":"2023","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Matsushima et al., 2012] Hiroyasu Matsushima, Shogo Mi- nami, and Keiki Takadama. Age-based sleep stage esti- mation by evolutionary algorithm. In 2012 AAAI Spring Symposium Series, 2012.","title":"Age-based sleep stage estimation by evolutionary algorithm","authors":"Hiroyasu Matsushima, Shogo Minami, Keiki Takadama","year":"2012","venue":"AAAI Spring Symposium Series","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Mirzadeh et al., 2020] Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant. In Proceedings of the AAAI con- ference on artificial intelligence, volume 34, pages 5191- 5198, 2020.","title":"Improved knowledge distillation via teacher assistant","authors":"Mehrdad Seyed Iman Mirzadeh, Ang Farajtabar, Nir Li, Akihiro Levine, Hassan Matsukawa, Ghasemzadeh","year":"2020","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Mousavi et al., 2019] Sajad Mousavi, Fatemeh Afghah, and U Rajendra Acharya. Sleepeegnet: Automated sleep stage scoring with sequence to sequence deep learning ap- proach. PloS one, 14(5):e0216456, 2019.","title":"Sleepeegnet: Automated sleep stage scoring with sequence to sequence deep learning approach","authors":"Sajad Mousavi, Fatemeh Afghah, U Rajendra, Acharya","year":"2019","venue":"PloS one","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Perslev et al., 2019] Mathias Perslev, Michael Jensen, Sune Darkner, Poul Jørgen Jennum, and Christian Igel. U-time: A fully convolutional network for time series segmentation applied to sleep staging. Advances in Neural Information Processing Systems, 32, 2019.","title":"U-time: A fully convolutional network for time series segmentation applied to sleep staging","authors":"Mathias Perslev, Michael Jensen, Sune Darkner, Poul Jørgen Jennum, Christian Igel","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Phan et al., 2019] Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y. Chén, and Maarten De Vos. Joint classi- fication and prediction cnn framework for automatic sleep stage classification. IEEE Transactions on Biomedical En- gineering, 66(5):1285-1296, 2019.","title":"Joint classification and prediction cnn framework for automatic sleep stage classification","authors":"Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y Chén, Maarten De Vos","year":"2019","venue":"IEEE Transactions on Biomedical Engineering","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Rechtschaffen, 1968] Allan Rechtschaffen. A manual for standardized terminology, techniques and scoring system for sleep stages in human subjects. Brain information ser- vice, 1968.","title":"A manual for standardized terminology, techniques and scoring system for sleep stages in human subjects","authors":"Allan Rechtschaffen","year":"1968","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Romero et al., 2014] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014.","title":"Fitnets: Hints for thin deep nets","authors":"Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sekkal et al., 2022] Rym Nihel Sekkal, Fethi Bereksi- Reguig, Daniel Ruiz-Fernandez, Nabil Dib, and Samira Sekkal. Automatic sleep stage classification: From clas- sical machine learning methods to deep learning. Biomed- ical Signal Processing and Control, 77:103751, 2022.","title":"Automatic sleep stage classification: From classical machine learning methods to deep learning","authors":"Sekkal","year":"2022","venue":"Biomedical Signal Processing and Control","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Son et al., 2021] Wonchul Son, Jaemin Na, Junyong Choi, and Wonjun Hwang. Densely guided knowledge distilla- tion using multiple teacher assistants. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 9395-9404, 2021.","title":"Densely guided knowledge distillation using multiple teacher assistants","authors":"Son","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Sun et al., 2019] Chenglu Sun, Chen Chen, Wei Li, Jiahao Fan, and Wei Chen. A hierarchical neural network for sleep stage classification based on comprehensive feature learning and multi-flow sequence learning. IEEE journal of biomedical and health informatics, 24(5):1351-1366, 2019.","title":"A hierarchical neural network for sleep stage classification based on comprehensive feature learning and multi-flow sequence learning","authors":"Chenglu Sun, Chen Chen, Wei Li, Jiahao Fan, Wei Chen","year":"2019","venue":"IEEE journal of biomedical and health informatics","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Sundararajan et al., 2021] Kalaivani Sundararajan, Sonja Georgievska, Bart HW Te Lindert, Philip R Gehrman, Jennifer Ramautar, Diego R Mazzotti, Séverine Sabia, Michael N Weedon, Eus JW van Someren, Lars Ridder, et al. Sleep classification from wrist-worn accelerometer data using random forests. Scientific Reports, 11(1):1-10, 2021.","title":"Sleep classification from wrist-worn accelerometer data using random forests","authors":"Kalaivani Sundararajan, Sonja Georgievska, Bart Hw Te Lindert, Jennifer Philip R Gehrman, Diego R Ramautar, Séverine Mazzotti, Sabia, Michael N Weedon, J W Eus, Lars Van Someren, Ridder","year":"2021","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Supratak et al., 2017] Akara Supratak, Hao Dong, Chao Wu, and Yike Guo. Deepsleepnet: A model for automatic sleep stage scoring based on raw single-channel eeg. IEEE Transactions on Neural Systems and Rehabilitation Engi- neering, 25(11):1998-2008, 2017.","title":"Deepsleepnet: A model for automatic sleep stage scoring based on raw single-channel eeg","authors":"Akara Supratak, Hao Dong, Chao Wu, Yike Guo","year":"2017","venue":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Tian et al., 2019] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive representation distillation. arXiv preprint arXiv:1910.10699, 2019.","title":"Contrastive representation distillation","authors":"Yonglong Tian, Dilip Krishnan, Phillip Isola","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Tobaru et al., 2019] Akari Tobaru, Yusuke Tajima, and Keiki Takadama. Sleep stage estimation using heart rate variability divided by sleep cycle with relative evaluation. In AAAI Spring Symposium: Interpretable AI for Well- being, 2019.","title":"Sleep stage estimation using heart rate variability divided by sleep cycle with relative evaluation","authors":"Akari Tobaru, Yusuke Tajima, Keiki Takadama","year":"2019","venue":"AAAI Spring Symposium: Interpretable AI for Wellbeing","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Tzimourta et al., 2018] Katerina D Tzimourta, Athanasios Tsilimbaris, Katerina Tzioukalia, Alexandros T Tzallas, Markos G Tsipouras, Loukas G Astrakas, and Nikolaos Giannakeas. Eeg-based automatic sleep stage classifica- tion. Biomed J, 1(6), 2018.","title":"Eeg-based automatic sleep stage classification","authors":"Athanasios Katerina D Tzimourta, Katerina Tsilimbaris, Alexandros T Tzioukalia, Markos G Tzallas, Loukas G Tsipouras, Nikolaos Astrakas, Giannakeas","year":"2018","venue":"Biomed J","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wang and Yoon, 2021] Lin Wang and Kuk-Jin Yoon. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.","title":"Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks","authors":"Wang, Yoon ; ], Lin Wang, Kuk-Jin Yoon","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Xu et al., 2020] Guodong Xu, Ziwei Liu, Xiaoxiao Li, and Chen Change Loy. Knowledge distillation meets self- supervision. In European Conference on Computer Vision, pages 588-604. Springer, 2020.","title":"Knowledge distillation meets selfsupervision","authors":"Guodong Xu, Ziwei Liu, Xiaoxiao Li, Chen Change Loy","year":"2020","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yang et al., 2018] Yang Yang, Xiangwei Zheng, and Feng Yuan. A study on automatic sleep stage classification based on cnn-lstm. In Proceedings of the 3rd International Conference on Crowd Science and Engineering, pages 1- 5, 2018.","title":"A study on automatic sleep stage classification based on cnn-lstm","authors":"Yang Yang, Xiangwei Zheng, Feng Yuan","year":"2018","venue":"Proceedings of the 3rd International Conference on Crowd Science and Engineering","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Yim et al., 2017] Junho Yim, Donggyu Joo, Jihoon Bae, and Junmo Kim. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4133-4141, 2017.","title":"A gift from knowledge distillation: Fast optimization, network minimization and transfer learning","authors":"Yim","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhang and Wu, 2017] Junming Zhang and Yan Wu. A new method for automatic sleep stage classification. IEEE transactions on biomedical circuits and systems, 11(5):1097-1110, 2017.","title":"A new method for automatic sleep stage classification","authors":"Wu ; ] Junming Zhang, Yan Zhang, Wu","year":"2017","venue":"IEEE transactions on biomedical circuits and systems","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhang et al., 2018] Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. Deep mutual learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4320-4328, 2018.","title":"Deep mutual learning","authors":"Ying Zhang, Tao Xiang, Timothy M Hospedales, Huchuan Lu","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhao et al., 2022] Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang. Decoupled knowledge dis- tillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11953- 11962, 2022.","title":"Decoupled knowledge distillation","authors":"Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, Jiajun Liang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}