{"type":"ScholarlyArticle","title":"A Mobile Application for Sound Event Detection","authors":[{"name":"Yingwei Fu","slug":"yingwei-fu","given":"Yingwei","family":"Fu","affiliation":"National Key Laboratory of Parallel and Distributed Processing; College of Computer , National University of Defense Technology"},{"name":"Kele Xu","slug":"kele-xu","given":"Kele","family":"Xu","affiliation":"National Key Laboratory of Parallel and Distributed Processing; College of Computer , National University of Defense Technology"},{"name":"Haibo Mi","slug":"haibo-mi","given":"Haibo","family":"Mi","affiliation":"National Key Laboratory of Parallel and Distributed Processing; College of Computer , National University of Defense Technology"},{"name":"Huaimin Wang","slug":"huaimin-wang","given":"Huaimin","family":"Wang","affiliation":"National Key Laboratory of Parallel and Distributed Processing; College of Computer , National University of Defense Technology"},{"name":"Dezhi Wang","slug":"dezhi-wang","given":"Dezhi","family":"Wang","affiliation":"College of Meteorology and Oceanography , National University of Defense Technology yingwei fu"},{"name":"Boqing Zhu","slug":"boqing-zhu","given":"Boqing","family":"Zhu","affiliation":"National Key Laboratory of Parallel and Distributed Processing; College of Computer , National University of Defense Technology"}],"abstract":"Sound event detection is intended to analyze and recognize the sound events in audio streams and it has widespread applications in real life. Recently, deep neural networks such as convolutional recurrent neural networks have shown state-of-the-art performance in this task. However, the previous methods were designed and implemented on devices with rich computing resources, and there are few applications on mobile devices. This paper focuses on the solution on the mobile platform for sound event detection. The architecture of the solution includes offline training and online detection. During offline training process, multi model-based distillation method is used to compress model to enable real-time detection. The online detection process includes acquisition of sensor data, processing of audio signals, and detecting and recording of sound events. Finally, we implement an application on the mobile device that can detect sound events in near real time.","doi":"10.24963/ijcai.2019/941","url":"https://ijcai.org/proceedings/2019/941","pdf":"https://ijcai.org/proceedings/2019/0941.pdf","markdown":"https://ijcai.org/proceedings/2019/941.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/941","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Demos","track":"Demos","number":941,"pages":{"first":6515,"last":6517},"keywords":["AI: Applications","AI: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning Phrase Representations using RNN Encoder-Decoder for Statis- tical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724-1734. Association for Computational Linguistics, 2014.","title":"Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation","authors":"Cho","year":"2014","venue":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chollet, 2017] Francois Chollet. Xception: Deep Learning with Depthwise Separable Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVRP), pages 1800-1807. IEEE, 2017.","title":"Xception: Deep Learning with Depthwise Separable Convolutions","authors":"Francois Chollet","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVRP)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Espi et al., 2015] Miquel Espi, Masakiyo Fujimoto, Keisuke Kinoshita, and Tomohiro Nakatani. Exploiting spectro-temporal locality in deep learning based acoustic event detection. EURASIP Journal on Audio, Speech, and Music Processing, 2015(1):26, 2015.","title":"Exploiting spectro-temporal locality in deep learning based acoustic event detection","authors":"Miquel Espi, Masakiyo Fujimoto, Keisuke Kinoshita, Tomohiro Nakatani","year":"2015","venue":"EURASIP Journal on Audio, Speech, and Music Processing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gemmeke et al., 2017] Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Chan- ning Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), pages 776- 780. IEEE, 2017.","title":"Audio set: An ontology and human-labeled dataset for audio events","authors":"Jort F Gemmeke, P W Daniel, Dylan Ellis, Aren Freedman, Wade Jansen, R Channing Lawrence, Manoj Moore, Marvin Plakal, Ritter","year":"2017","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Harma et al., 2005] Aki Harma, Martin F McKinney, and Janto Skowronek. Automatic surveillance of the acoustic activity in our living environment. In 2005 IEEE Interna- tional Conference on Multimedia and Expo, pages 4 pp.-. IEEE, 2005.","title":"Automatic surveillance of the acoustic activity in our living environment","authors":"Harma","year":"2005","venue":"2005 IEEE International Conference on Multimedia and Expo","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVRP), pages 770- 778. IEEE, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVRP)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. CoRR, abs/1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Mesaros et al., 2016] Annamaria Mesaros, Toni Heittola, and Tuomas Virtanen. Metrics for polyphonic sound event detection. Applied Sciences, 6(6):162, 2016.","title":"Metrics for polyphonic sound event detection","authors":"Annamaria Mesaros, Toni Heittola, Tuomas Virtanen","year":"2016","venue":"Applied Sciences","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Parascandolo et al., 2016] Giambattista Parascandolo, Heikki Huttunen, and Tuomas Virtanen. Recurrent neural networks for polyphonic sound event detection in real life recordings. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6440-6444. IEEE, 2016.","title":"Recurrent neural networks for polyphonic sound event detection in real life recordings","authors":"Giambattista Parascandolo, Heikki Huttunen, Tuomas Virtanen","year":"2016","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Wang et al., 2016] Yun Wang, Leonardo Neves, and Florian Metze. Audio-based multimedia event detection using deep recurrent neural networks. In 2016 IEEE Interna- tional Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 2742-2746. IEEE, 2016.","title":"Audio-based multimedia event detection using deep recurrent neural networks","authors":"Yun Wang, Leonardo Neves, Florian Metze","year":"2016","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Wang et al., 2018] Dezhi Wang, Lilun Zhang, Changchun Bao, Kele Xu, Boqing Zhu, and Qiuqiang Kong. Weakly supervised CRNN system for sound event detection with large-scale unlabeled in-domain data. CoRR, abs/1811.00301, 2018.","title":"Weakly supervised CRNN system for sound event detection with large-scale unlabeled in-domain data","authors":"Wang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Xu et al., 2018] Kele Xu, Dawei Feng, Haibo Mi, Bo- qing Zhu, Dezhi Wang, Lilun Zhang, Hengxing Cai, and Shuwen Liu. Mixup-based acoustic scene classifi- cation using multi-channel convolutional neural network. In Pacific Rim Conference on Multimedia, pages 14-23. Springer, 2018.","title":"Mixup-based acoustic scene classification using multi-channel convolutional neural network","authors":"Kele Xu, Dawei Feng, Haibo Mi, Boqing Zhu, Dezhi Wang, Lilun Zhang, Hengxing Cai, Shuwen Liu","year":"2018","venue":"Pacific Rim Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Xu et al., 2019] Kele Xu, Boqing Zhu, Qiuqiang Kong, Haibo Mi, Bo Ding, Dezhi Wang, and Huaimin Wang. General audio tagging with ensembling convolutional neu- ral network and statistical features. The Journal of the Acoustical Society of America, 145(3), 2019.","title":"General audio tagging with ensembling convolutional neural network and statistical features","authors":"Kele Xu, Boqing Zhu, Qiuqiang Kong, Haibo Mi, Bo Ding, Dezhi Wang, Huaimin Wang","year":"2019","venue":"The Journal of the Acoustical Society of America","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Zhu et al., 2018] Boqing Zhu, Kele Xu, Dezhi Wang, Lilun Zhang, Bo Li, and Yuxing Peng. Environmental Sound Classification Based on Multi-temporal Resolution Convo- lutional Neural Network Combining with Multi-level Fea- tures. In Pacific Rim Conference on Multimedia, pages 528-537. Springer, 2018.","title":"Environmental Sound Classification Based on Multi-temporal Resolution Convolutional Neural Network Combining with Multi-level Features","authors":"Boqing Zhu, Kele Xu, Dezhi Wang, Lilun Zhang, Bo Li, Yuxing Peng","year":"2018","venue":"Pacific Rim Conference on Multimedia","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}