{"type":"ScholarlyArticle","title":"Learning Sound Events from Webly Labeled Data","authors":[{"name":"Anurag Kumar","slug":"anurag-kumar","given":"Anurag","family":"Kumar","affiliation":"Language Technologies Institute , School of Computer Science , Carnegie Mellon University"},{"name":"Ankit Shah","slug":"ankit-shah","given":"Ankit","family":"Shah","affiliation":"Language Technologies Institute , School of Computer Science , Carnegie Mellon University"},{"name":"Alexander Hauptmann","slug":"alexander-hauptmann","given":"Alexander","family":"Hauptmann","affiliation":"Language Technologies Institute , School of Computer Science , Carnegie Mellon University"},{"name":"Bhiksha Raj","slug":"bhiksha-raj","given":"Bhiksha","family":"Raj","affiliation":"Language Technologies Institute , School of Computer Science , Carnegie Mellon University"}],"abstract":"In the last couple of years, weakly labeled learning has turned out to be an exciting approach for audio event detection. In this work, we introduce webly labeled learning for sound events which aims to remove human supervision altogether from the learning process. We first develop a method of obtaining labeled audio data from the web (albeit noisy), in which no manual labeling is involved. We then describe methods to efficiently learn from these webly labeled audio recordings. In our proposed system, WeblyNet, two deep neural networks co-teach each other to robustly learn from webly labeled data, leading to around 17% relative improvement over the baseline method. The method also involves transfer learning to obtain efficient representations.","doi":"10.24963/ijcai.2019/384","url":"https://ijcai.org/proceedings/2019/384","pdf":"https://ijcai.org/proceedings/2019/0384.pdf","markdown":"https://ijcai.org/proceedings/2019/384.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/384","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":384,"pages":{"first":2772,"last":2778},"keywords":["Machine Learning: Deep Learning","Machine Learning: Multi-instance;Multi-label;Multi-view learning","Machine Learning: Classification","Machine Learning Applications: Applications of Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Banerjee et al., 2005] Arindam Banerjee, Srujana Merugu, Inder- jit S Dhillon, and Joydeep Ghosh. Clustering with bregman di- vergences. Journal of machine learning research, 6(Oct), 2005.","title":"Clustering with bregman divergences","authors":"Arindam Banerjee, Srujana Merugu, Joydeep Inderjit S Dhillon, Ghosh","year":"2005","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blum and Mitchell, 1998] Avrim Blum and Tom Mitchell. Com- bining labeled and unlabeled data with co-training. In Proceed- ings of the eleventh annual conference on Computational learn- ing theory, pages 92-100. ACM, 1998.","title":"Combining labeled and unlabeled data with co-training","authors":"Avrim Blum, Tom Mitchell","year":"1998","venue":"Proceedings of the eleventh annual conference on Computational learning theory","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Gupta, 2015] Xinlei Chen and Abhinav Gupta. Webly supervised learning of convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision, 2015.","title":"Webly supervised learning of convolutional networks","authors":"Chen, Gupta, Xinlei Chen, Abhinav Gupta","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chou et al., 2018] Szu-Yu Chou, Jyh-Shing Roger Jang, and Yi- Hsuan Yang. Learning to recognize transient sound events using attentional supervision. In IJCAI, pages 3336-3342, 2018.","title":"Learning to recognize transient sound events using attentional supervision","authors":"Szu-Yu Chou, Jyh-Shing Roger Jang, Yi-Hsuan Yang","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/463"},{"position":5,"raw":"[Divvala et al., 2014] Santosh K Divvala, Ali Farhadi, and Carlos Guestrin. Learning everything about anything: Webly-supervised visual concept learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3270-3277, 2014.","title":"Learning everything about anything: Webly-supervised visual concept learning","authors":"Ali Santosh K Divvala, Carlos Farhadi, Guestrin","year":"2014","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fergus et al., 2010] Rob Fergus, Li Fei-Fei, Pietro Perona, and Andrew Zisserman. Learning object categories from internet im- age searches. Proceedings of the IEEE, 98(8):1453-1466, 2010.","title":"Learning object categories from internet image searches","authors":"Rob Fergus, Li Fei-Fei, Pietro Perona, Andrew Zisserman","year":"2010","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fonseca et al., 2017] Eduardo Fonseca, Jordi Pons Puig, Xavier Favory, Frederic Font Corbera, Dmitry Bogdanov, Andres Ferraro, Sergio Oramas, Alastair Porter, and Xavier Serra. Freesound datasets: a platform for the creation of open audio datasets. In Hu X, Cunningham SJ, Turnbull D, Duan Z, edi- tors. Proceedings of the 18th ISMIR Conference; 2017 oct 23-27; Suzhou, China.[Canada]: International Society for Music Infor- mation Retrieval; 2017. p. 486-93., 2017.","title":"Freesound datasets: a platform for the creation of open audio datasets","authors":"Eduardo Fonseca, Jordi Pons Puig, Xavier Favory, Frederic Font Corbera, Dmitry Bogdanov, Andres Ferraro, Sergio Oramas, Alastair Porter, Xavier Serra","year":"2017","venue":"Proceedings of the 18th ISMIR Conference","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Frénay and Verleysen, 2014] Benoît Frénay and Michel Verleysen. Classification in the presence of label noise: a survey. IEEE transactions on neural networks and learning systems, 2014.","title":"Classification in the presence of label noise: a survey","authors":"Verleysen Frénay, Benoît Frénay, Michel Verleysen","year":"2014","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gemmeke et al., 2017] Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 776-780. IEEE, 2017.","title":"Audio set: An ontology and human-labeled dataset for audio events","authors":"Jort F Gemmeke, P W Daniel, Dylan Ellis, Aren Freedman, Wade Jansen, R Channing Lawrence, Manoj Moore, Marvin Plakal, Ritter","year":"2017","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goldberger and Ben-Reuven, 2016] Jacob Goldberger and Ehud Ben-Reuven. Training deep neural-networks using a noise adap- tation layer. 2016.","title":"Training deep neural-networks using a noise adaptation layer","authors":"Ben-Reuven ; Goldberger, Jacob Goldberger, Ehud Ben-Reuven","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Han et al., 2018] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co- teaching: robust training deep neural networks with extremely noisy labels. 2018.","title":"Coteaching: robust training deep neural networks with extremely noisy labels","authors":"Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, Masashi Sugiyama","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hershey et al., 2017] Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Sey- bold, et al. Cnn architectures for large-scale audio classification. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 131-135. IEEE, 2017.","title":"Cnn architectures for large-scale audio classification","authors":"Shawn Hershey, Sourish Chaudhuri, P W Daniel, Ellis, Aren Jort F Gemmeke, R Channing Jansen, Manoj Moore, Devin Plakal, Rif A Platt, Bryan Saurous, Seybold","year":"2017","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by re- ducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.","title":"Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kumar and Raj, 2016] Anurag Kumar and Bhiksha Raj. Audio event detection using weakly labeled data. In 24th ACM Inter- national Conference on Multimedia. ACM Multimedia, 2016.","title":"Audio event detection using weakly labeled data","authors":"Raj ; Kumar, Anurag Kumar, Bhiksha Raj","year":"2016","venue":"24th ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kumar and Raj, 2017] Anurag Kumar and Bhiksha Raj. Audio event and scene recognition: A unified approach using strongly and weakly labeled data. In Neural Networks (IJCNN), 2017 In- ternational Joint Conference on, pages 3475-3482. IEEE, 2017.","title":"Audio event and scene recognition: A unified approach using strongly and weakly labeled data","authors":"Raj ; Kumar, Anurag Kumar, Bhiksha Raj","year":"2017","venue":"Neural Networks (IJCNN), 2017 International Joint Conference on","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kumar et al., 2017] Anurag Kumar, Bhiksha Raj, and Ndapandula Nakashole. Discovering sound concepts and acoustic relations in text. submitted IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017.","title":"Discovering sound concepts and acoustic relations in text","authors":"Anurag Kumar, Bhiksha Raj, Ndapandula Nakashole","year":"2017","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kumar et al., 2018] Anurag Kumar, M. Khadkevich, and C. Fu- gen. Knowledge transfer from weakly labeled audio using con- volutional neural network for sound events and scenes. In Acous- tics, Speech and Signal Processing (ICASSP), 2018 IEEE Inter- national Conference on. IEEE, 2018.","title":"Knowledge transfer from weakly labeled audio using convolutional neural network for sound events and scenes","authors":"Anurag Kumar, M Khadkevich, C Fugen","year":"2018","venue":"Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kumar, 2018] Anurag Kumar. Acoustic Intelligence in Machines. PhD thesis, Carnegie Mellon University, 2018.","title":"Acoustic Intelligence in Machines","authors":"Anurag Kumar","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liang et al., 2016] Junwei Liang, Lu Jiang, Deyu Meng, and Alexander Hauptmann. Learning to detect concepts from webly- labeled video data. IJCAI, 2016.","title":"Learning to detect concepts from weblylabeled video data","authors":"Junwei Liang, Lu Jiang, Deyu Meng, Alexander Hauptmann","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lu et al., 2018] Xugang Lu, Peng Shen, Sheng Li, Yu Tsao, and Hisashi Kawai. Temporal attentive pooling for acoustic event detection. Proc. Interspeech 2018, pages 1354-1357, 2018.","title":"Temporal attentive pooling for acoustic event detection","authors":"Xugang Lu, Peng Shen, Sheng Li, Yu Tsao, Hisashi Kawai","year":"2018","venue":"Proc. Interspeech 2018","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Malach and Shalev-Shwartz, 2017] Eran Malach and Shai Shalev- Shwartz. Decoupling\" when to update\" from\" how to update\". In Advances in Neural Information Processing Systems, pages 960- 970, 2017.","title":"Decoupling\" when to update\" from\" how to update","authors":"Shalev-Shwartz ; Malach, Eran Malach, Shai Shalev-Shwartz","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[McFee et al., 2018] Brian McFee, Justin Salamon, and Juan Pablo Bello. Adaptive pooling operators for weakly labeled sound event detection. arXiv preprint arXiv:1804.10070, 2018.","title":"Adaptive pooling operators for weakly labeled sound event detection","authors":"Brian Mcfee, Justin Salamon, Juan Pablo Bello","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Reed et al., 2014] Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy labels with bootstrap- ping. arXiv preprint arXiv:1412.6596, 2014.","title":"Training deep neural networks on noisy labels with bootstrapping","authors":"Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, Andrew Rabinovich","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sun, 2013] Shiliang Sun. A survey of multi-view machine learn- ing. Neural Computing and Applications, 23(7-8):2031-2038, 2013.","title":"A survey of multi-view machine learning","authors":"Shiliang Sun","year":"2013","venue":"Neural Computing and Applications","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tanaka et al., 2018] Daiki Tanaka, Daiki Ikami, Toshihiko Ya- masaki, and Kiyoharu Aizawa. Joint optimization framework for learning with noisy labels. arXiv preprint arXiv:1803.11364, 2018.","title":"Joint optimization framework for learning with noisy labels","authors":"Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, Kiyoharu Aizawa","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xia et al., 2014] Yan Xia, Xudong Cao, Fang Wen, and Jian Sun. Well begun is half done: Generating high-quality seeds for auto- matic image dataset construction from web. In European Confer- ence on Computer Vision, pages 387-400. Springer, 2014.","title":"Well begun is half done: Generating high-quality seeds for automatic image dataset construction from web","authors":"Yan Xia, Xudong Cao, Fang Wen, Jian Sun","year":"2014","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu et al., 2017] Yong Xu, Qiuqiang Kong, Qiang Huang, Wenwu Wang, and Mark D Plumbley. Attention and localization based on a deep convolutional recurrent model for weakly supervised audio tagging. arXiv preprint arXiv:1703.06052, 2017.","title":"Attention and localization based on a deep convolutional recurrent model for weakly supervised audio tagging","authors":"Yong Xu, Qiuqiang Kong, Qiang Huang, Wenwu Wang, Mark D Plumbley","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2016a] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530, 2016.","title":"Understanding deep learning requires rethinking generalization","authors":"Zhang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2016b] Han Zhang, Tao Xu, Mohamed Elhoseiny, Xiaolei Huang, Shaoting Zhang, Ahmed Elgammal, and Dim- itris Metaxas. Spda-cnn: Unifying semantic part detection and abstraction for fine-grained recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1143-1152, 2016.","title":"Spda-cnn: Unifying semantic part detection and abstraction for fine-grained recognition","authors":"Zhang","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}