{"type":"ScholarlyArticle","title":"Robust Audio Adversarial Example for a Physical Attack","authors":[{"name":"Hiromu Yakura","slug":"hiromu-yakura","given":"Hiromu","family":"Yakura","affiliation":"University of Tsukuba; RIKEN Center for Advanced Intelligence Project"},{"name":"Jun Sakuma","slug":"jun-sakuma","given":"Jun","family":"Sakuma","affiliation":"University of Tsukuba; RIKEN Center for Advanced Intelligence Project"}],"abstract":"We propose a method to generate audio adversarial examples that can attack a state-of-the-art speech recognition model in the physical world. Previous work assumes that generated adversarial examples are directly fed to the recognition model, and is not able to perform such a physical attack because of reverberation and noise from playback environments. In contrast, our method obtains robust adversarial examples by simulating transformations caused by playback or recording in the physical world and incorporating the transformations into the generation process. Evaluation and a listening experiment demonstrated that our adversarial examples are able to attack without being noticed by humans. This result suggests that audio adversarial examples generated by the proposed method may become a real threat.","doi":"10.24963/ijcai.2019/741","url":"https://ijcai.org/proceedings/2019/741","pdf":"https://ijcai.org/proceedings/2019/0741.pdf","markdown":"https://ijcai.org/proceedings/2019/741.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/741","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":741,"pages":{"first":5334,"last":5341},"keywords":["Machine Learning: Adversarial Machine Learning","Natural Language Processing: Speech","Machine Learning: Deep Learning","Multidisciplinary Topics and Applications: Security and Privacy"],"videos":[],"references":[{"position":1,"raw":"[Alzantot et al., 2018] Moustafa Alzantot, Bharathan Balaji, and Mani B. Srivastava. Did you hear that? adversarial ex- amples against automatic speech recognition. In Proceed- ings of the 2017 NIPS Workshop on Machine Deception, pages 1-6, 2018.","title":"Did you hear that? adversarial examples against automatic speech recognition","authors":"Moustafa Alzantot, Bharathan Balaji, Mani B Srivastava","year":"2018","venue":"Proceedings of the 2017 NIPS Workshop on Machine Deception","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amodei et al., 2016] Dario Amodei, Sundaram Anantha- narayanan, Rishita Anubhai, Jingliang Bai, Eric Batten- berg, Carl Case, Jared Casper, Bryan Catanzaro, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse Engel, Linxi Fan, Christopher Fougner, Awni Y. Hannun, Billy Jun, Tony Han, Patrick LeGres- ley, Xiangang Li, Libby Lin, Sharan Narang, Andrew Y. Ng, Sherjil Ozair, Ryan Prenger, Sheng Qian, Jonathan Raiman, Sanjeev Satheesh, David Seetapun, Shubho Sen- gupta, Chong Wang, Yi Wang, Zhiqian Wang, Bo Xiao, Yan Xie, Dani Yogatama, Jun Zhan, and Zhenyao Zhu. Deep speech 2: End-to-end speech recognition in english and mandarin. In Proceedings of the 33rd International Conference on Machine Learning, volume 48, pages 173- 182, 2016.","title":"Deep speech 2: End-to-end speech recognition in english and mandarin","authors":"Dario Amodei, Rishita Sundaram Ananthanarayanan, Jingliang Anubhai, Eric Bai, Carl Battenberg, Jared Case, Bryan Casper, Jingdong Catanzaro, Mike Chen, Adam Chrzanowski, Greg Coates, Erich Diamos, Jesse Elsen, Linxi Engel, Christopher Fan, Fougner, Y Awni, Billy Hannun, Tony Jun, Patrick Han, Xiangang Legresley, Libby Li, Sharan Lin, Andrew Y Narang, Sherjil Ng, Ryan Ozair, Sheng Prenger, Jonathan Qian, Sanjeev Raiman, David Satheesh, Shubho Seetapun, Chong Sengupta, Yi Wang, Zhiqian Wang, Bo Wang, Yan Xiao, Dani Xie, Jun Yogatama, Zhenyao Zhan, Zhu","year":"2016","venue":"Proceedings of the 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Athalye et al., 2018] Anish Athalye, Logan Engstrom, An- drew Ilyas, and Kevin Kwok. Synthesizing robust adver- sarial examples. In Proceedings of the 35rd International Conference on Machine Learning, pages 284-293, 2018.","title":"Synthesizing robust adversarial examples","authors":"Anish Athalye, Logan Engstrom, Andrew Ilyas, Kevin Kwok","year":"2018","venue":"Proceedings of the 35rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carlini and Wagner, 2018] Nicholas Carlini and David A. Wagner. Audio adversarial examples: Targeted attacks on speech-to-text. In Proceedings of the 1st IEEE Workshop on Deep Learning and Security, pages 1-7, 2018.","title":"Audio adversarial examples: Targeted attacks on speech-to-text","authors":"Nicholas Carlini, David A Wagner","year":"2018","venue":"Proceedings of the 1st IEEE Workshop on Deep Learning and Security","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cissé et al., 2017] Moustapha Cissé, Yossi Adi, Natalia Neverova, and Joseph Keshet. Houdini: Fooling deep structured visual and speech recognition models with ad- versarial examples. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems, pages 6980-6990, 2017.","title":"Houdini: Fooling deep structured visual and speech recognition models with adversarial examples","authors":"Moustapha Cissé, Yossi Adi, Natalia Neverova, Joseph Keshet","year":"2017","venue":"Proceedings of the 31st Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2015] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proceedings of the 3rd Inter- national Conference on Learning Representations, pages 1-11, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"Proceedings of the 3rd International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hannun et al., 2014] Awni Y. Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, and Andrew Y. Ng. Deep speech: Scaling up end- to-end speech recognition. arXiv, 1412.5567:1-12, 2014.","title":"Deep speech: Scaling up endto-end speech recognition","authors":"Y Awni, Carl Hannun, Jared Case, Bryan Casper, Greg Catanzaro, Erich Diamos, Ryan Elsen, Sanjeev Prenger, Shubho Satheesh, Adam Sengupta, Andrew Y Coates, Ng","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hansen and Pellom, 1998] John H. L. Hansen and Bryan L. Pellom. An effective quality evaluation protocol for speech enhancement algorithms. In Proceedings of the 1998 In- ternational Conference on Spoken Language Processing, pages 2819-2822, 1998.","title":"An effective quality evaluation protocol for speech enhancement algorithms","authors":"H L John, Bryan L Hansen, Pellom","year":"1998","venue":"Proceedings of the 1998 International Conference on Spoken Language Processing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Härmä, 2001] Aki Härmä. Acoustic measurement data from the varechoic chamber. Technical report, Agere Systems, 2001.","title":"Acoustic measurement data from the varechoic chamber","authors":"Aki Härmä","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jeub et al., 2009] Marco Jeub, Magnus Schafer, and Peter Vary. A binaural room impulse response database for the evaluation of dereverberation algorithms. In Proceedings of the 16th International Conference on Digital Signal Processing, pages 1-5, July 2009.","title":"A binaural room impulse response database for the evaluation of dereverberation algorithms","authors":"Marco Jeub, Magnus Schafer, Peter Vary","year":"2009","venue":"Proceedings of the 16th International Conference on Digital Signal Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceed- ings of the 3rd International Conference on Learning Rep- resentations, pages 1-13, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"Proceedings of the 3rd International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kinoshita et al., 2013] Keisuke Kinoshita, Marc Delcroix, Takuya Yoshioka, Tomohiro Nakatani, Armin Sehr, Wal- ter Kellermann, and Roland Maas. The reverb challenge: Acommon evaluation framework for dereverberation and recognition of reverberant speech. In Proceedings of the 2013 IEEE Workshop on Applications of Signal Process- ing to Audio and Acoustics, pages 1-4, 2013.","title":"The reverb challenge: Acommon evaluation framework for dereverberation and recognition of reverberant speech","authors":"Keisuke Kinoshita, Marc Delcroix, Takuya Yoshioka, Tomohiro Nakatani, Armin Sehr, Walter Kellermann, Roland Maas","year":"2013","venue":"Proceedings of the 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- offrey E. Hinton. Deep learning. Nature, 521(7553):436- 444, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey E Hinton","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nakamura et al., 2000] Satoshi Nakamura, Kazuo Hiyane, Futoshi Asano, Takanobu Nishiura, and Takeshi Yamada. Acoustical sound database in real environments for sound scene understanding and hands-free speech recognition. In Proceedings of the 2nd Language Resources and Evalua- tion Conference, pages 965-968, 2000.","title":"Acoustical sound database in real environments for sound scene understanding and hands-free speech recognition","authors":"Satoshi Nakamura, Kazuo Hiyane, Futoshi Asano, Takanobu Nishiura, Takeshi Yamada","year":"2000","venue":"Proceedings of the 2nd Language Resources and Evaluation Conference","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Peddinti et al., 2015] Vijayaditya Peddinti, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Reverberation ro- bust acoustic modeling using i-vectors with time delay neural networks. In Proceedings of the 16th Annual Con- ference of the International Speech Communication Asso- ciation, pages 2440-2444, 2015.","title":"Reverberation robust acoustic modeling using i-vectors with time delay neural networks","authors":"Peddinti","year":"2015","venue":"Proceedings of the 16th Annual Conference of the International Speech Communication Association","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Povey et al., 2011] Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, Jan Silovsky, Georg Stemmer, and Karel Vesely. The kaldi speech recognition toolkit. In Pro- ceedings of the 2011 IEEE Workshop on Automatic Speech Recognition and Understanding, pages 1-4, 2011.","title":"The kaldi speech recognition toolkit","authors":"Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, Jan Silovsky, Georg Stemmer, Karel Vesely","year":"2011","venue":"Proceedings of the 2011 IEEE Workshop on Automatic Speech Recognition and Understanding","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sainath and Parada, 2015] Tara N. Sainath and Carolina Parada. Convolutional neural networks for small-footprint keyword spotting. In Proceedings of the 16th Annual Con- ference of the International Speech Communication Asso- ciation, pages 1478-1482, 2015.","title":"Convolutional neural networks for small-footprint keyword spotting","authors":"Tara N Sainath, Carolina Parada","year":"2015","venue":"Proceedings of the 16th Annual Conference of the International Speech Communication Association","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schönherr et al., 2018] Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa. Ad- versarial attacks against automatic speech recognition sys- tems via psychoacoustic hiding. arXiv, 1808.05665:1-18, 2018.","title":"Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding","authors":"Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, Dorothea Kolossa","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Er- han, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Proceedings of the 2nd International Conference on Learning Representations, pages 1-10, 2014.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2014","venue":"Proceedings of the 2nd International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Taori et al., 2018] Rohan Taori, Amog Kamsetty, Brenton Chu, and Nikita Vemuri. Targeted adversarial examples for black box audio systems. arXiv, 1805.07820:1-9, 2018.","title":"Targeted adversarial examples for black box audio systems","authors":"Rohan Taori, Amog Kamsetty, Brenton Chu, Nikita Vemuri","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wen et al., 2006] Jimi Y. C. Wen, Nikolay D. Gaubitch, Emanuël A. P. Habets, Tony Myatt, and Patrick A. Nay- lor. Evaluation of speech dereverberation algorithms using the mardy database. In Proceedings of the 10th Interna- tional Workshop on Acoustic Signal Enhancement, pages 1-4, 2006.","title":"Evaluation of speech dereverberation algorithms using the mardy database","authors":"Y C Jimi, Nikolay D Wen, Gaubitch, A P Emanuël, Tony Habets, Patrick A Myatt, Naylor","year":"2006","venue":"Proceedings of the 10th International Workshop on Acoustic Signal Enhancement","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yuan et al., 2018] Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiaofeng Wang, and Carl A. Gunter. Com- mandersong: A systematic approach for practical ad- versarial voice recognition. In Proceedings of the 27th USENIX Security Symposium, pages 49-64, 2018.","title":"Commandersong: A systematic approach for practical adversarial voice recognition","authors":"Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiaofeng Wang, Carl A Gunter","year":"2018","venue":"Proceedings of the 27th USENIX Security Symposium","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}