{"type":"ScholarlyArticle","title":"Generating Robust Audio Adversarial Examples with Temporal Dependency","authors":[{"name":"Hongting Zhang","slug":"hongting-zhang","given":"Hongting","family":"Zhang","affiliation":"Huazhong University of Science and Technology"},{"name":"Pan Zhou","slug":"pan-zhou","given":"Pan","family":"Zhou","affiliation":""},{"name":"Qiben Yan","slug":"qiben-yan","given":"Qiben","family":"Yan","affiliation":"Michigan State University"},{"name":"Xiao-Yang Liu","slug":"xiao-yang-liu","given":"Xiao-Yang","family":"Liu","affiliation":"Columbia University"}],"abstract":"Audio adversarial examples, imperceptible to humans, have been constructed to attack automatic speech recognition (ASR) systems. However, the adversarial examples generated by existing approaches usually incorporate noticeable noises, especially during the periods of silences and pauses. Moreover, the added noises often break temporal dependency property of the original audio, which can be easily detected by state-of-the-art defense mechanisms. In this paper, we propose a new Iterative Proportional Clipping (IPC) algorithm that preserves temporal dependency in audios for generating more robust adversarial examples. We are motivated by an observation that the temporal dependency in audios imposes a significant effect on human perception. Following our observation, we leverage a proportional clipping strategy to reduce noise during the low-intensity periods. Experimental results and user study both suggest that the generated adversarial examples can significantly reduce human-perceptible noises and resist the defenses based on the temporal structure.","doi":"10.24963/ijcai.2020/438","url":"https://ijcai.org/proceedings/2020/438","pdf":"https://ijcai.org/proceedings/2020/0438.pdf","markdown":"https://ijcai.org/proceedings/2020/438.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/438","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":438,"pages":{"first":3167,"last":3173},"keywords":["Machine Learning: Adversarial Machine Learning","Natural Language Processing: Speech"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24109/file_24109.m3u8","length_sec":287,"provider":"hls"}],"references":[{"position":1,"raw":"[Abdullah et al., 2019] Hadi Abdullah, Washington Garcia, Christian Peeters, Patrick Traynor, Kevin RB Butler, and Joseph Wilson. Practical hidden voice attacks against speech and speaker recognition systems. arXiv preprint arXiv:1904.05734, 2019.","title":"Practical hidden voice attacks against speech and speaker recognition systems","authors":"Hadi Abdullah, Washington Garcia, Christian Peeters, Patrick Traynor, Kevin Rb Butler, Joseph Wilson","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Assembly, 1994] ITU Radiocommunication Assembly. Methods for the subjective assessment of small impair- ments in audio systems including multichannel sound systems, 1994.","title":"ITU Radiocommunication Assembly. Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems","authors":null,"year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Biggio et al., 2013] Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European Confer- ence on Machine Learning and Knowledge Discovery in Databases, pages 387-402. Springer, 2013.","title":"Evasion attacks against machine learning at test time","authors":"Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, Fabio Roli","year":"2013","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39-57. IEEE, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Carlini and Wagner, 2018] Nicholas Carlini and David Wagner. Audio adversarial examples: Targeted attacks on speech-to-text. In 2018 IEEE Security and Privacy Workshops (SPW), pages 1-7. IEEE, 2018.","title":"Audio adversarial examples: Targeted attacks on speech-to-text","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2018","venue":"2018 IEEE Security and Privacy Workshops (SPW)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Carlini et al., 2016] Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David Wagner, and Wenchao Zhou. Hidden voice com- mands. In 25th USENIX Security Symposium (USENIX Security 16), pages 513-530, 2016.","title":"Hidden voice commands","authors":"Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David Wagner, Wenchao Zhou","year":"2016","venue":"25th USENIX Security Symposium (USENIX Security 16)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cisse et al., 2017] Moustapha M Cisse, Yossi Adi, Natalia Neverova, and Joseph Keshet. Houdini: Fooling deep structured visual and speech recognition models with ad- versarial examples. In Advances in Neural Information Processing Systems 30, pages 6977-6987.","title":"Houdini: Fooling deep structured visual and speech recognition models with adversarial examples","authors":"Yossi Moustapha M Cisse, Natalia Adi, Joseph Neverova, Keshet","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Collobert et al., 2016] Ronan Collobert, Christian Puhrsch, and Gabriel Synnaeve. Wav2letter: an end-to-end convnet-based speech recognition system. arXiv preprint arXiv:1609.03193, 2016.","title":"Christian Puhrsch, and Gabriel Synnaeve. Wav2letter: an end-to-end convnet-based speech recognition system","authors":"Collobert","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fayek, 2016] Haytham Fayek. Speech processing for ma- chine learning: Filter banks, mel-frequency cepstral coef- ficients (mfccs) and what's in-between, 2016.","title":"Speech processing for machine learning: Filter banks, mel-frequency cepstral coefficients (mfccs) and what's in-between","authors":"Haytham Fayek","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gong and Poellabauer, 2017] Yuan Gong and Chris- tian Poellabauer. Crafting adversarial examples for speech paralinguistics applications. arXiv preprint arXiv:1711.03280, 2017.","title":"Crafting adversarial examples for speech paralinguistics applications","authors":"Yuan Gong, Christian Poellabauer","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goodfellow et al., 2015] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Machine Learning, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huber and Kollmeier, 2006] Rainer Huber and Birger Kollmeier. Pemo-q-a new method for objective audio quality assessment using a model of auditory perception. IEEE Transactions on Audio, Speech and Language Processing, 14(6):1902-1911, 2006.","title":"Pemo-q-a new method for objective audio quality assessment using a model of auditory perception","authors":"Kollmeier Huber, Rainer Huber, Birger Kollmeier","year":"2006","venue":"IEEE Transactions on Audio, Speech and Language Processing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kreuk et al., 2018] Felix Kreuk, Yossi Adi, Moustapha Cisse, and Joseph Keshet. Fooling end-to-end speaker ver- ification with adversarial examples. In 2018 IEEE Inter- national Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 1962-1966. IEEE, 2018.","title":"Fooling end-to-end speaker verification with adversarial examples","authors":"Kreuk","year":"2018","venue":"2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liptchinsky et al., 2017] Vitaliy Liptchinsky, Gabriel Syn- naeve, and Ronan Collobert. Letter-based speech recognition with gated convnets. arXiv preprint arXiv:1712.09444, 2017.","title":"Letter-based speech recognition with gated convnets","authors":"Vitaliy Liptchinsky, Gabriel Synnaeve, Ronan Collobert","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Olden and Neff, 2001] Julian D Olden and Bryan D Neff. Cross-correlation bias in lag analysis of aquatic time se- ries. Marine Biology, 138(5):1063-1070, 2001.","title":"Cross-correlation bias in lag analysis of aquatic time series","authors":"D Julian, Bryan D Olden, Neff","year":"2001","venue":"Marine Biology","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Panayotov et al., 2015] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206-5210. IEEE, 2015.","title":"Librispeech: an asr corpus based on public domain audio books","authors":"Vassil Panayotov, Guoguo Chen, Daniel Povey, Sanjeev Khudanpur","year":"2015","venue":"2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Recommendation, 2001] ITUR Recommendation. Methods for objective measurements of perceived audio quality. ITU-R BS, 13871, 2001.","title":"ITUR Recommendation. Methods for objective measurements of perceived audio quality","authors":null,"year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schönherr et al., 2019] Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa. Ad- versarial attacks against automatic speech recognition sys- tems via psychoacoustic hiding. In Network and Dis- tributed Systems Security (NDSS) Symposium, 2019.","title":"Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding","authors":"Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, Dorothea Kolossa","year":"2019","venue":"Network and Distributed Systems Security (NDSS) Symposium","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vaidya et al., 2015] Tavish Vaidya, Yuankai Zhang, Micah Sherr, and Clay Shields. Cocaine noodles: exploiting the gap between human and machine speech recognition. In 9th USENIX Workshop on Offensive Technologies (WOOT 15), 2015.","title":"Cocaine noodles: exploiting the gap between human and machine speech recognition","authors":"Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields","year":"2015","venue":"9th USENIX Workshop on Offensive Technologies (WOOT 15)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yakura and Sakuma, 2019] Hiromu Yakura and Jun Sakuma. Robust audio adversarial example for a physical attack. In International Joint Conferences on Artificial Intelligence, 2019.","title":"Hiromu Yakura and Jun Sakuma. Robust audio adversarial example for a physical attack","authors":"Sakuma Yakura","year":"2019","venue":"International Joint Conferences on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yan et al., 2020] Qiben Yan, Kehai Liu, Qin Zhou, Hanqing Guo, and Ning Zhang. Surfingattack: Interactive hidden attack on voice assistants using ultrasonic guided wave. In Network and Distributed Systems Security (NDSS) Sympo- sium, 2020.","title":"Surfingattack: Interactive hidden attack on voice assistants using ultrasonic guided wave","authors":"Yan","year":"2020","venue":"Network and Distributed Systems Security (NDSS) Symposium","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2019] Zhuolin Yang, Bo Li, Pin-Yu Chen, and Dawn Song. Characterizing audio adversarial examples using temporal dependency. In International Conference on Learning Representation, 2019.","title":"Characterizing audio adversarial examples using temporal dependency","authors":"Yang","year":"2019","venue":"International Conference on Learning Representation","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yuan et al., 2018] Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, XiaoFeng Wang, and Carl A Gunter. Com- mandersong: A systematic approach for practical adver- sarial voice recognition. In 27th USENIX Security Sympo- sium (USENIX Security 18), pages 49-64, 2018.","title":"Commandersong: A systematic approach for practical adversarial voice recognition","authors":"Yuan","year":"2018","venue":"27th USENIX Security Symposium (USENIX Security 18)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2017] Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, and Wenyuan Xu. Dol- phinattack: Inaudible voice commands. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 103-117. ACM, 2017.","title":"Dolphinattack: Inaudible voice commands","authors":"Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, Wenyuan Xu","year":"2017","venue":"Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}