{"type":"ScholarlyArticle","title":"Model Stealing Defense against Exploiting Information Leak through the Interpretation of Deep Neural Nets","authors":[{"name":"Jeonghyun Lee","slug":"jeonghyun-lee","given":"Jeonghyun","family":"Lee","affiliation":"School of Cybersecurity , Korea University {nomar0107 ,"},{"name":"Sungmin Han","slug":"sungmin-han","given":"Sungmin","family":"Han","affiliation":"School of Cybersecurity , Korea University {nomar0107 ,"},{"name":"Sangkyun Lee","slug":"sangkyun-lee","given":"Sangkyun","family":"Lee","affiliation":"School of Cybersecurity , Korea University {nomar0107 ,"}],"abstract":"Model stealing techniques allow adversaries to create attack models that mimic the functionality of black-box machine learning models, querying only class membership or probability outcomes. Recently, interpretable AI is getting increasing attention, to enhance our understanding of AI models, provide additional information for diagnoses, or satisfy legal requirements. However, it has been recently reported that providing such additional information can make AI models more vulnerable to model stealing attacks. In this paper, we propose DeepDefense, the first defense mechanism that protects an AI model against model stealing attackers exploiting both class probabilities and interpretations. DeepDefense uses a misdirection model to hide the critical information of the original model against model stealing attacks, with minimal degradation on both the class probability and the interpretability of prediction output. DeepDefense is highly applicable for any model stealing scenario since it makes minimal assumptions about the model stealing adversary. In our experiments, DeepDefense shows significantly higher defense performance than the existing state-of-the-art defenses on various datasets and interpreters.","doi":"10.24963/ijcai.2022/100","url":"https://ijcai.org/proceedings/2022/100","pdf":"https://ijcai.org/proceedings/2022/0100.pdf","markdown":"https://ijcai.org/proceedings/2022/100.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/100","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":100,"pages":{"first":710,"last":716},"keywords":["AI Ethics, Trust, Fairness: Safety & Robustness","Machine Learning: Explainable/Interpretable Machine Learning","AI Ethics, Trust, Fairness: Trustworthy AI","Computer Vision: Interpretability and Transparency"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986815","length_sec":90,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984958","length_sec":301,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bach et al., 2015] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wo- jciech Samek. On pixel-wise explanations for non-linear classi- fier decisions by layer-wise relevance propagation. PLOS ONE, 10(7):1-46, 07 2015.","title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","authors":"Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, Wojciech Samek","year":"2015","venue":"PLOS ONE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chattopadhay et al., 2018] Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth Balasubramanian. Grad-CAM++: Generalized gradient-based visual explanations for deep convolu- tional networks. In WACV, 2018.","title":"Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks","authors":"Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, Vineeth Balasubramanian","year":"2018","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chouldechova and Roth, 2020] Alexandra Chouldechova and Aaron Roth. A snapshot of the frontiers of fairness in machine learning. Commun. ACM, 63(5):82-89, April 2020.","title":"Alexandra Chouldechova and Aaron Roth. A snapshot of the frontiers of fairness in machine learning","authors":"Roth Chouldechova","year":"2020","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[GDPR, 2016] GDPR. EU General Data Protection Regulation: Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural per- sons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/EC (general data protection regulation), OJ 2016 L 119/1. European Commission, 2016.","title":"GDPR. EU General Data Protection Regulation: Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/EC (general data protection regulation)","authors":"Gdpr","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gu et al., 2018] Jindong Gu, Yinchong Yang, and Volker Tresp. Understanding individual decisions of cnns via contrastive back- propagation. In ACCV, 2018.","title":"Understanding individual decisions of cnns via contrastive backpropagation","authors":"Jindong Gu, Yinchong Yang, Volker Tresp","year":"2018","venue":"ACCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gunning, 2016] David Gunning. Explainable artificial intelli- gence (XAI), DARPA-BAA-16-53. Defense Advanced Research Projects Agency, August 2016.","title":"Explainable artificial intelligence (XAI)","authors":"David Gunning","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gur et al., 2021] Shir Gur, Ameen Ali, and Lior Wolf. Visualiza- tion of supervised and self-supervised neural networks via attri- bution guided factorization. In AAAI, 2021.","title":"Visualization of supervised and self-supervised neural networks via attribution guided factorization","authors":"Shir Gur, Ameen Ali, Lior Wolf","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huber, 1964] Peter Huber. Robust Estimation of a Location Pa- rameter. The Annals of Mathematical Statistics, 35(1):73 -101, 1964.","title":"Robust Estimation of a Location Parameter","authors":"Peter Huber","year":"1964","venue":"The Annals of Mathematical Statistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kapishnikov et al., 2019] Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viégas, and Michael Terry. XRAI: Better attributions through regions. In ICCV, 2019.","title":"XRAI: Better attributions through regions","authors":"Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viégas, Michael Terry","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kariyappa and Qureshi, 2020] Sanjay Kariyappa and Moinuddin Qureshi. Defending against model stealing attacks with adaptive misinformation. In CVPR, 2020.","title":"Defending against model stealing attacks with adaptive misinformation","authors":"Qureshi Kariyappa, Sanjay Kariyappa, Moinuddin Qureshi","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kariyappa et al., 2021] Sanjay Kariyappa, Atul Prakash, and Moinuddin Qureshi. Protecting dnns from theft using an ensem- ble of diverse models. In ICLR, 2021.","title":"Protecting dnns from theft using an ensemble of diverse models","authors":"Sanjay Kariyappa, Atul Prakash, Moinuddin Qureshi","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lee et al., 2021] Jeong Ryong Lee, Sewon Kim, Inyong Park, Tae- joon Eo, and Dosik Hwang. Relevance-CAM: Your model al- ready knows where to look. In CVPR, 2021.","title":"Relevance-CAM: Your model already knows where to look","authors":"Jeong Ryong Lee, Sewon Kim, Inyong Park, Taejoon Eo, Dosik Hwang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lundberg and Lee, 2017] Scott Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Neurips, 2017.","title":"A unified approach to interpreting model predictions","authors":"Lee ; Lundberg, Scott Lundberg, Su-In Lee","year":"2017","venue":"Neurips","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Milli et al., 2019] Smitha Milli, Ludwig Schmidt, Anca Dragan, and Moritz Hardt. Model reconstruction from model explana- tions. In Proceedings of the Conference on Fairness, Account- ability, and Transparency, 2019.","title":"Model reconstruction from model explanations","authors":"Smitha Milli, Ludwig Schmidt, Anca Dragan, Moritz Hardt","year":"2019","venue":"Proceedings of the Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Montavon et al., 2018] Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. Methods for interpreting and understand- ing deep neural networks. Digital Signal Processing, 73:1-15, 2018.","title":"Methods for interpreting and understanding deep neural networks","authors":"Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller","year":"2018","venue":"Digital Signal Processing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nam et al., 2020] Woo-Jeoung Nam, Jaesik Choi, and Seong- Whan Lee. Relative attributing propagation: Interpreting the comparative contributions of individual units in deep neural net- works. In AAAI, 2020.","title":"Relative attributing propagation: Interpreting the comparative contributions of individual units in deep neural networks","authors":"Nam","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Orekondy et al., 2019] Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of black- box models. In CVPR, 2019.","title":"Knockoff nets: Stealing functionality of blackbox models","authors":"Orekondy","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Orekondy et al., 2020] Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Prediction poisoning: Towards defenses against dnn model stealing attacks. In ICLR, 2020.","title":"Prediction poisoning: Towards defenses against dnn model stealing attacks","authors":"Orekondy","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pal et al., 2020] Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, and Vinod Ganapathy. Activethief: Model extraction using active learning and unannotated public data. AAAI, 2020.","title":"Activethief: Model extraction using active learning and unannotated public data","authors":"Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, Vinod Ganapathy","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Papernot et al., 2017] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In ACM on ASIACCS, 2017.","title":"Practical black-box attacks against machine learning","authors":"Nicolas Papernot, Patrick Mcdaniel, Ian Goodfellow, Somesh Jha, Berkay Celik, Ananthram Swami","year":"2017","venue":"ACM on ASIACCS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Petsiuk et al., 2018] Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. In BMVC, 2018.","title":"Rise: Randomized input sampling for explanation of black-box models","authors":"Abir Vitali Petsiuk, Kate Das, Saenko","year":"2018","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sattarzadeh et al., 2021] Sam Sattarzadeh, Mahesh Sudhakar, Anthony Lem, Shervin Mehryar, Konstantinos Plataniotis, Jongseong Jang, Hyunwoo Kim, Yeonjeong Jeong, Sangmin Lee, and Kyunghoon Bae. Explaining convolutional neural networks through attribution-based input sampling and block-wise feature aggregation. In AAAI, 2021.","title":"Explaining convolutional neural networks through attribution-based input sampling and block-wise feature aggregation","authors":"Sam Sattarzadeh, Mahesh Sudhakar, Anthony Lem, Shervin Mehryar, Konstantinos Plataniotis, Jongseong Jang, Hyunwoo Kim, Yeonjeong Jeong, Sangmin Lee, Kyunghoon Bae","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Selvaraju et al., 2017] Ramprasaath Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In ICCV, 2017.","title":"Grad-CAM: Visual explanations from deep networks via gradient-based localization","authors":"Selvaraju","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shrikumar et al., 2017] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagat- ing activation differences. In ICML, 2017.","title":"Learning important features through propagating activation differences","authors":"Shrikumar","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Stoica et al., 2017] Ion Stoica, Dawn Song, Raluca Ada Popa, David Patterson, Michael Mahoney, Randy Katz, Anthony Joseph, Michael Jordan, Joseph Hellerstein, Joseph Gonzalez, Ken Goldberg, Ali Ghodsi, David Culler, and Pieter Abbeel. A berkeley view of systems challenges for AI. Technical Report UCB/EECS-2017-159, University of California, Berkeley, 2017.","title":"A berkeley view of systems challenges for AI","authors":"Ion Stoica, Dawn Song, Ada Raluca, David Popa, Michael Patterson, Randy Mahoney, Anthony Katz, Michael Joseph, Joseph Jordan, Joseph Hellerstein, Ken Gonzalez, Ali Goldberg, David Ghodsi, Pieter Culler, Abbeel","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tramèr et al., 2016] Florian Tramèr, Fan Zhang, Ari Juels, Michael Reiter, and Thomas Ristenpart. Stealing machine learn- ing models via prediction APIs. In USENIX Security Symposium, 2016.","title":"Stealing machine learning models via prediction APIs","authors":"Florian Tramèr, Fan Zhang, Ari Juels, Michael Reiter, Thomas Ristenpart","year":"2016","venue":"USENIX Security Symposium","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2021] Qinglong Zhang, Lu Rao, and Yubin Yang. A novel visual interpretability for deep neural networks by optimiz- ing activation maps with perturbation. In AAAI, 2021.","title":"A novel visual interpretability for deep neural networks by optimizing activation maps with perturbation","authors":"Qinglong Zhang, Lu Rao, Yubin Yang","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2016] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In CVPR, 2016.","title":"Learning deep features for discriminative localization","authors":"Zhou","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}