{"type":"ScholarlyArticle","title":"Efficient Label Contamination Attacks Against Black-Box Learning Models","authors":[{"name":"Mengchen Zhao","slug":"mengchen-zhao","given":"Mengchen","family":"Zhao","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Bo An","slug":"bo-an","given":"Bo","family":"An","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Wei Gao","slug":"wei-gao","given":"Wei","family":"Gao","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing , China"},{"name":"Teng Zhang","slug":"teng-zhang","given":"Teng","family":"Zhang","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing , China"}],"abstract":"Label contamination attack (LCA) is an important type of data poisoning attack where an attacker manipulates the labels of training data to make the learned model beneficial to him. Existing work on LCA assumes that the attacker has full knowledge of the victim learning model, whereas the victim model is usually a black-box to the attacker. In this paper, we develop a Projected Gradient Ascent (PGA) algorithm to compute LCAs on a family of empirical risk minimizations and show that an attack on one victim model can also be effective on other victim models. This makes it possible that the attacker designs an attack against a substitute model and transfers it to a black-box victim model. Based on the observation of the transferability, we develop a defense algorithm to identify the data points that are most likely to be attacked. Empirical studies show that PGA significantly outperforms existing baselines and linear learning models are better substitute models than nonlinear ones.","doi":"10.24963/ijcai.2017/551","url":"https://ijcai.org/proceedings/2017/551","pdf":"https://ijcai.org/proceedings/2017/0551.pdf","markdown":"https://ijcai.org/proceedings/2017/551.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/551","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":551,"pages":{"first":3945,"last":3951},"keywords":["Multidisciplinary Topics and Applications: AI&Security and Privacy","Multidisciplinary Topics and Applications: Multidisciplinary Topics and Applications"],"videos":[],"references":[{"position":1,"raw":"[Alfeld et al., 2016] Scott Alfeld, Xiaojin Zhu, and Paul Bar- ford. Data poisoning attacks against autoregressive mod- els. In Proceedings of the 30th AAAI Conference on Arti- ficial Intelligence, pages 1452-1458, 2016.","title":"Data poisoning attacks against autoregressive models","authors":"Scott Alfeld, Xiaojin Zhu, Paul Barford","year":"2016","venue":"Proceedings of the 30th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alfeld et al., 2017] Scott Alfeld, Xiaojin Zhu, and Paul Bar- ford. Explicit defense actions against test-set attacks. In Proceedings of the 31th AAAI Conference on Artificial In- telligence, pages 1274-1280, 2017.","title":"Explicit defense actions against test-set attacks","authors":"Scott Alfeld, Xiaojin Zhu, Paul Barford","year":"2017","venue":"Proceedings of the 31th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[An et al., 2015] Bo An, Milind Tambe, and Arunesh Sinha. Stackelberg security games (ssg): Basics and application overview. Improving Homeland Security Decisions. Cam- bridge University Press, forthcoming, 2015.","title":"Stackelberg security games (ssg): Basics and application overview. Improving Homeland Security Decisions","authors":"Bo An, Milind Tambe, Arunesh Sinha","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Barreno et al., 2006] Marco Barreno, Blaine Nelson, Rus- sell Sears, Anthony D Joseph, and J Doug Tygar. Can machine learning be secure? In Proceedings of the 2006 ACM Symposium on Information, Computer and Commu- nications Security, pages 16-25, 2006.","title":"Can machine learning be secure?","authors":"Marco Barreno, Blaine Nelson, Russell Sears, Anthony D Joseph, Doug Tygar","year":"2006","venue":"Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Barreno et al., 2010] Marco Barreno, Blaine Nelson, An- thony D Joseph, and JD Tygar. The security of machine learning. Machine Learning, 81(2):121-148, 2010.","title":"The security of machine learning","authors":"Marco Barreno, Blaine Nelson, Anthony D Joseph, Tygar","year":"2010","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Biggio et al., 2011] Battista Biggio, Blaine Nelson, and Pavel Laskov. Support vector machines under adversarial label noise. The 3rd Asian Conference on Machine Learn- ing, 20:97-112, 2011.","title":"Support vector machines under adversarial label noise","authors":"Battista Biggio, Blaine Nelson, Pavel Laskov","year":"2011","venue":"The 3rd Asian Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Biggio et al., 2012] Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines. In Proceedings of the 29th International Con- ference on Machine Learning, pages 1807-1814, 2012.","title":"Poisoning attacks against support vector machines","authors":"Battista Biggio, Blaine Nelson, Pavel Laskov","year":"2012","venue":"Proceedings of the 29th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chan et al., 2017] Patrick PK Chan, Zhi-Min He, Hongjiang Li, and Chien-Chang Hsu. Data sanitiza- tion against adversarial label contamination based on data complexity. International Journal of Machine Learning and Cybernetics, pages 1-14, 2017.","title":"Data sanitization against adversarial label contamination based on data complexity","authors":"Zhi-Min Patrick Pk Chan, Hongjiang He, Chien-Chang Li, Hsu","year":"2017","venue":"International Journal of Machine Learning and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chang and Lin, 2011] Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011.","title":"LIBSVM: A library for support vector machines","authors":"Chang, Lin ; ] Chih-Chung Chang, Chih-Jen Lin","year":"2011","venue":"ACM Transactions on Intelligent Systems and Technology","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fan et al., 2008] Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. Journal of Machine Learning Research, 9(Aug):1871-1874, 2008.","title":"Liblinear: A library for large linear classification","authors":"Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, Chih-Jen Lin","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fefilatyev et al., 2012] Sergiy Fefilatyev, Matthew Shreve, Kurt Kramer, Lawrence Hall, Dmitry Goldgof, Rangachar Kasturi, Kendra Daly, Andrew Remsen, and Horst Bunke. Label-noise reduction with support vector machines. In Proceedings of the 21st International Conference on Pat- tern Recognition, pages 3504-3508, 2012.","title":"Label-noise reduction with support vector machines","authors":"Sergiy Fefilatyev, Matthew Shreve, Kurt Kramer, Lawrence Hall, Dmitry Goldgof, Rangachar Kasturi, Kendra Daly, Andrew Remsen, Horst Bunke","year":"2012","venue":"Proceedings of the 21st International Conference on Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2011] Ling Huang, Anthony D Joseph, Blaine Nelson, Benjamin IP Rubinstein, and JD Tygar. Adversar- ial machine learning. In Proceedings of the 4th ACM Work- shop on Security and Artificial Intelligence, pages 43-58, 2011.","title":"Adversarial machine learning","authors":"Ling Huang, Anthony D Joseph, Blaine Nelson, Benjamin Ip Rubinstein, Tygar","year":"2011","venue":"Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kimeldorf and Wahba, 1970] George S Kimeldorf and Grace Wahba. A correspondence between bayesian esti- mation on stochastic processes and smoothing by splines. The Annals of Mathematical Statistics, 41(2):495-502, 1970.","title":"A correspondence between bayesian estimation on stochastic processes and smoothing by splines","authors":"S George, Grace Kimeldorf, Wahba","year":"1970","venue":"The Annals of Mathematical Statistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kloft and Laskov, 2010] Marius Kloft and Pavel Laskov. Online anomaly detection under adversarial impact. In Proceedings of the 13th International Conference on Ar- tificial Intelligence and Statistics, pages 405-412, 2010.","title":"Online anomaly detection under adversarial impact","authors":"Laskov Kloft, Marius Kloft, Pavel Laskov","year":"2010","venue":"Proceedings of the 13th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li and Vorobeychik, 2014] Bo Li and Yevgeniy Vorobey- chik. Feature cross-substitution in adversarial classifica- tion. In Prodeedings of the 28th Advances in Neural Infor- mation Processing Systems, pages 2087-2095, 2014.","title":"Feature cross-substitution in adversarial classification","authors":"Bo Li, Yevgeniy Vorobeychik","year":"2014","venue":"Prodeedings of the 28th Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2016] Bo Li, Yining Wang, Aarti Singh, and Yevgeniy Vorobeychik. Data poisoning attacks on factorization-based collaborative filtering. In Prodeedings of the 30th Annual Conference on Neural Information Pro- cessing Systems, pages 1885-1893, 2016.","title":"Data poisoning attacks on factorization-based collaborative filtering","authors":"Bo Li, Yining Wang, Aarti Singh, Yevgeniy Vorobeychik","year":"2016","venue":"Prodeedings of the 30th Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lichman, 2013] M. Lichman. UCI machine learning repos- itory, 2013.","title":"UCI machine learning repository","authors":"M Lichman","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lowd and Meek, 2005] Daniel Lowd and Christopher Meek. Adversarial learning. In Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 641-647, 2005.","title":"Adversarial learning","authors":"Meek Lowd, Daniel Lowd, Christopher Meek","year":"2005","venue":"Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mei and Zhu, 2015a] Shike Mei and Xiaojin Zhu. The se- curity of latent dirichlet allocation. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, pages 681-689, 2015.","title":"The security of latent dirichlet allocation","authors":"Zhu Mei, Shike Mei, Xiaojin Zhu","year":"2015","venue":"Proceedings of the 18th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mei and Zhu, 2015b] Shike Mei and Xiaojin Zhu. Using machine teaching to identify optimal training-set attacks on machine learners. In Proceedings of the 29st AAAI con- ference on Artificial intelligence, pages 2871-2877, 2015.","title":"Using machine teaching to identify optimal training-set attacks on machine learners","authors":"Zhu Mei, Shike Mei, Xiaojin Zhu","year":"2015","venue":"Proceedings of the 29st AAAI conference on Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Papernot et al., 2016a] Nicolas Papernot, Patrick Mc- Daniel, and Ian Goodfellow. Transferability in machine learning: From phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277, 2016.","title":"Transferability in machine learning: From phenomena to black-box attacks using adversarial samples","authors":"Papernot","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Papernot et al., 2016b] Nicolas Papernot, Patrick Mc- Daniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In Proceedings of the 1st IEEE European Symposium on Security and Privacy, pages 372-387, 2016.","title":"The limitations of deep learning in adversarial settings","authors":"Papernot","year":"2016","venue":"Proceedings of the 1st IEEE European Symposium on Security and Privacy","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rahimi and Recht, 2009] Ali Rahimi and Benjamin Recht. Weighted sums of random kitchen sinks: Replacing min- imization with randomization in learning. In Proceedings of the 23rd Annual Conference on Neural Information Pro- cessing Systems, pages 1313-1320, 2009.","title":"Weighted sums of random kitchen sinks: Replacing minimization with randomization in learning","authors":"Recht Rahimi, Ali Rahimi, Benjamin Recht","year":"2009","venue":"Proceedings of the 23rd Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rahimi et al., 2007] Ali Rahimi, Benjamin Recht, et al. Random features for large-scale kernel machines. In Pro- ceedings of the 21st Annual Conference on Neural Infor- mation Processing Systems, number 4, page 5, 2007.","title":"Random features for large-scale kernel machines","authors":"Ali Rahimi, Benjamin Recht","year":"2007","venue":"Proceedings of the 21st Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xiao et al., 2012] Han Xiao, Huang Xiao, and Claudia Eck- ert. Adversarial label flips attack on support vector ma- chines. In Proceedings of the 20th European Conference on Artificial Intelligence, pages 870-875, 2012.","title":"Adversarial label flips attack on support vector machines","authors":"Han Xiao, Xiao Huang, Claudia Eckert","year":"2012","venue":"Proceedings of the 20th European Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xiao et al., 2015] Huang Xiao, Battista Biggio, Gavin Brown, Giorgio Fumera, Claudia Eckert, and Fabio Roli. Is feature selection secure against training data poisoning? In Proceedings of the 32th International Conference on Machine Learning, pages 1689-1698, 2015.","title":"Is feature selection secure against training data poisoning?","authors":"Battista Huang Xiao, Gavin Biggio, Giorgio Brown, Claudia Fumera, Fabio Eckert, Roli","year":"2015","venue":"Proceedings of the 32th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2016] Mengchen Zhao, Bo An, and Christo- pher Kiekintveld. Optimizing personalized email filtering thresholds to mitigate sequential spear phishing attacks. In Proceedings of the 30th AAAI Conference on Artificial In- telligence, pages 658-665, 2016.","title":"Optimizing personalized email filtering thresholds to mitigate sequential spear phishing attacks","authors":"Zhao","year":"2016","venue":"Proceedings of the 30th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}