{"type":"ScholarlyArticle","title":"DeepView: Visualizing Classification Boundaries of Deep Neural Networks as Scatter Plots Using Discriminative Dimensionality Reduction","authors":[{"name":"Alexander Schulz","slug":"alexander-schulz","given":"Alexander","family":"Schulz","affiliation":"Machine Learning Group , Bielefeld University , D-33619 Bielefeld , Germany"},{"name":"Fabian Hinder","slug":"fabian-hinder","given":"Fabian","family":"Hinder","affiliation":"Machine Learning Group , Bielefeld University , D-33619 Bielefeld , Germany"},{"name":"Barbara Hammer","slug":"barbara-hammer","given":"Barbara","family":"Hammer","affiliation":"Machine Learning Group , Bielefeld University , D-33619 Bielefeld , Germany"}],"abstract":"Machine learning algorithms using deep architectures have been able to implement increasingly powerful and successful models. However, they also become increasingly more complex, more difficult to comprehend and easier to fool. So far, most methods in the literature investigate the decision of the model for a single given input datum. In this paper, we propose to visualize a part of the decision function of a deep neural network together with a part of the data set in two dimensions with discriminative dimensionality reduction. This enables us to inspect how different properties of the data are treated by the model, such as outliers, adversaries or poisoned data. Further, the presented approach is complementary to the mentioned interpretation methods from the literature and hence might be even more useful in combination with those. Code is available at https://github.com/LucaHermes/DeepView","doi":"10.24963/ijcai.2020/319","url":"https://ijcai.org/proceedings/2020/319","pdf":"https://ijcai.org/proceedings/2020/0319.pdf","markdown":"https://ijcai.org/proceedings/2020/319.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/319","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":319,"pages":{"first":2305,"last":2311},"keywords":["Machine Learning: Interpretability","Machine Learning: Deep Learning","Machine Learning: Explainable Machine Learning","Machine Learning: Dimensionality Reduction and Manifold Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25181/file_25181.m3u8","length_sec":299,"provider":"hls"}],"references":[{"position":1,"raw":"[Alvarez-Melis and Jaakkola, 2017] David Alvarez-Melis and Tommi S. Jaakkola. A causal framework for explaining the predictions of black-box sequence-to-sequence models. CoRR, abs/1707.01943, 2017.","title":"A causal framework for explaining the predictions of black-box sequence-to-sequence models","authors":"Alvarez-Melis, David Jaakkola, Tommi S Alvarez-Melis, Jaakkola","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amari and Nagaoka, 2007] Shun-ichi Amari and Hiroshi Nagaoka. Methods of Information Geometry. Translations of mathematical monographs. American Mathematical So- ciety, 2007.","title":"Shun","authors":"Nagaoka ; Ichi Amari, Hiroshi Amari, Nagaoka","year":"2007","venue":"Methods of Information Geometry","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Amari, 1985] Shun-ichi Amari. Differential-geometrical methods in statistics. Lecture Notes on Statistics, 28:1, 1985.","title":"Shun-ichi Amari. Differential-geometrical methods in statistics","authors":"Amari","year":"1985","venue":"Lecture Notes on Statistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Crooks, 2007] Gavin E. Crooks. Measuring thermodynamic length. arXiv preprint arXiv:0706.0559v2, 2007.","title":"Measuring thermodynamic length","authors":"Gavin E Crooks","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Elsayed et al., 2018] Gamaleldin F. Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alex Kurakin, Ian J. Goodfellow, and Jascha Sohl-Dickstein. Adversarial examples that fool both human and computer vision. CoRR, abs/1802.08195, 2018.","title":"Adversarial examples that fool both human and computer vision","authors":"F Gamaleldin, Shreya Elsayed, Brian Shankar, Nicolas Cheung, Alex Papernot, Ian J Kurakin, Jascha Goodfellow, Sohl-Dickstein","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feinman et al., 2017] Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. Detect- ing adversarial samples from artifacts. arXiv preprint arXiv:1703.00410, 2017.","title":"Detecting adversarial samples from artifacts","authors":"Reuben Feinman, Saurabh Ryan R Curtin, Andrew B Shintre, Gardner","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gisbrecht et al., 2015] Andrej Gisbrecht, Alexander Schulz, and Barbara Hammer. Parametric nonlinear dimensionality reduction using kernel t-sne. Neurocomputing, 147:71 -82, January 2015.","title":"Parametric nonlinear dimensionality reduction using kernel t-sne","authors":"Gisbrecht","year":"2015","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goldberger et al., 2004] Jacob Goldberger, Sam Roweis, Ge- off Hinton, and Ruslan Salakhutdinov. Neighbourhood components analysis. In Advances in Neural Information Processing Systems 17, pages 513-520. MIT Press, 2004.","title":"Neighbourhood components analysis","authors":"Jacob Goldberger, Sam Roweis, Geoff Hinton, Ruslan Salakhutdinov","year":"2004","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. arXiv preprint arXiv:1412.6572, 2014.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gu et al., 2017] Tianyu Gu, Brendan Dolan-Gavitt, and Sid- dharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain, 2017.","title":"Badnets: Identifying vulnerabilities in the machine learning model supply chain","authors":"Tianyu Gu, Brendan Dolan-Gavitt, Siddharth Garg","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jagielski et al., 2018] Matthew Jagielski, Alina Oprea, Bat- tista Biggio, Chang Liu, Cristina Nita-Rotaru, and Bo Li. Manipulating machine learning: Poisoning attacks and countermeasures for regression learning. In IEEE Sym- posium on Security and Privacy, pages 19-35, May 2018.","title":"Manipulating machine learning: Poisoning attacks and countermeasures for regression learning","authors":"Matthew Jagielski, Alina Oprea, Battista Biggio, Chang Liu, Cristina Nita-Rotaru, Bo Li","year":"2018","venue":"IEEE Symposium on Security and Privacy","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kaski et al., 2001] Samuel Kaski, Janne Sinkkonen, and Jaakko Peltonen. Bankruptcy analysis with self-organizing maps in learning metrics. IEEE Transactions on Neural Networks, 12:936-947, 2001.","title":"Bankruptcy analysis with self-organizing maps in learning metrics","authors":"Kaski","year":"2001","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lage et al., 2019] Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Sam Gershman, and Finale Doshi-Velez. An evaluation of the human-interpretability of explanation. CoRR, abs/1902.00006, 2019.","title":"An evaluation of the human-interpretability of explanation","authors":"Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Sam Gershman, Finale Doshi-Velez","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lapuschkin et al., 2019] Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Woj- ciech Samek, and Klaus-Robert Müller. Unmasking clever hans predictors and assessing what machines really learn. Nature Communications, 2019.","title":"Unmasking clever hans predictors and assessing what machines really learn","authors":"Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller","year":"2019","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Leavy, 2018] Susan Leavy. Gender bias in artificial intelli- gence: The need for diversity and gender theory in machine learning. In 2018 IEEE/ACM 1st International Workshop on Gender Equality in Software Engineering (GE), pages 14-16, May 2018.","title":"Gender bias in artificial intelligence: The need for diversity and gender theory in machine learning","authors":"Susan Leavy, Leavy","year":"2018","venue":"IEEE/ACM 1st International Workshop on Gender Equality in Software Engineering (GE)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Losing et al., 2018] Viktor Losing, Barbara Hammer, and Heiko Wersing. Tackling heterogeneous concept drift with the self-adjusting memory (SAM). Knowl. Inf. Syst., 54(1):171-201, 2018.","title":"Tackling heterogeneous concept drift with the self-adjusting memory (SAM)","authors":"Losing","year":"2018","venue":"Knowl. Inf. Syst","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[McInnes et al., 2018] Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approxima- tion and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018.","title":"Umap: Uniform manifold approximation and projection for dimension reduction","authors":"Leland Mcinnes, John Healy, James Melville","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Montiel et al., 2018] Jacob Montiel, Jesse Read, Albert Bifet, and Talel Abdessalem. Scikit-multiflow: A multi- output streaming framework. Journal of Machine Learning Research, 19:72:1-72:5, 2018.","title":"Talel Abdessalem. Scikit-multiflow: A multioutput streaming framework","authors":"Montiel","year":"2018","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mueller et al., 2019] Shane T. Mueller, Robert R. Hoffman, William J. Clancey, Abigail Emrey, and Gary Klein. Ex- planation in human-ai systems: A literature meta-review, synopsis of key ideas and publications, and bibliography for explainable AI. CoRR, abs/1902.01876, 2019.","title":"Explanation in human-ai systems: A literature meta-review, synopsis of key ideas and publications, and bibliography for explainable AI","authors":"T Shane, Robert R Mueller, William J Hoffman, Abigail Clancey, Gary Emrey, Klein","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Peltonen et al., 2004] Jaakko Peltonen, Arto Klami, and Samuel Kaski. Improved learning of riemannian metrics for exploratory analysis. Neural Networks, 17:1087-1100, 2004.","title":"Improved learning of riemannian metrics for exploratory analysis","authors":"Jaakko Peltonen, Arto Klami, Samuel Kaski","year":"2004","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ribeiro et al., 2018] Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High-precision model- agnostic explanations. In AAAI Conference, (AAAI-18), (IAAI-18), and (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 1527-1535, 2018.","title":"Anchors: High-precision modelagnostic explanations","authors":"Marco Túlio Ribeiro, Sameer Singh, Carlos Guestrin","year":"2018","venue":"IAAI-18), and (EAAI-18)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ruiz et al., 2013] Héctor Ruiz, Terence A Etchells, Ian H Jarman, José D Martín, and Paulo JG Lisboa. A principled approach to network-based classification and data represen- tation. Neurocomputing, 112:79-91, 2013.","title":"A principled approach to network-based classification and data representation","authors":"Héctor Ruiz, Terence A Etchells, Ian H Jarman, José D Martín, Paulo Jg Lisboa","year":"2013","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Samek et al., 2017] Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. CoRR, abs/1708.08296, 2017.","title":"Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models","authors":"Wojciech Samek, Thomas Wiegand, Klaus-Robert Müller","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Scholkopf and Smola, 2001] Bernhard Scholkopf and Alexander J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA, 2001.","title":"Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond","authors":"Smola Scholkopf, Bernhard Scholkopf, Alexander J Smola","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schulz et al., 2015] Alexander Schulz, Andrej Gisbrecht, and Barbara Hammer. Using discriminative dimension- ality reduction to visualize classifiers. Neural Processing Letters, 42(1):27-54, August 2015.","title":"Using discriminative dimensionality reduction to visualize classifiers","authors":"Alexander Schulz, Andrej Gisbrecht, Barbara Hammer","year":"2015","venue":"Neural Processing Letters","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sokol and Flach, 2018] Kacper Sokol and Peter Flach. Glass-box: Explaining ai decisions with counterfactual statements through conversation with a voice-enabled vir- tual assistant. In IJCAI-18, pages 5868-5870, 7 2018.","title":"Glass-box: Explaining ai decisions with counterfactual statements through conversation with a voice-enabled virtual assistant","authors":"Flach Sokol, Kacper Sokol, Peter Flach","year":"2018","venue":"IJCAI-18","doi":null,"url":null,"internal_url":"/proceedings/2018/865"},{"position":28,"raw":"[van der Maaten and Hinton, 2008] Laurens J.P. van der Maaten and Geoffrey E. Hinton. Visualizing high- dimensional data using t-sne. Journal of Machine Learning Research, 9:2579-2605, 2008.","title":"Visualizing highdimensional data using t-sne","authors":"J P Laurens, Geoffrey E Van Der Maaten, Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Venna et al., 2010] Jarkko Venna, Jaakko Peltonen, Kristian Nybo, Helena Aidos, and Samuel Kaski. Information re- trieval perspective to nonlinear dimensionality reduction for data visualization. Journal of Machine Learning Research, 11:451-490, 2010.","title":"Information retrieval perspective to nonlinear dimensionality reduction for data visualization","authors":"Venna","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Voll- graf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zahavy et al., 2016] Tom Zahavy, Nir Ben-Zrihem, and Shie Mannor. Graying the black box: Understanding dqns. In Proceedings of ICML, volume 48, pages 1899-1908, New York, New York, USA, 20-22 Jun 2016. PMLR.","title":"Graying the black box: Understanding dqns","authors":"Zahavy","year":"2016","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}