{"type":"ScholarlyArticle","title":"Glass-Box: Explaining AI Decisions With Counterfactual Statements Through Conversation With a Voice-enabled Virtual Assistant","authors":[{"name":"Kacper Sokol","slug":"kacper-sokol","given":"Kacper","family":"Sokol","affiliation":"Department of Computer Science , University of Bristol"},{"name":"Peter Flach","slug":"peter-flach","given":"Peter","family":"Flach","affiliation":"Department of Computer Science , University of Bristol"}],"abstract":"The prevalence of automated decision making, influencing important aspects of our lives -- e.g., school admission, job market, insurance and banking -- has resulted in increasing pressure from society and regulators to make this process more transparent and ensure its explainability, accountability and fairness. We demonstrate a prototype voice-enabled device, called Glass-Box, which users can question to understand automated decisions and identify the underlying model's biases and errors. Our system explains algorithmic predictions with class-contrastive counterfactual statements (e.g., ``Had a number of conditions been different:...the prediction would change...''), which show a difference in a particular scenario that causes an algorithm to ``change its mind''. Such explanations do not require any prior technical knowledge to understand, hence are suitable for a lay audience, who interact with the system in a natural way -- through an interactive dialogue. We demonstrate the capabilities of the device by allowing users to impersonate a loan applicant who can question the system to understand the automated decision that he received.","doi":"10.24963/ijcai.2018/865","url":"https://ijcai.org/proceedings/2018/865","pdf":"https://ijcai.org/proceedings/2018/0865.pdf","markdown":"https://ijcai.org/proceedings/2018/865.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/865","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Demos","track":"Demos","number":865,"pages":{"first":5868,"last":5870},"keywords":["Machine Learning: Machine Learning","Humans and AI: Human-Computer Interaction","Machine Learning: Classification","Natural Language Processing: Dialogue"],"videos":[],"references":[{"position":1,"raw":"[Kim et al., 2014] Been Kim, Cynthia Rudin, and Julie A Shah. The bayesian case model: A generative approach for case-based reasoning and prototype classification. In Ad- vances in Neural Information Processing Systems, pages 1952-1960, 2014.","title":"The bayesian case model: A generative approach for case-based reasoning and prototype classification","authors":"Kim","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Kusner et al., 2017] Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Ad- vances in Neural Information Processing Systems, pages 4069-4079, 2017.","title":"Counterfactual fairness","authors":"Matt J Kusner, Joshua Loftus, Chris Russell, Ricardo Silva","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Miller et al., 2017] Tim Miller, Piers Howe, and Liz Sonen- berg. Explainable ai: Beware of inmates running the asy- lum. In IJCAI-17 Workshop on Explainable AI (XAI), page 36, 2017.","title":"Explainable ai: Beware of inmates running the asylum","authors":"Tim Miller, Piers Howe, Liz Sonenberg","year":"2017","venue":"IJCAI-17 Workshop on Explainable AI (XAI)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Miller, 2017] Tim Miller. Explanation in artificial intelli- gence: Insights from the social sciences. arXiv preprint arXiv:1706.07269, 2017.","title":"Explanation in artificial intelligence: Insights from the social sciences","authors":"Tim Miller, Miller","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ribeiro et al., 2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135-1144. ACM, 2016.","title":"Why should i trust you?: Explaining the predictions of any classifier","authors":"Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Smilkov et al., 2016] Daniel Smilkov, Nikhil Thorat, Charles Nicholson, Emily Reif, Fernanda B Viégas, and Martin Wattenberg. Embedding projector: Interactive visualization and interpretation of embeddings. arXiv preprint arXiv:1611.05469, 2016.","title":"Embedding projector: Interactive visualization and interpretation of embeddings","authors":"Daniel Smilkov, Nikhil Thorat, Charles Nicholson, Emily Reif, Fernanda B Viégas, Martin Wattenberg","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Tolomei et al., 2017] Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. Interpretable predic- tions of tree-based ensembles via actionable feature tweak- ing. In Proceedings of the 23rd ACM SIGKDD Interna- tional Conference on Knowledge Discovery and Data Min- ing, pages 465-474. ACM, 2017.","title":"Interpretable predictions of tree-based ensembles via actionable feature tweaking","authors":"Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, Mounia Lalmas","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Wachter et al., 2017] Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology, (Forth- coming), 2017.","title":"Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR","authors":"Sandra Wachter, Brent Mittelstadt, Chris Russell","year":"2017","venue":"Harvard Journal of Law & Technology","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}