{"type":"ScholarlyArticle","title":"Predictive Uncertainty Estimation for Tractable Deep Probabilistic Models","authors":[{"name":"Julissa Villanueva Llerena","slug":"julissa-villanueva-llerena","given":"Julissa","family":"Villanueva Llerena","affiliation":"Institute of Mathematics and Statistics University of São Paulo , Brazil"}],"abstract":"Tractable Deep Probabilistic Models (TPMs) are generative models based on arithmetic circuits that allow for exact marginal inference in linear time. These models have obtained promising results in several machine learning tasks. Like many other models, TPMs can produce over-confident incorrect inferences, especially on regions with small statistical support. In this work, we will develop efficient estimators of the predictive uncertainty that are robust to data scarcity and outliers. We investigate two approaches. The first approach measures the variability of the output to perturbations of the model weights. The second approach captures the variability of the prediction to changes in the model architecture. We will evaluate the approaches on challenging tasks such as image completion and multilabel classification.","doi":"10.24963/ijcai.2020/745","url":"https://ijcai.org/proceedings/2020/745","pdf":"https://ijcai.org/proceedings/2020/0745.pdf","markdown":"https://ijcai.org/proceedings/2020/745.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/745","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Doctoral Consortium","track":"Doctoral Consortium","number":745,"pages":{"first":5210,"last":5211},"keywords":["Trust, Fairness, Bias: General","Uncertainty in AI: Graphical Models","Machine Learning: Probabilistic Machine Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25479/file_25479.m3u8","length_sec":260,"provider":"hls"}],"references":[{"position":1,"raw":"[Antonucci et al., 2019] Alessandro Antonucci, Alessandro Facchini, and Lilith Mattei. Credal sentential decision di- agrams. In ISIPTA, 2019.","title":"Credal sentential decision diagrams","authors":"Alessandro Antonucci, Alessandro Facchini, Lilith Mattei","year":"2019","venue":"ISIPTA","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caruana et al., 2015] Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In ACM KDD, 2015.","title":"Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission","authors":"Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, Noemie Elhadad","year":"2015","venue":"ACM KDD","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gal and Ghahramani, 2016] Yarin Gal and Zoubin Ghahra- mani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In ICML, 2016.","title":"Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning","authors":"Ghahramani Gal","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gal, 2016] Yarin Gal. Uncertainty in deep learning. PhD thesis, University of Cambridge, 2016.","title":"Uncertainty in deep learning","authors":"Yarin Gal","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hu et al., 2020] Shi Hu, Nicola Pezzotti, Dimitrios Mavroeidis, and Max Welling. Simple and accurate uncer- tainty quantification from bias-variance decomposition. CoRR, abs/2002.05582, 2020.","title":"Simple and accurate uncertainty quantification from bias-variance decomposition","authors":"Shi Hu, Nicola Pezzotti, Dimitrios Mavroeidis, Max Welling","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jiang et al., 2018] Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust a classi- fier. In NeurIPS, 2018.","title":"To trust or not to trust a classifier","authors":"Heinrich Jiang, Been Kim, Melody Guan, Maya Gupta","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kisa et al., 2014] Doga Kisa, Guy Van den Broeck, Arthur Choi, and Adnan Darwiche. Probabilistic sentential deci- sion diagrams. In KR, 2014.","title":"Probabilistic sentential decision diagrams","authors":"Kisa","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lakshminarayanan et al., 2017] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scal- able predictive uncertainty estimation using deep ensem- bles. In NeurIPS, 2017.","title":"Simple and scalable predictive uncertainty estimation using deep ensembles","authors":"Alexander Balaji Lakshminarayanan, Charles Pritzel, Blundell","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Mattei et al., 2019] Lilith Mattei, Decio Soares, Alessandro Antonucci, Denis Mauà, and Alessandro Facchini. Explor- ing the space of probabilistic sentential decision diagrams. In Workshop of TPM, 2019.","title":"Exploring the space of probabilistic sentential decision diagrams","authors":"Mattei","year":"2019","venue":"Workshop of TPM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Mauá et al., 2018] Denis Deratani Mauá, Diarmaid Conaty, Fabio Gagliardi Cozman, Katja Poppenhaeger, and Cas- sio Polpo de Campos. Robustifying sum-product net- works. International Journal of Approximate Reasoning, 2018.","title":"Robustifying sum-product networks","authors":"Denis Deratani Mauá, Diarmaid Conaty, Fabio Gagliardi Cozman, Katja Poppenhaeger, Cassio Polpo De Campos","year":"2018","venue":"International Journal of Approximate Reasoning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Peharz et al., 2014] Robert Peharz, Robert Gens, and Pedro Domingos. Learning selective sum-product networks. In Workshop of TPM, 2014.","title":"Learning selective sum-product networks","authors":"Robert Peharz, Robert Gens, Pedro Domingos","year":"2014","venue":"Workshop of TPM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Poon and Domingos, 2011] Hoifung Poon and Pedro Domingos. Sum-product networks: A new deep architec- ture. In UAI, 2011.","title":"Hoifung Poon and Pedro Domingos. Sum-product networks: A new deep architecture","authors":"Domingos Poon","year":"2011","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Rashwan et al., 2016] Abdullah Rashwan, Han Zhao, and Pascal Poupart. Online and distributed bayesian moment matching for parameter learning in sum-product networks. In Artificial Intelligence and Statistics, 2016.","title":"Online and distributed bayesian moment matching for parameter learning in sum-product networks","authors":"Abdullah Rashwan, Han Zhao, Pascal Poupart","year":"2016","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Vergari et al., 2019] Antonio Vergari, Alejandro Molina, Robert Peharz, Zoubin Ghahramani, Kristian Kersting, and Isabel Valera. Automatic bayesian density analysis. In Proceedings of the AAAI Conference on Artificial Intel- ligence, 2019.","title":"Automatic bayesian density analysis","authors":"Vergari","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Villanueva and Mauá, 2019] Julissa Villanueva and Denis Mauá. Robust analysis of map inference in selective sum- product networks. In ISIPTA, 2019.","title":"Robust analysis of map inference in selective sumproduct networks","authors":"Mauá Villanueva, Julissa Villanueva, Denis Mauá","year":"2019","venue":"ISIPTA","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Zhao et al., 2016] Han Zhao, Tameem Adel, Geoff Gordon, and Brandon Amos. Collapsed variational inference for sum-product networks. In ICML, 2016.","title":"Collapsed variational inference for sum-product networks","authors":"Han Zhao, Tameem Adel, Geoff Gordon, Brandon Amos","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}