{"type":"ScholarlyArticle","title":"COMET Flows: Towards Generative Modeling of Multivariate Extremes and Tail Dependence","authors":[{"name":"Andrew McDonald","slug":"andrew-mcdonald","given":"Andrew","family":"McDonald","affiliation":"Department of Computer Science and Engineering , Michigan State University"},{"name":"Pang-Ning Tan","slug":"pang-ning-tan","given":"Pang-Ning","family":"Tan","affiliation":"Department of Computer Science and Engineering , Michigan State University"},{"name":"Lifeng Luo","slug":"lifeng-luo","given":"Lifeng","family":"Luo","affiliation":"Department of Geography, Environment, and Spatial Sciences , Michigan State University"}],"abstract":"Normalizing flows—a popular class of deep generative models—often fail to represent extreme phenomena observed in real-world processes. In particular, existing normalizing flow architectures struggle to model multivariate extremes, characterized by heavy-tailed marginal distributions and asymmetric tail dependence among variables. In light of this shortcoming, we propose COMET (COpula Multivariate ExTreme) Flows, which decompose the process of modeling a joint distribution into two parts: (i) modeling its marginal distributions, and (ii) modeling its copula distribution. COMET Flows capture heavy-tailed marginal distributions by combining a parametric tail belief at extreme quantiles of the marginals with an empirical kernel density function at mid-quantiles. In addition, COMET Flows capture asymmetric tail dependence among multivariate extremes by viewing such dependence as inducing a low-dimensional manifold structure in feature space. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of COMET flows in capturing both heavy-tailed marginals and asymmetric tail dependence compared to other state-of-the-art baseline architectures. All code is available at https://github.com/andrewmcdonald27/COMETFlows.","doi":"10.24963/ijcai.2022/462","url":"https://ijcai.org/proceedings/2022/462","pdf":"https://ijcai.org/proceedings/2022/0462.pdf","markdown":"https://ijcai.org/proceedings/2022/462.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/462","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":462,"pages":{"first":3328,"last":3334},"keywords":["Machine Learning: Applications","Multidisciplinary Topics and Applications: Physical Science","Data Mining: Mining Spatial and/or Temporal Data","Machine Learning: Probabilistic Machine Learning","Machine Learning: Unsupervised Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987124","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985267","length_sec":631,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Ayala et al., 2020] Alexis Ayala, Christopher Drazic, Brian Hutchinson, Ben Kravitz, and Claudia Tebaldi. Loosely Conditioned Emulation of Global Climate Models With Generative Adversarial Networks. In NeurIPS Work- shop on Tackling Climate Change with Machine Learning, 2020.","title":"Loosely Conditioned Emulation of Global Climate Models With Generative Adversarial Networks","authors":"Alexis Ayala, Christopher Drazic, Brian Hutchinson, Ben Kravitz, Claudia Tebaldi","year":"2020","venue":"NeurIPS Workshop on Tackling Climate Change with Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beirlant et al., 2005] Jan Beirlant, Yuri Goegebeur, Jozef Teugels, Johan Segers, Daniel De Waal, and Chris Ferro. Statistics of Extremes: Theory and Applications. Springer, 2005.","title":"Statistics of Extremes: Theory and Applications","authors":"Beirlant","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bossemeyer, 2020] Leonie Bossemeyer. CM Flows -Cop- ula Density Estimation with Normalizing Flows. Master's thesis, University of Edinburgh, 2020.","title":"CM Flows -Copula Density Estimation with Normalizing Flows","authors":"Leonie Bossemeyer","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brehmer and Cranmer, 2020] Johann Brehmer and Kyle Cranmer. Flows for simultaneous manifold learning and density estimation. In NeurIPS, 2020.","title":"Flows for simultaneous manifold learning and density estimation","authors":"Cranmer Brehmer, Johann Brehmer, Kyle Cranmer","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Coles, 2001] Stuart Coles. An Introduction to Statistical Modeling of Extreme Values. Springer, 2001.","title":"An Introduction to Statistical Modeling of Extreme Values","authors":"Stuart Coles","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cunningham et al., 2020] Edmond Cunningham, Renos Zabounidis, Abhinav Agrawal, Ina Fiterau, and Daniel Sheldon. Normalizing Flows Across Dimensions. In ICML Workshop on INNF, 2020.","title":"Normalizing Flows Across Dimensions","authors":"Edmond Cunningham, Renos Zabounidis, Abhinav Agrawal, Ina Fiterau, Daniel Sheldon","year":"2020","venue":"ICML Workshop on INNF","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Czado, 2019] Claudia Czado. Analyzing Dependent Data with Vine Copulas. Springer, 2019.","title":"Analyzing Dependent Data with Vine Copulas","authors":"Claudia Czado","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dinh et al., 2017] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using Real NVP. In ICLR, 2017.","title":"Density estimation using Real NVP","authors":"Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Durkan et al., 2019] Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. Neural spline flows. In NeurIPS, 2019.","title":"Neural spline flows","authors":"Conor Durkan, Artur Bekasov, Iain Murray, George Papamakarios","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial networks. In NeurIPS, 2014.","title":"Generative adversarial networks","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Grathwohl et al., 2019] Will Grathwohl, Ricky T.Q. Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. FFJORD: Free-form continuous dynamics for scalable re- versible generative models. In ICLR, 2019.","title":"FFJORD: Free-form continuous dynamics for scalable reversible generative models","authors":"Grathwohl","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Horvat and Pfister, 2021] Christian Horvat and Jean-Pascal Pfister. Density estimation on low-dimensional manifolds: an inflation-deflation approach. arXiv:2105.12152, 2021.","title":"Density estimation on low-dimensional manifolds: an inflation-deflation approach","authors":"Pfister Horvat","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2018] Chin-Wei Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. Neural autoregressive flows. In ICML, 2018.","title":"Neural autoregressive flows","authors":"Chin-Wei Huang, David Krueger, Alexandre Lacoste, Aaron Courville","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jaini et al., 2020] Priyank Jaini, Ivan Kobyzev, Yaoliang Yu, and Marcus A. Brubaker. Tails of lipschitz triangu- lar flows. In ICML, 2020.","title":"Tails of lipschitz triangular flows","authors":"Priyank Jaini, Ivan Kobyzev, Yaoliang Yu, Marcus A Brubaker","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Joe, 2014] Harry Joe. Dependence Modeling with Copulas. CRC Press, 2014.","title":"Dependence Modeling with Copulas","authors":"Harry Joe","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kamthe et al., 2021] Sanket Kamthe, Samuel Assefa, and Marc Deisenroth. Copula Flows for Synthetic Data Gen- eration. arXiv:2101.00598, 2021.","title":"Copula Flows for Synthetic Data Generation","authors":"Sanket Kamthe, Samuel Assefa, Marc Deisenroth","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kim et al., 2020] Hyeongju Kim, Hyeonseung Lee, Woo Hyun Kang, Joun Yeop Lee, and Nam Soo Kim. SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds. In NeurIPS, 2020.","title":"SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds","authors":"Kim","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Klemmer et al., 2021] Konstantin Klemmer, Sudipan Saha, Matthias Kahl, Tianlin Xu, and Xiao Xiang Zhu. Gen- erative modeling of spatio-temporal weather patterns with extreme event conditioning. In ICLR Workshop on Model- ing Oceans and Climate Change, 2021.","title":"Generative modeling of spatio-temporal weather patterns with extreme event conditioning","authors":"Konstantin Klemmer, Sudipan Saha, Matthias Kahl, Tianlin Xu, Xiao Xiang Zhu","year":"2021","venue":"ICLR Workshop on Modeling Oceans and Climate Change","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kobyzev et al., 2020] Ivan Kobyzev, Simon Prince, and Marcus Brubaker. Normalizing Flows: An Introduction and Review of Current Methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.","title":"Normalizing Flows: An Introduction and Review of Current Methods","authors":"Ivan Kobyzev, Simon Prince, Marcus Brubaker","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Laszkiewicz et al., 2021] Mike Laszkiewicz, Johannes Led- erer, and Asja Fischer. Copula-Based Normalizing Flows. In ICML Workshop on INNF, 2021.","title":"Copula-Based Normalizing Flows","authors":"Mike Laszkiewicz, Johannes Lederer, Asja Fischer","year":"2021","venue":"ICML Workshop on INNF","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ling et al., 2020] Chun Kai Ling, Fei Fang, and J. Zico Kolter. Deep Archimedean Copulas. In NeurIPS, 2020.","title":"Deep Archimedean Copulas","authors":"Chun Kai, Ling, Fei Fang, J Zico Kolter","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nelsen, 2006] Roger B. Nelsen. An Introduction to Copu- las. Springer, 2006.","title":"An Introduction to Copulas","authors":"Roger B Nelsen","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Onken and Ruthotto, 2020] Derek Onken and Lars Ruthotto. Discretize-optimize vs. optimize-discretize for time-series regression and continuous normalizing flows. arXiv:2005.13420, 2020.","title":"Discretize-optimize vs. optimize-discretize for time-series regression and continuous normalizing flows","authors":"Ruthotto Onken, Derek Onken, Lars Ruthotto","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Papamakarios et al., 2017] George Papamakarios, Theo Pavlakou, and Iain Murray. Masked autoregressive flow for density estimation. In NeurIPS, 2017.","title":"Masked autoregressive flow for density estimation","authors":"Papamakarios","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Papamakarios et al., 2021] George Papamakarios, Eric Nal- isnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing Flows for Prob- abilistic Modeling and Inference. Journal of Machine Learning Research, 2021.","title":"Normalizing Flows for Probabilistic Modeling and Inference","authors":"George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, Balaji Lakshminarayanan","year":"2021","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Puchko et al., 2019] Alexandra Puchko, Robert Link, Brian Hutchinson, Ben Kravitz, and Abigail Snyder. Deep- ClimGAN: A High-Resolution Climate Data Generator. In NeurIPS Workshop on Tackling Climate Change with Ma- chine Learning, 2019.","title":"Deep-ClimGAN: A High-Resolution Climate Data Generator","authors":"Alexandra Puchko, Robert Link, Brian Hutchinson, Ben Kravitz, Abigail Snyder","year":"2019","venue":"NeurIPS Workshop on Tackling Climate Change with Machine Learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Resnick, 2007] Sidney I. Resnick. Heavy Tail Phenomena: Probabilistic and Statistical Modeling. Springer, 2007.","title":"Heavy Tail Phenomena: Probabilistic and Statistical Modeling","authors":"Sidney I Resnick","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sillmann et al., 2013] Jana Sillmann, Viatcheslav V. Kharin, Francis W. Zwiers, Xuebin Zhang, and David Bronaugh. Climate extremes indices in the CMIP5 multimodel ensemble: Part 2. Future climate projections. Journal of Geophysical Research: Atmospheres, 2013.","title":"Climate extremes indices in the CMIP5 multimodel ensemble: Part 2. Future climate projections","authors":"Jana Sillmann, V Viatcheslav, Francis W Kharin, Xuebin Zwiers, David Zhang, Bronaugh","year":"2013","venue":"Journal of Geophysical Research: Atmospheres","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tagasovska et al., 2019] Natasa Tagasovska, Damien Ack- erer, and Thibault Vatter. Copulas as high-dimensional generative models: Vine copula autoencoders. In NeurIPS, 2019.","title":"Copulas as high-dimensional generative models: Vine copula autoencoders","authors":"Natasa Tagasovska, Damien Ackerer, Thibault Vatter","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wiese et al., 2019] Magnus Wiese, Robert Knobloch, and Ralf Korn. Copula & Marginal Flows: Disentangling the Marginal from its Joint. arXiv:1907.03361, 2019.","title":"Copula & Marginal Flows: Disentangling the Marginal from its Joint","authors":"Magnus Wiese, Robert Knobloch, Ralf Korn","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zscheischler et al., 2018] Jakob Zscheischler, Seth Westra, Bart JJM Van Den Hurk, Sonia I Seneviratne, Philip J Ward, Andy Pitman, Amir AghaKouchak, David N Bresch, Michael Leonard, Thomas Wahl, et al. Future cli- mate risk from compound events. Nature Climate Change, 2018.","title":"Future climate risk from compound events","authors":"Jakob Zscheischler, Seth Westra, Jjm Bart, Van Den, Sonia I Hurk, Philip J Seneviratne, Andy Ward, Amir Pitman, David N Aghakouchak, Michael Bresch, Thomas Leonard, Wahl","year":"2018","venue":"Nature Climate Change","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}