{"type":"ScholarlyArticle","title":"Deep Variational Koopman Models: Inferring Koopman Observations for Uncertainty-Aware Dynamics Modeling and Control","authors":[{"name":"Jeremy Morton","slug":"jeremy-morton","given":"Jeremy","family":"Morton","affiliation":"Department of Aeronautics and Astronautics , Stanford University"},{"name":"Freddie D. Witherden","slug":"freddie-d-witherden","given":"Freddie D.","family":"Witherden","affiliation":"Department of Ocean Engineering , Texas A&M University"},{"name":"Mykel J. Kochenderfer","slug":"mykel-j-kochenderfer","given":"Mykel J.","family":"Kochenderfer","affiliation":"Department of Aeronautics and Astronautics , Stanford University"}],"abstract":"Koopman theory asserts that a nonlinear dynamical system can be mapped to a linear system, where the Koopman operator advances observations of the state forward in time. However, the observable functions that map states to observations are generally unknown. We introduce the Deep Variational Koopman (DVK) model, a method for inferring distributions over observations that can be propagated linearly in time. By sampling from the inferred distributions, we obtain a distribution over dynamical models, which in turn provides a distribution over possible outcomes as a modeled system advances in time. Experiments show that the DVK model is effective at long-term prediction for a variety of dynamical systems. Furthermore, we describe how to incorporate the learned models into a control framework, and demonstrate that accounting for the uncertainty present in the distribution over dynamical models enables more effective control.","doi":"10.24963/ijcai.2019/440","url":"https://ijcai.org/proceedings/2019/440","pdf":"https://ijcai.org/proceedings/2019/0440.pdf","markdown":"https://ijcai.org/proceedings/2019/440.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/440","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":440,"pages":{"first":3173,"last":3179},"keywords":["Planning and Scheduling: Planning under Uncertainty","Robotics: Behavior and Control","Machine Learning: Deep Learning","Machine Learning: Ensemble Methods"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2015] Martın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. Tensorflow: Large-scale machine learning on hetero- geneous systems. arXiv preprint arXiv:1603.04467, 2015.","title":"Tensorflow: Large-scale machine learning on heterogeneous systems","authors":"Abadi","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Banijamali et al., 2018] Ershad Banijamali, Rui Shu, Mo- hammad Ghavamzadeh, Hung Hai Bui, and Ali Ghodsi. Robust locally-linear controllable embedding. In Interna- tional Conference on Artificial Intelligence and Statistics, (AISTATS), 2018.","title":"Robust locally-linear controllable embedding","authors":"Ershad Banijamali, Rui Shu, Mohammad Ghavamzadeh, Hung Hai Bui, Ali Ghodsi","year":"2018","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brockman et al., 2016] Greg Brockman, Vicki Cheung, Lud- wig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. OpenAI Gym. arXiv preprint arXiv:1606.01540, 2016.","title":"OpenAI Gym","authors":"Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chua et al., 2018] Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learn- ing in a handful of trials using probabilistic dynamics mod- els. In Advances in Neural Information Processing Systems (NeurIPS), 2018.","title":"Deep reinforcement learning in a handful of trials using probabilistic dynamics models","authors":"Chua","year":"2018","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fraccaro et al., 2017] Marco Fraccaro, Simon Kamronn, Ul- rich Paquet, and Ole Winther. A disentangled recognition and nonlinear dynamics model for unsupervised learning. In Advances in Neural Information Processing Systems (NeurIPS), 2017.","title":"A disentangled recognition and nonlinear dynamics model for unsupervised learning","authors":"Marco Fraccaro, Simon Kamronn, Ulrich Paquet, Ole Winther","year":"2017","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hafner et al., 2018] Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. arXiv preprint arXiv:1811.04551, 2018.","title":"Learning latent dynamics for planning from pixels","authors":"Hafner","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kaiser et al., 2017] Eurika Kaiser, J Nathan Kutz, and Steven L Brunton. Data-driven discovery of Koop- man eigenfunctions for control. arXiv preprint arXiv:1707.01146, 2017.","title":"Data-driven discovery of Koopman eigenfunctions for control","authors":"Eurika Kaiser, Nathan Kutz, Steven L Brunton","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Karl et al., 2017] Maximilian Karl, Maximilian Soelch, Justin Bayer, and Patrick van der Smagt. Deep variational Bayes filters: Unsupervised learning of state space models from raw data. In International Conference on Learning Representations (ICLR), 2017.","title":"Deep variational Bayes filters: Unsupervised learning of state space models from raw data","authors":"Maximilian Karl, Maximilian Soelch, Justin Bayer, Patrick Van Der Smagt","year":"2017","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Koopman, 1931] B. O. Koopman. Hamiltonian systems and transformation in Hilbert space. Proceedings of the Na- tional Academy of Sciences, 17(5):315-318, 1931.","title":"Hamiltonian systems and transformation in Hilbert space","authors":"B O Koopman","year":"1931","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Korda and Mezić, 2018] Milan Korda and Igor Mezić. Lin- ear predictors for nonlinear dynamical systems: Koop- man operator meets model predictive control. Automatica, 93:149-160, 2018.","title":"Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control","authors":"Mezić Korda, Milan Korda, Igor Mezić","year":"2018","venue":"Automatica","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krishnan et al., 2017] Rahul G Krishnan, Uri Shalit, and David Sontag. Structured inference networks for nonlin- ear state space models. In AAAI Conference on Artificial Intelligence, 2017.","title":"Structured inference networks for nonlinear state space models","authors":"Uri Rahul G Krishnan, David Shalit, Sontag","year":"2017","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2017] Qianxiao Li, Felix Dietrich, Erik M Bollt, and Ioannis G Kevrekidis. Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator. Chaos: An Interdisciplinary Journal of Nonlinear Science, 27(10):103111, 2017.","title":"Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator","authors":"Li","year":"2017","venue":"Chaos: An Interdisciplinary Journal of Nonlinear Science","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lusch et al., 2018] Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Deep learning for universal linear em- beddings of nonlinear dynamics. Nature Communications, 9(1):4950, 2018.","title":"Deep learning for universal linear embeddings of nonlinear dynamics","authors":"Lusch","year":"2018","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Moerland et al., 2017] Thomas M Moerland, Joost Broekens, and Catholijn M Jonker. Learning multimodal transition dynamics for model-based reinforcement learning. arXiv preprint arXiv:1705.00470, 2017.","title":"Learning multimodal transition dynamics for model-based reinforcement learning","authors":"Joost Thomas M Moerland, Catholijn M Broekens, Jonker","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Morton et al., 2018] Jeremy Morton, Freddie D Witherden, Antony Jameson, and Mykel J Kochenderfer. Deep dy- namical modeling and control of unsteady fluid flows. In Advances in Neural Information Processing Systems (NeurIPS), 2018.","title":"Deep dynamical modeling and control of unsteady fluid flows","authors":"Jeremy Morton, Freddie D Witherden, Antony Jameson, Mykel, Kochenderfer","year":"2018","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nair et al., 2018] Ashvin V Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, and Sergey Levine. Visual reinforcement learning with imagined goals. In Advances in Neural Information Processing Systems (NeurIPS), 2018.","title":"Visual reinforcement learning with imagined goals","authors":"Vitchyr Ashvin V Nair, Murtaza Pong, Shikhar Dalal, Steven Bahl, Sergey Lin, Levine","year":"2018","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Otto and Rowley, 2017] Samuel E Otto and Clarence W Rowley. Linearly-recurrent autoencoder networks for learn- ing dynamics. arXiv preprint arXiv:1712.01378, 2017.","title":"Linearly-recurrent autoencoder networks for learning dynamics","authors":"Rowley ; Otto, E Samuel, Clarence W Otto, Rowley","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Proctor et al., 2018] Joshua L Proctor, Steven L Brunton, and J Nathan Kutz. Generalizing Koopman theory to allow for inputs and control. SIAM Journal on Applied Dynamical Systems, 17(1):909-930, 2018.","title":"Generalizing Koopman theory to allow for inputs and control","authors":"Joshua L Proctor, Steven L Brunton, Nathan Kutz","year":"2018","venue":"SIAM Journal on Applied Dynamical Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rangapuram et al., 2018] Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. In Advances in Neural Information Processing Systems (NeurIPS), 2018.","title":"Deep state space models for time series forecasting","authors":"Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, Tim Januschowski","year":"2018","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Takeishi et al., 2017] Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. Learning Koopman invariant subspaces for dynamic mode decomposition. In Advances in Neural Information Processing Systems (NeurIPS), 2017.","title":"Learning Koopman invariant subspaces for dynamic mode decomposition","authors":"Naoya Takeishi, Yoshinobu Kawahara, Takehisa Yairi","year":"2017","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tassa et al., 2012] Yuval Tassa, Tom Erez, and Emanuel Todorov. Synthesis and stabilization of complex behav- iors through online trajectory optimization. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4906-4913. IEEE, 2012.","title":"Synthesis and stabilization of complex behaviors through online trajectory optimization","authors":"Tassa","year":"2012","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Venkatraman et al., 2015] Arun Venkatraman, Martial Hebert, and J. Andrew Bagnell. Improving multi-step prediction of learned time series models. In AAAI Conference on Artificial Intelligence, 2015.","title":"Improving multi-step prediction of learned time series models","authors":"Arun Venkatraman, Martial Hebert, J Andrew Bagnell","year":"2015","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Watter et al., 2015] Manuel Watter, Jost Springenberg, Joschka Boedecker, and Martin Riedmiller. Embed to con- trol: A locally linear latent dynamics model for control from raw images. In Advances in Neural Information Pro- cessing Systems (NeurIPS), 2015.","title":"Embed to control: A locally linear latent dynamics model for control from raw images","authors":"Manuel Watter, Jost Springenberg, Joschka Boedecker, Martin Riedmiller","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}