{"type":"ScholarlyArticle","title":"Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting","authors":[{"name":"Longyuan Li","slug":"longyuan-li","given":"Longyuan","family":"Li","affiliation":"State Key Lab of Advanced Optical Communication System and Network; MoE Key Lab of Artificial Intelligence , AI Institute"},{"name":"Junchi Yan","slug":"junchi-yan","given":"Junchi","family":"Yan","affiliation":"MoE Key Lab of Artificial Intelligence , AI Institute; Department of Computer Science and Engineering Shanghai Jiao Tong University"},{"name":"Xiaokang Yang","slug":"xiaokang-yang","given":"Xiaokang","family":"Yang","affiliation":"MoE Key Lab of Artificial Intelligence , AI Institute; Department of Computer Science and Engineering Shanghai Jiao Tong University"},{"name":"Yaohui Jin","slug":"yaohui-jin","given":"Yaohui","family":"Jin","affiliation":"State Key Lab of Advanced Optical Communication System and Network; MoE Key Lab of Artificial Intelligence , AI Institute"}],"abstract":"Probabilistic time series forecasting involves estimating the distribution of future based on its history, which is essential for risk management in downstream decision-making. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks and the dependency is modeled by recurrent neural nets. We take the automatic relevance determination (ARD) view and devise a network to exploit the exogenous variables in addition to time series. In particular, our ARD network can incorporate the uncertainty of the exogenous variables and eventually helps identify useful exogenous variables and suppress those irrelevant for forecasting. The distribution of multi-step ahead forecasts are approximated by Monte Carlo simulation. We show in experiments that our model produces accurate and sharp probabilistic forecasts. The estimated uncertainty of our forecasting also realistically increases over time, in a spontaneous manner.","doi":"10.24963/ijcai.2019/402","url":"https://ijcai.org/proceedings/2019/402","pdf":"https://ijcai.org/proceedings/2019/0402.pdf","markdown":"https://ijcai.org/proceedings/2019/402.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/402","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":402,"pages":{"first":2901,"last":2908},"keywords":["Machine Learning: Interpretability","Machine Learning: Probabilistic Machine Learning","Uncertainty in AI: Uncertainty in AI","Machine Learning: Time-series;Data Streams"],"videos":[],"references":[{"position":1,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-1828, 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bishop and Tipping, 2000] Christopher M Bishop and Michael E Tipping. Variational relevance vector ma- chines. In Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence, pages 46-53.","title":"Variational relevance vector machines","authors":"M Christopher, Michael E Bishop, Tipping","year":null,"venue":"Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Box et al., 2015] George EP Box, Gwilym M Jenkins, Gre- gory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.","title":"Time series analysis: forecasting and control","authors":"George Ep Box, M Gwilym, Gregory C Jenkins, Greta M Reinsel, Ljung","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brockwell et al., 2002] Peter J Brockwell, Richard A Davis, and Matthew V Calder. Introduction to time series and forecasting, volume 2. Springer, 2002.","title":"Introduction to time series and forecasting","authors":"J Peter, Richard A Brockwell, Matthew V Davis, Calder","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Burda et al., 2015] Y. Burda, R. Grosse, and R. Salakhut- dinov. Importance weighted autoencoders. In arXiv:1509.00519, 2015.","title":"Importance weighted autoencoders","authors":"Y Burda, R Grosse, R Salakhutdinov","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cho et al., 2014] K. Cho, B. Van Merriënboer, C. Gul- cehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In arXiv:1406.1078, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"K Cho, B Van Merriënboer, C Gulcehre, D Bahdanau, F Bougares, H Schwenk, Y Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chung et al., 2015] J. Chung, K. Kastner, L. Dinh, K. Goel, A. Courville, and Y. Bengio. A recurrent latent variable model for sequential data. In NIPS, 2015.","title":"A recurrent latent variable model for sequential data","authors":"J Chung, K Kastner, L Dinh, K Goel, A Courville, Y Bengio","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Durbin and Koopman, 2012] James Durbin and Siem Jan Koopman. Time series analysis by state space methods, volume 38. Oxford University Press, 2012.","title":"Time series analysis by state space methods","authors":"Koopman Durbin, James Durbin, Siem Jan Koopman","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Eleftheriadis et al., 2017] Stefanos Eleftheriadis, Tom Nicholson, Marc Deisenroth, and James Hensman. Identification of gaussian process state space models. In Advances in neural information processing systems, pages 5309-5319, 2017.","title":"Identification of gaussian process state space models","authors":"Stefanos Eleftheriadis, Tom Nicholson, Marc Deisenroth, James Hensman","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Flunkert et al., 2017] Valentin Flunkert, David Salinas, and Jan Gasthaus. Deepar: Probabilistic forecasting with autoregressive recurrent networks. arXiv preprint arXiv:1704.04110, 2017.","title":"Deepar: Probabilistic forecasting with autoregressive recurrent networks","authors":"Valentin Flunkert, David Salinas, Jan Gasthaus","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fraccaro et al., 2016] M. Fraccaro, S. Sønderby, U. Paquet, and O. Winther. Sequential neural models with stochastic layers. In NIPS, 2016.","title":"Sequential neural models with stochastic layers","authors":"M Fraccaro, S Sønderby, U Paquet, O Winther","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gneiting and Katzfuss, 2014] Tilmann Gneiting and Matthias Katzfuss. Probabilistic forecasting. Annual Review of Statistics and Its Application, 1:125-151, 2014.","title":"Probabilistic forecasting","authors":"Tilmann Gneiting, Matthias Katzfuss","year":"2014","venue":"Annual Review of Statistics and Its Application","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hong et al., 2016] Tao Hong, Pierre Pinson, Shu Fan, Hamidreza Zareipour, Alberto Troccoli, and Rob J Hynd- man. Probabilistic energy forecasting: Global energy fore- casting competition 2014 and beyond, 2016.","title":"Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond","authors":"Tao Hong, Pierre Pinson, Shu Fan, Hamidreza Zareipour, Alberto Troccoli, Rob J Hyndman","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hyndman et al., 2007] Rob J Hyndman, Yeasmin Khan- dakar, et al. Automatic time series for forecasting: the forecast package for R. Number 6/07. Monash University, Department of Econometrics and Business Statistics . . . , 2007.","title":"Automatic time series for forecasting: the forecast package for R","authors":"Rob J Hyndman, Yeasmin Khandakar","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jordan et al., 1999] M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul. An introduction to variational methods for graphical models. Machine learning, 37(2):183-233, 1999.","title":"An introduction to variational methods for graphical models","authors":"M Jordan, Z Ghahramani, T Jaakkola, L Saul","year":"1999","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Ba, 2014] D. Kingma and J. Ba. Adam: A method for stochastic optimization. In arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"D Kingma, J Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kingma and Welling, 2013] D. Kingma and M. Welling. Auto-encoding variational bayes. In arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"D Kingma, M Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Krishnan et al., 2015] Rahul G Krishnan, Uri Shalit, and David Sontag. Deep kalman filters. In arXiv:1511.05121, 2015.","title":"Deep kalman filters","authors":"Uri Rahul G Krishnan, David Shalit, Sontag","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Krishnan et al., 2017] Rahul G Krishnan, Uri Shalit, and David Sontag. Structured inference networks for nonlin- ear state space models. In Thirty-First AAAI Conference on Artificial Intelligence, 2017.","title":"Structured inference networks for nonlinear state space models","authors":"Uri Rahul G Krishnan, David Shalit, Sontag","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Längkvist et al., 2014] Martin Längkvist, Lars Karlsson, and Amy Loutfi. A review of unsupervised feature learn- ing and deep learning for time-series modeling. Pattern Recognition Letters, 42:11-24, 2014.","title":"A review of unsupervised feature learning and deep learning for time-series modeling","authors":"Martin Längkvist, Lars Karlsson, Amy Loutfi","year":"2014","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Leskovec et al., 2014] J. Leskovec, A. Rajaraman, and Jef- frey D. Ullman. Mining of massive datasets. Cambridge university press, 2014.","title":"Mining of massive datasets","authors":"J Leskovec, A Rajaraman, Jeffrey D Ullman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liang et al., 2015] Xuan Liang, Tao Zou, Bin Guo, Shuo Li, Haozhe Zhang, Shuyi Zhang, Hui Huang, and Song Xi Chen. Assessing beijing's pm2. 5 pollution: severity, weather impact, apec and winter heating. Proceedings of the Royal Society A: Mathematical, Physical and En- gineering Sciences, 471(2182):20150257, 2015.","title":"Assessing beijing's pm2. 5 pollution: severity, weather impact, apec and winter heating","authors":"Xuan Liang, Tao Zou, Bin Guo, Shuo Li, Haozhe Zhang, Shuyi Zhang, Hui Huang, Song Xi, Chen","year":"2015","venue":"Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mnih and Rezende, 2016] A. Mnih and D. Rezende. Vari- ational inference for monte carlo objectives. In arXiv:1602.06725, 2016.","title":"Variational inference for monte carlo objectives","authors":"A Mnih, D Rezende","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Neal, 2012] Radford M Neal. Bayesian learning for neu- ral networks, volume 118. Springer Science & Business Media, 2012.","title":"Bayesian learning for neural networks","authors":"M Radford, Neal","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rangapuram et al., 2018] Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. In Advances in Neural Information Processing Systems, pages 7796-7805, 2018.","title":"Deep state space models for time series forecasting","authors":"Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, Tim Januschowski","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wen et al., 2017] Ruofeng Wen, Kari Torkkola, Balakrish- nan Narayanaswamy, and Dhruv Madeka. A multi- horizon quantile recurrent forecaster. arXiv preprint arXiv:1711.11053, 2017.","title":"A multihorizon quantile recurrent forecaster","authors":"Wen","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wipf and Nagarajan, 2008] David P Wipf and Srikantan S Nagarajan. A new view of automatic relevance determi- nation. In Advances in neural information processing sys- tems, pages 1625-1632, 2008.","title":"A new view of automatic relevance determination","authors":"P David, Wipf, Srikantan S Nagarajan","year":"2008","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yu et al., 2016] Hsiang-Fu Yu, Nikhil Rao, and Inderjit S Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. In Advances in neural information processing systems, pages 847-855, 2016.","title":"Temporal regularized matrix factorization for high-dimensional time series prediction","authors":"Hsiang-Fu Yu, Nikhil Rao, Inderjit S Dhillon","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}