{"type":"ScholarlyArticle","title":"pTSE: A Multi-model Ensemble Method for Probabilistic Time Series Forecasting","authors":[{"name":"Yunyi Zhou","slug":"yunyi-zhou","given":"Yunyi","family":"Zhou","affiliation":"University of Virginia"},{"name":"Zhixuan Chu","slug":"zhixuan-chu","given":"Zhixuan","family":"Chu","affiliation":"University of Virginia"},{"name":"Yijia Ruan","slug":"yijia-ruan","given":"Yijia","family":"Ruan","affiliation":"University of Virginia"},{"name":"Ge Jin","slug":"ge-jin","given":"Ge","family":"Jin","affiliation":"University of Virginia"},{"name":"Yuchen Huang","slug":"yuchen-huang","given":"Yuchen","family":"Huang","affiliation":"University of Virginia"},{"name":"Sheng Li","slug":"sheng-li","given":"Sheng","family":"Li","affiliation":"University of Virginia"}],"abstract":"Various probabilistic time series forecasting models have sprung up and shown remarkably good performance. However, the choice of model highly relies on the characteristics of the input time series and the fixed distribution that model is based on. Due to the fact that the probability distributions cannot be averaged over different models straightforwardly, the current time series model ensemble methods cannot be directly applied to improve the robustness and accuracy of forecasting. To address this issue, we propose pTSE, a multi-model distribution ensemble method for probabilistic forecasting based on Hidden Markov Model (HMM). pTSE only takes off-the-shelf outputs from member models without requiring further information about each model. Besides, we provide a complete theoretical analysis of pTSE to prove that the empirical distribution of time series subject to an HMM will converge to the stationary distribution almost surely. Experiments on benchmarks show the superiority of pTSE over all member models and competitive ensemble methods.","doi":"10.24963/ijcai.2023/521","url":"https://ijcai.org/proceedings/2023/521","pdf":"https://ijcai.org/proceedings/2023/0521.pdf","markdown":"https://ijcai.org/proceedings/2023/521.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/521","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":521,"pages":{"first":4684,"last":4692},"keywords":["Machine Learning: ML: Probabilistic machine learning","Machine Learning: ML: Time series and data streams"],"videos":[],"references":[{"position":1,"raw":"[Adhikari et al., 2015] Ratnadip Adhikari, Ghanshyam Verma, and Ina Khandelwal. A model ranking based selective ensemble approach for time series forecasting. Procedia Computer Science, 48:14-21, 2015. Interna- tional Conference on Computer, Communication and Convergence (ICCC 2015).","title":"A model ranking based selective ensemble approach for time series forecasting","authors":"Adhikari","year":"2015","venue":"International Conference on Computer, Communication and Convergence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Adhikari, 2015] Ratnadip Adhikari. A neural network based linear ensemble framework for time series forecast- ing. Neurocomputing, 157:231-242, 2015.","title":"Ratnadip Adhikari. A neural network based linear ensemble framework for time series forecasting","authors":"Adhikari","year":"2015","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Akyuz et al., 2017] A. Okay Akyuz, Mitat Uysal, Berna Atak Bulbul, and M. Ozan Uysal. Ensemble approach for time series analysis in demand forecast- ing: Ensemble learning. In 2017 IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA), pages 7-12, 2017.","title":"Ensemble approach for time series analysis in demand forecasting: Ensemble learning","authors":"A Okay Akyuz, Mitat Uysal, Berna Atak Bulbul, M Ozan Uysal","year":"2017","venue":"IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bilmes and others, 1998] Jeff A Bilmes et al. A gentle tu- torial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov mod- els. International Computer Science Institute, 4(510):126, 1998.","title":"A gentle tutorial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models","authors":"Jeff A Bilmes","year":"1998","venue":"International Computer Science Institute","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chu and Li, 2023] Zhixuan Chu and Sheng Li. Continual treatment effect estimation: Challenges and opportunities. arXiv preprint arXiv:2301.01026, 2023.","title":"Continual treatment effect estimation: Challenges and opportunities","authors":"Chu, Li ;, Zhixuan Chu, Sheng Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chu et al., 2023] Zhixuan Chu, Ruopeng Li, Stephen Rath- bun, and Sheng Li. Continual causal inference with incremental observational data. arXiv preprint arXiv:2303.01775, 2023.","title":"Continual causal inference with incremental observational data","authors":"Chu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Faraway and Jhun, 1990] Julian J Faraway and Myoungshic Jhun. Bootstrap choice of bandwidth for density esti- mation. Journal of the American Statistical Association, 85(412):1119-1122, 1990.","title":"Bootstrap choice of bandwidth for density estimation","authors":"J Julian, Myoungshic Faraway, Jhun","year":"1990","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Godahewa et al., 2021] Rakshitha Godahewa, Kasun Ban- dara, Geoffrey I Webb, Slawek Smyl, and Christoph Bergmeir. Ensembles of localised models for time se- ries forecasting. Knowledge-Based Systems, 233:107518, 2021.","title":"Ensembles of localised models for time series forecasting","authors":"Godahewa","year":"2021","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Januschowski et al., 2020] Tim Januschowski, Jan Gasthaus, Yuyang Wang, David Salinas, Valentin Flunkert, Michael Bohlke-Schneider, and Laurent Callot. Criteria for classifying forecasting methods. Interna- tional Journal of Forecasting, 36(1):167-177, 2020. M4 Competition.","title":"Criteria for classifying forecasting methods","authors":"Tim Januschowski, Jan Gasthaus, Yuyang Wang, David Salinas, Valentin Flunkert, Michael Bohlke-Schneider, Laurent Callot","year":"2020","venue":"International Journal of Forecasting","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Koochali et al., 2021] Alireza Koochali, Andreas Dengel, and Sheraz Ahmed. If you like it, gan it-probabilistic multivariate times series forecast with gan. In Engineering Proceedings, volume 5, page 40. Multidisciplinary Digital Publishing Institute, 2021.","title":"If you like it, gan it-probabilistic multivariate times series forecast with gan","authors":"Alireza Koochali, Andreas Dengel, Sheraz Ahmed","year":"2021","venue":"Engineering Proceedings","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lim et al., 2021] Bryan Lim, Sercan Ö Arık, Nicolas Lo- eff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. Inter- national Journal of Forecasting, 2021.","title":"Temporal fusion transformers for interpretable multi-horizon time series forecasting","authors":"Bryan Lim, Ö Sercan, Nicolas Arık, Tomas Loeff, Pfister","year":"2021","venue":"International Journal of Forecasting","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2019] Fagui Liu, Muqing Cai, Liangming Wang, and Yunsheng Lu. An ensemble model based on adaptive noise reducer and over-fitting prevention lstm for multi- variate time series forecasting. IEEE Access, 7:26102- 26115, 2019.","title":"An ensemble model based on adaptive noise reducer and over-fitting prevention lstm for multivariate time series forecasting","authors":"Liu","year":"2019","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Montero-Manso et al., 2020] Pablo Montero-Manso, George Athanasopoulos, Rob J. Hyndman, and Thiyanga S. Talagala. Fforma: Feature-based forecast model averaging. International Journal of Forecasting, 36(1):86-92, 2020. M4 Competition.","title":"Fforma: Feature-based forecast model averaging","authors":"Montero-Manso","year":"2020","venue":"International Journal of Forecasting","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Oliveira and Torgo, 2015] Mariana Oliveira and Luis Torgo. Ensembles for time series forecasting. In Dinh Phung and Hang Li, editors, Proceedings of the Sixth Asian Confer- ence on Machine Learning, volume 39 of Proceedings of Machine Learning Research, pages 360-370, Nha Trang City, Vietnam, 26-28 Nov 2015. PMLR.","title":"Ensembles for time series forecasting","authors":"Mariana Oliveira, Luis Torgo","year":"2015","venue":"Proceedings of the Sixth Asian Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Qiu et al., 2014] Xueheng Qiu, Le Zhang, Ye Ren, Pon- nuthurai N Suganthan, and Gehan Amaratunga. Ensemble deep learning for regression and time series forecasting. In 2014 IEEE symposium on computational intelligence in ensemble learning (CIEL), pages 1-6. IEEE, 2014.","title":"Ensemble deep learning for regression and time series forecasting","authors":"Qiu","year":"2014","venue":"IEEE symposium on computational intelligence in ensemble learning (CIEL)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Qiu et al., 2017] Xueheng Qiu, Ye Ren, Ponnuthurai Na- garatnam Suganthan, and Gehan A.J. Amaratunga. Em- pirical mode decomposition based ensemble deep learn- ing for load demand time series forecasting. Applied Soft Computing, 54:246-255, 2017.","title":"Empirical mode decomposition based ensemble deep learning for load demand time series forecasting","authors":"Qiu","year":"2017","venue":"Applied Soft Computing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rangapuram et al., 2018] Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. Advances in neural information processing systems, 31:7785-7794, 2018.","title":"Deep state space models for time series forecasting","authors":"Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, Tim Januschowski","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rasul et al., 2020] Kashif Rasul, Abdul-Saboor Sheikh, Ingmar Schuster, Urs Bergmann, and Roland Voll- graf. Multivariate probabilistic time series forecast- ing via conditioned normalizing flows. arXiv preprint arXiv:2002.06103, 2020.","title":"Multivariate probabilistic time series forecasting via conditioned normalizing flows","authors":"Kashif Rasul, Abdul-Saboor Sheikh, Ingmar Schuster, Urs Bergmann, Roland Vollgraf","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ravagli, 2021] Davide Ravagli. Mixture autoregressive models with applications to heteroskedastic time series. The University of Manchester (United Kingdom), 2021.","title":"Mixture autoregressive models with applications to heteroskedastic time series","authors":"Davide Ravagli","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Salinas et al., 2019a] David Salinas, Michael Bohlke- Schneider, Laurent Callot, Roberto Medico, and Jan Gasthaus. High-dimensional multivariate forecasting with low-rank gaussian copula processes. arXiv preprint arXiv:1910.03002, 2019.","title":"High-dimensional multivariate forecasting with low-rank gaussian copula processes","authors":"Salinas","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Salinas et al., 2019b] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. Inter- national Journal of Forecasting, 2019.","title":"DeepAR: Probabilistic forecasting with autoregressive recurrent networks","authors":"Salinas","year":"2019","venue":"International Journal of Forecasting","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Salinas et al., 2020] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. Inter- national Journal of Forecasting, 36(3):1181-1191, 2020.","title":"Deepar: Probabilistic forecasting with autoregressive recurrent networks","authors":"David Salinas, Valentin Flunkert, Jan Gasthaus, Tim Januschowski","year":"2020","venue":"International Journal of Forecasting","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shih et al., 2019] Shun-Yao Shih, Fan-Keng Sun, and Hung-yi Lee. Temporal pattern attention for multivariate time series forecasting. Machine Learning, 108(8):1421- 1441, 2019.","title":"Temporal pattern attention for multivariate time series forecasting","authors":"Shun-Yao Shih, Fan-Keng Sun, Hung-Yi Lee","year":"2019","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Taylor and Letham, 2017] Sean J. Taylor and Benjamin Letham. Forecasting at scale. PeerJ Prepr., 5:e3190, 2017.","title":"Forecasting at scale","authors":"Sean J Taylor, Benjamin Letham","year":"2017","venue":"PeerJ Prepr","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu and Yao, 2016] Qiang Wu and Weixin Yao. Mixtures of quantile regressions. Computational Statistics & Data Analysis, 93:162-176, 2016.","title":"Mixtures of quantile regressions","authors":"Qiang Wu, Weixin Yao","year":"2016","venue":"Computational Statistics & Data Analysis","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2016] Hsiang-Fu Yu, Nikhil Rao, and Inderjit S Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, edi- tors, Advances in Neural Information Processing Systems, volume 29.","title":"Temporal regularized matrix factorization for high-dimensional time series prediction","authors":"Hsiang-Fu Yu, Nikhil Rao, Inderjit S Dhillon","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"Curran Associates, Inc., 2016.","title":null,"authors":"Curran Associates, Inc","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yuan and Kitani, 2019] Ye Yuan and Kris Kitani. Diverse trajectory forecasting with determinantal point processes. arXiv preprint arXiv:1907.04967, 2019.","title":"Diverse trajectory forecasting with determinantal point processes","authors":"Kitani Yuan, Ye Yuan, Kris Kitani","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}