{"type":"ScholarlyArticle","title":"Self-adaptive Extreme Penalized Loss for Imbalanced Time Series Prediction","authors":[{"name":"Yiyang Wang","slug":"yiyang-wang","given":"Yiyang","family":"Wang","affiliation":"College of Artificial Intelligence , Dalian Maritime University"},{"name":"Yuchen Han","slug":"yuchen-han","given":"Yuchen","family":"Han","affiliation":"College of Artificial Intelligence , Dalian Maritime University"},{"name":"Yuhan Guo","slug":"yuhan-guo","given":"Yuhan","family":"Guo","affiliation":"College of Transportation Engineering , Dalian Maritime University"}],"abstract":"Forecasting time series in imbalanced data presents a significant research challenge that requires considerable attention. Although there are specialized techniques available to tackle imbalanced time series prediction, existing approaches tend to prioritize extreme predictions at the expense of compromising the forecasting accuracy of normal samples. We in this paper propose an extreme penalized loss function that relaxes the constraint on overestimating extreme events, thereby imposing great penalties on both normal and underestimating extreme events. In addition, we provide a self-adaptive way for setting the hyperparameters of the loss function. Then, both the proposed loss function and an attention module are integrated with LSTM networks in a decomposition-based framework. Extensive experiments conducted on real-world datasets demonstrate the superiority of our framework compared to other state-of-the-art approaches for both time series prediction and block maxima prediction tasks.","doi":"10.24963/ijcai.2024/568","url":"https://ijcai.org/proceedings/2024/568","pdf":"https://ijcai.org/proceedings/2024/0568.pdf","markdown":"https://ijcai.org/proceedings/2024/568.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/568","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":568,"pages":{"first":5135,"last":5143},"keywords":["Machine Learning: ML: Time series and data streams"],"videos":[],"references":[{"position":1,"raw":"[Bishop and Nasrabadi, 2006] Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.","title":"Pattern recognition and machine learning","authors":"M Christopher, Bishop, M Nasser, Nasrabadi","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Box et al., 2015] George EP Box, Gwilym M Jenkins, Gre- gory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.","title":"Time series analysis: forecasting and control","authors":"George Ep Box, M Gwilym, Gregory C Jenkins, Greta M Reinsel, Ljung","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chang et al., 2020] Yue-Shan Chang, Hsin-Ta Chiao, Satheesh Abimannan, Yo-Ping Huang, Yi-Ting Tsai, and Kuan-Ming Lin. An lstm-based aggregated model for air pollution forecasting. Atmospheric Pollution Research, 11(8):1451-1463, 2020.","title":"An lstm-based aggregated model for air pollution forecasting","authors":"Yue-Shan Chang, Hsin-Ta Chiao, Satheesh Abimannan, Yo-Ping Huang, Yi-Ting Tsai, Kuan-Ming Lin","year":"2020","venue":"Atmospheric Pollution Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Connor et al., 1994] Jerome T Connor, R Douglas Martin, and Les E Atlas. Recurrent neural networks and robust time series prediction. IEEE Transactions on Neural Net- works, 5(2):240-254, 1994.","title":"Recurrent neural networks and robust time series prediction","authors":"R Jerome T Connor, Les E Douglas Martin, Atlas","year":"1994","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ding et al., 2019] Daizong Ding, Mi Zhang, Xudong Pan, Min Yang, and Xiangnan He. Modeling extreme events in time series prediction. In ACM SIGKDD, 2019.","title":"Modeling extreme events in time series prediction","authors":"Ding","year":"2019","venue":"ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fu et al., 2023] Yang Fu, Feixiang Ying, Lingling Huang, and Yang Liu. Multi-step-ahead significant wave height prediction using a hybrid model based on an innovative two-layer decomposition framework and lstm. Renewable Energy, 203:455-472, 2023.","title":"Multi-step-ahead significant wave height prediction using a hybrid model based on an innovative two-layer decomposition framework and lstm","authors":"Yang Fu, Feixiang Ying, Lingling Huang, Yang Liu","year":"2023","venue":"Renewable Energy","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Galib et al., 2022] Asadullah Hill Galib, Andrew McDon- ald, Tyler Wilson, Lifeng Luo, and Pang-Ning Tan. Deep- extrema: A deep learning approach for forecasting block maxima in time series data. In IJCAI, 2022.","title":"Deepextrema: A deep learning approach for forecasting block maxima in time series data","authors":"Asadullah Hill Galib, Andrew Mcdonald, Tyler Wilson, Lifeng Luo, Pang-Ning Tan","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/413"},{"position":8,"raw":"[Galib et al., 2023] Abu Hasnat Galib, Aaron McDonald, Ping-Nam Tan, et al. Self-recover: Forecasting block max- ima in time series from predictors with disparate temporal coverage using self-supervised learning. In IJCAI, 2023.","title":"Self-recover: Forecasting block maxima in time series from predictors with disparate temporal coverage using self-supervised learning","authors":"Galib","year":"2023","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2023/414"},{"position":9,"raw":"[Geurts et al., 2006] Pierre Geurts, Damien Ernst, and Louis Wehenkel. Extremely randomized trees. Machine Learn- ing, 63:3-42, 2006.","title":"Extremely randomized trees","authors":"Geurts","year":"2006","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gilles, 2013] Jérôme Gilles. Empirical wavelet transform. IEEE Transactions on Signal Processing, 61(16):3999- 4010, 2013.","title":"Empirical wavelet transform","authors":"Jérôme Gilles","year":"2013","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hou et al., 2021] Cheng Hou, Jiahui Wu, Bin Cao, and et al. A deep-learning prediction model for imbalanced time se- ries data forecasting. Big Data Mining and Analytics, 4(4):266-278, 2021.","title":"A deep-learning prediction model for imbalanced time series data forecasting","authors":"Cheng Hou, Jiahui Wu, Bin Cao","year":"2021","venue":"Big Data Mining and Analytics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Karevan and Suykens, 2020] Zahra Karevan and Jo- han A.K. Suykens. Transductive lstm for time-series prediction: An application to weather forecasting. Neural Networks, 125:1-9, 2020.","title":"Transductive lstm for time-series prediction: An application to weather forecasting","authors":"Suykens Karevan, Zahra Karevan, Johan A K Suykens","year":"2020","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Laptev et al., 2017] Nikolay Laptev, Jason Yosinski, Li Er- ran Li, and Slawek Smyl. Time-series extreme event fore- casting with neural networks at uber. In ICML, 2017.","title":"Time-series extreme event forecasting with neural networks at uber","authors":"Laptev","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2023] Yanhong Li, Jack Xu, and David C Anas- tasiu. An extreme-adaptive time series prediction model based on probability-enhanced lstm neural networks. In AAAI, 2023.","title":"An extreme-adaptive time series prediction model based on probability-enhanced lstm neural networks","authors":"Yanhong Li, Jack Xu, David C Anastasiu","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2023] Hengbo Liu, Ziqing Ma, Linxiao Yang, Tian Zhou, Rui Xia, Yi Wang, Qingsong Wen, and Liang Sun. Sadi: A self-adaptive decomposed interpretable framework for electric load forecasting under extreme events. In ICASSP, 2023.","title":"Sadi: A self-adaptive decomposed interpretable framework for electric load forecasting under extreme events","authors":"Liu","year":"2023","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Luo et al., 2022] Qin-Rui Luo, Hang Xu, and Long-Hu Bai. Prediction of significant wave height in hurricane area of the atlantic ocean using the bi-lstm with attention model. Ocean Engineering, 266:112747, 2022.","title":"Prediction of significant wave height in hurricane area of the atlantic ocean using the bi-lstm with attention model","authors":"Qin-Rui Luo, Hang Xu, Long-Hu Bai","year":"2022","venue":"Ocean Engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Oreshkin et al., 2020] Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series fore- casting. In ICLR, 2020.","title":"N-beats: Neural basis expansion analysis for interpretable time series forecasting","authors":"Dmitri Boris N Oreshkin, Nicolas Carpov, Yoshua Chapados, Bengio","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Polson and Sokolov, 2020] Nicholas G Polson and Vadim Sokolov. Deep learning for energy markets. Applied Stochastic Models in Business and Industry, 36(1):195- 209, 2020.","title":"Deep learning for energy markets","authors":"G Nicholas, Vadim Polson, Sokolov","year":"2020","venue":"Applied Stochastic Models in Business and Industry","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rajbhandari et al., 2021] Yaju Rajbhandari, Anup Mara- hatta, Bishal Ghimire, Ashish Shrestha, Anand Gachhadar, Anup Thapa, Kamal Chapagain, and Petr Korba. Impact study of temperature on the time series electricity demand of urban nepal for short-term load forecasting. Applied System Innovation, 4(3):43, 2021.","title":"Impact study of temperature on the time series electricity demand of urban nepal for short-term load forecasting","authors":"Yaju Rajbhandari, Anup Marahatta, Bishal Ghimire, Ashish Shrestha, Anand Gachhadar, Anup Thapa, Petr Kamal Chapagain, Korba","year":"2021","venue":"Applied System Innovation","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sapankevych and Sankar, 2009] Nicholas I. Sapankevych and Ravi Sankar. Time series prediction using support vec- tor machines: A survey. IEEE Computational Intelligence Magazine, 4(2):24-38, 2009.","title":"Time series prediction using support vector machines: A survey","authors":"Nicholas I Sapankevych, Ravi Sankar","year":"2009","venue":"IEEE Computational Intelligence Magazine","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schuster and Paliwal, 1997] Mike Schuster and Kuldip K Paliwal. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673-2681, 1997.","title":"Bidirectional recurrent neural networks","authors":"Mike Schuster, Kuldip K Paliwal","year":"1997","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Siami-Namini et al., 2018] Sima Siami-Namini, Neda Tavakoli, and Akbar Siami Namin. A comparison of arima and lstm in forecasting time series. In IEEE International Conference on Machine Learning and Applications, 2018.","title":"A comparison of arima and lstm in forecasting time series","authors":"Sima Siami-Namini, Neda Tavakoli, Akbar Siami Namin","year":"2018","venue":"IEEE International Conference on Machine Learning and Applications","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Siami-Namini et al., 2019] Soheil Siami-Namini, Nader Tavakoli, and Ahmad S Namin. The performance of lstm and bilstm in forecasting time series. In Big Data, 2019.","title":"The performance of lstm and bilstm in forecasting time series","authors":"Soheil Siami-Namini, Nader Tavakoli, Ahmad S Namin","year":"2019","venue":"Big Data","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tyralis and Papacharalampous, 2017] Hristos Tyralis and Georgia Papacharalampous. Variable selection in time series forecasting using random forests. Algorithms, 10(4):114, 2017.","title":"Hristos Tyralis and Georgia Papacharalampous. Variable selection in time series forecasting using random forests","authors":"Papacharalampous Tyralis","year":"2017","venue":"Algorithms","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wambura et al., 2020] Stephen Wambura, He Li, and Alemu Nigussie. Fast memory-efficient extreme events prediction in complex time series. In ICRSA, 2020.","title":"Fast memory-efficient extreme events prediction in complex time series","authors":"Stephen Wambura, He Li, Alemu Nigussie","year":"2020","venue":"ICRSA","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Weerakody et al., 2021] Philip B. Weerakody, Kok Wai Wong, Guanjin Wang, and Wendell Ela. A review of irreg- ular time series data handling with gated recurrent neural networks. Neurocomputing, 2021.","title":"A review of irregular time series data handling with gated recurrent neural networks","authors":"Philip B Weerakody, Wai Kok, Guanjin Wong, Wendell Wang, Ela","year":"2021","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wilson et al., 2022a] Tom Wilson, Ping-Nam Tan, and Liang Luo. Deepgpd: A deep learning approach for simu- lating geospatial temporal extreme events. In AAAI, 2022.","title":"Deepgpd: A deep learning approach for simulating geospatial temporal extreme events","authors":"Wilson","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wilson et al., 2022b] Tyler Wilson, Andrew McDonald, Asadullah Hill Galib, Pang-Ning Tan, and Lifeng Luo. Be- yond point prediction: Capturing zero-inflated & heavy- tailed spatiotemporal data with deep extreme mixture models. In ACM SIGKDD, 2022.","title":"Beyond point prediction: Capturing zero-inflated & heavytailed spatiotemporal data with deep extreme mixture models","authors":"Wilson","year":"2022","venue":"ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wu et al., 2021] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transform- ers with auto-correlation for long-term series forecasting. In NeurIPS, 2021.","title":"Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting","authors":"Wu","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zha et al., 2022] Wenting Zha, Jie Liu, Yalong Li, and Yingyu Liang. Ultra-short-term power forecast method for the wind farm based on feature selection and tempo- ral convolution network. ISA Transactions, 129:405-414, 2022.","title":"Ultra-short-term power forecast method for the wind farm based on feature selection and temporal convolution network","authors":"Wenting Zha, Jie Liu, Yalong Li, Yingyu Liang","year":"2022","venue":"ISA Transactions","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhou et al., 2021] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, 2021.","title":"Informer: Beyond efficient transformer for long sequence time-series forecasting","authors":"Zhou","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhou et al., 2022] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Fre- quency enhanced decomposed transformer for long-term series forecasting. In ICML, 2022.","title":"Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting","authors":"Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, Rong Jin","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhu and Laptev, 2017] Lingxue Zhu and Nikolay Laptev. Deep and confident prediction for time series at uber. In ICDM, 2017.","title":"Deep and confident prediction for time series at uber","authors":"Laptev Zhu, Lingxue Zhu, Nikolay Laptev","year":"2017","venue":"ICDM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}