{"type":"ScholarlyArticle","title":"Hybrid Neural Networks for Learning the Trend in Time Series","authors":[{"name":"Tao Lin","slug":"tao-lin","given":"Tao","family":"Lin","affiliation":"School of Computer and Communication Sciences Ecole polytechnique federale de Lausanne Lausanne , Switzerland"},{"name":"Tian Guo","slug":"tian-guo","given":"Tian","family":"Guo","affiliation":"School of Computer and Communication Sciences Ecole polytechnique federale de Lausanne Lausanne , Switzerland"},{"name":"Karl Aberer","slug":"karl-aberer","given":"Karl","family":"Aberer","affiliation":"School of Computer and Communication Sciences Ecole polytechnique federale de Lausanne Lausanne , Switzerland"}],"abstract":"The trend of time series characterizes the intermediate upward and downward behaviour of time series. Learning and forecasting the trend in time series data play an important role in many real applications, ranging from resource allocation in data centers, load schedule in smart grid, and so on. Inspired by the recent successes of neural networks, in this paper we propose TreNet, a novel end-to-end hybrid neural network to learn local and global contextual features for predicting the trend of time series. TreNet leverages convolutional neural networks (CNNs) to extract salient features from local raw data of time series. Meanwhile, considering the long-range dependency existing in the sequence of historical trends of time series, TreNet uses a long-short term memory recurrent neural network (LSTM) to capture such dependency. Then, a feature fusion layer is to learn joint representation for predicting the trend. TreNet demonstrates its effectiveness by outperforming CNN, LSTM, the cascade of CNN and LSTM, Hidden Markov Model based method and various kernel based baselines on real datasets.","doi":"10.24963/ijcai.2017/316","url":"https://ijcai.org/proceedings/2017/316","pdf":"https://ijcai.org/proceedings/2017/0316.pdf","markdown":"https://ijcai.org/proceedings/2017/316.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/316","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":316,"pages":{"first":2273,"last":2279},"keywords":["Machine Learning: Deep Learning","Machine Learning: Time-series/Data Streams"],"videos":[],"references":[{"position":1,"raw":"[Atsalakis and Valavanis, 2009] George S Atsalakis and Ki- mon P Valavanis. Forecasting stock market short-term trends using a neuro-fuzzy based methodology. Ex- pert Systems with Applications, 36(7):10696-10707, July 2009.","title":"Forecasting stock market short-term trends using a neuro-fuzzy based methodology","authors":"S George, Kimon P Atsalakis, Valavanis","year":"2009","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ballas et al., 2015] Nicolas Ballas, Li Yao, Chris Pal, and Aaron Courville. Delving deeper into convolutional net- works for learning video representations. arXiv preprint arXiv:1511.06432, 2015.","title":"Delving deeper into convolutional networks for learning video representations","authors":"Nicolas Ballas, Li Yao, Chris Pal, Aaron Courville","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bao et al., 2014] Yukun Bao, Tao Xiong, and Zhongyi Hu. Multi-step-ahead time series prediction using multiple- output support vector regression. Neurocomputing, 129:482-493, 2014.","title":"Multi-step-ahead time series prediction using multipleoutput support vector regression","authors":"Yukun Bao, Tao Xiong, Zhongyi Hu","year":"2014","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bashivan et al., 2015] Pouya Bashivan, Irina Rish, Mo- hammed Yeasin, and Noel Codella. Learning represen- tations from eeg with deep recurrent-convolutional neural networks. arXiv preprint arXiv:1511.06448, 2015.","title":"Learning representations from eeg with deep recurrent-convolutional neural networks","authors":"Pouya Bashivan, Irina Rish, Mohammed Yeasin, Noel Codella","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chang et al., 2012] Li-Chiu Chang, Pin-An Chen, and Fi- John Chang. Reinforced two-step-ahead weight adjust- ment technique for online training of recurrent neural net- works. IEEE transactions on neural networks and learn- ing systems, 23(8):1269-1278, 2012.","title":"Reinforced two-step-ahead weight adjustment technique for online training of recurrent neural networks","authors":"Li-Chiu Chang, Pin-An Chen, Fi-John Chang","year":"2012","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chung et al., 2014] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evalua- tion of gated recurrent neural networks on sequence mod- eling. arXiv preprint arXiv:1412.3555, 2014.","title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","authors":"Chung","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Donahue et al., 2015] Jeffrey Donahue, Lisa Anne Hen- dricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Darrell. Long- term recurrent convolutional networks for visual recogni- tion and description. In IEEE CVPR, pages 2625-2634, 2015.","title":"Longterm recurrent convolutional networks for visual recognition and description","authors":"Jeffrey Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, Trevor Darrell","year":"2015","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2016] Tian Guo, Zhao Xu, Xin Yao, Haifeng Chen, Karl Aberer, and Koichi Funaya. Robust online time series prediction with recurrent neural networks. In 2016 IEEE DSAA, pages 816-825. IEEE, 2016.","title":"Robust online time series prediction with recurrent neural networks","authors":"Tian Guo, Zhao Xu, Xin Yao, Haifeng Chen, Karl Aberer, Koichi Funaya","year":"2016","venue":"IEEE DSAA","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hammerla et al., 2016] Nils Y Hammerla, Shane Halloran, and Thomas Ploetz. Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880, 2016.","title":"Deep, convolutional, and recurrent models for human activity recognition using wearables","authors":"Y Nils, Shane Hammerla, Thomas Halloran, Ploetz","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2016/220"},{"position":10,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lipton et al., 2015] Zachary C Lipton, David C Kale, Charles Elkan, and Randall Wetzell. Learning to diag- nose with lstm recurrent neural networks. arXiv preprint arXiv:1511.03677, 2015.","title":"Learning to diagnose with lstm recurrent neural networks","authors":"David C Zachary C Lipton, Charles Kale, Randall Elkan, Wetzell","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2015] Jiajun Liu, Kun Zhao, Brano Kusy, Ji-rong Wen, and Raja Jurdak. Temporal embedding in convolu- tional neural networks for robust learning of abstract snip- pets. arXiv preprint arXiv:1502.05113, 2015.","title":"Temporal embedding in convolutional neural networks for robust learning of abstract snippets","authors":"Jiajun Liu, Kun Zhao, Brano Kusy, Ji-Rong Wen, Raja Jurdak","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Malhotra et al., 2015] Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal. Long short term memory networks for anomaly detection in time series. In European Symposium on Artificial Neural Networks, vol- ume 23, 2015.","title":"Long short term memory networks for anomaly detection in time series","authors":"Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, Puneet Agarwal","year":"2015","venue":"European Symposium on Artificial Neural Networks","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mao et al., 2014] Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan Yuille. Deep captioning with multimodal recurrent neural networks (m-rnn). arXiv preprint arXiv:1412.6632, 2014.","title":"Deep captioning with multimodal recurrent neural networks (m-rnn)","authors":"Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, Alan Yuille","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Matsubara et al., 2014] Yasuko Matsubara, Yasushi Saku- rai, and Christos Faloutsos. Autoplait: Automatic mining of co-evolving time sequences. In ACM SIGMOD, pages 193-204. ACM, 2014.","title":"Autoplait: Automatic mining of co-evolving time sequences","authors":"Yasuko Matsubara, Yasushi Sakurai, Christos Faloutsos","year":"2014","venue":"ACM SIGMOD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Taieb and Atiya, 2016] Souhaib Ben Taieb and Amir F Atiya. A bias and variance analysis for multistep-ahead time series forecasting. IEEE transactions on neural net- works and learning systems, 27(1):62-76, 2016.","title":"A bias and variance analysis for multistep-ahead time series forecasting","authors":"Atiya ; Taieb, Ben Souhaib, Taieb, Amir F Atiya","year":"2016","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Venkatraman et al., 2015] Arun Venkatraman, Martial Hebert, and J Andrew Bagnell. Improving multi-step prediction of learned time series models. In AAAI, pages 3024-3030, 2015.","title":"Improving multi-step prediction of learned time series models","authors":"Arun Venkatraman, Martial Hebert, Andrew Bagnell","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vinyals et al., 2015] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neu- ral image caption generator. In IEEE CVPR, pages 3156- 3164, 2015.","title":"Show and tell: A neural image caption generator","authors":"Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan","year":"2015","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang and Oates, 2015] Zhiguang Wang and Tim Oates. En- coding time series as images for visual inspection and clas- sification using tiled convolutional neural networks. In Workshops at AAAI, 2015.","title":"Encoding time series as images for visual inspection and classification using tiled convolutional neural networks","authors":"Zhiguang Wang, Tim Oates","year":"2015","venue":"Workshops at AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2011] Peng Wang, Haixun Wang, and Wei Wang. Finding semantics in time series. In ACM KDD, pages 385-396. ACM, 2011.","title":"Finding semantics in time series","authors":"Peng Wang, Haixun Wang, Wei Wang","year":"2011","venue":"ACM KDD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2016a] Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, and Wei Xu. Cnn-rnn: A unified framework for multi-label image classification. arXiv preprint arXiv:1604.04573, 2016.","title":"Cnn-rnn: A unified framework for multi-label image classification","authors":"Wang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2016b] Linlin Wang, Zhu Cao, Yu Xia, and Gerard de Melo. Morphological segmentation with win- dow lstm neural networks. In AAAI, 2016.","title":"Morphological segmentation with window lstm neural networks","authors":"Wang","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2015] Jian Bo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, and Shonali Krishnaswamy. Deep convolutional neural networks on multichannel time series for human activity recognition. In IJCAI, pages 25- 31, 2015.","title":"Deep convolutional neural networks on multichannel time series for human activity recognition","authors":"Bo Jian, Minh Nhut Yang, Phyo Nguyen, Xiao Li Phyo San, Shonali Li, Krishnaswamy","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/561"},{"position":25,"raw":"[Yuan, 2015] Chao Yuan. Unsupervised machine condition monitoring using segmental hidden markov models. In AAAI, pages 4009-4016. AAAI Press, 2015.","title":"Unsupervised machine condition monitoring using segmental hidden markov models","authors":"Chao Yuan","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":"/proceedings/2015/563"},{"position":26,"raw":"[Zhao and Magoulès, 2012] Hai-xiang Zhao and Frédéric Magoulès. A review on the prediction of building en- ergy consumption. Renewable and Sustainable Energy Re- views, 16(6):3586-3592, 2012.","title":"A review on the prediction of building energy consumption","authors":"Hai-Xiang Zhao, Frédéric Magoulès","year":"2012","venue":"Renewable and Sustainable Energy Reviews","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}