{"type":"ScholarlyArticle","title":"CLVSA: A Convolutional LSTM Based Variational Sequence-to-Sequence Model with Attention for Predicting Trends of Financial Markets","authors":[{"name":"Jia Wang","slug":"jia-wang","given":"Jia","family":"Wang","affiliation":"Department of Computer Science , University of Massachusetts Lowell"},{"name":"Tong Sun","slug":"tong-sun","given":"Tong","family":"Sun","affiliation":"Department of Computer Science , University of Massachusetts Lowell"},{"name":"Benyuan Liu","slug":"benyuan-liu","given":"Benyuan","family":"Liu","affiliation":"Department of Computer Science , University of Massachusetts Lowell"},{"name":"Yu Cao","slug":"yu-cao","given":"Yu","family":"Cao","affiliation":"Department of Computer Science , University of Massachusetts Lowell"},{"name":"Hongwei Zhu","slug":"hongwei-zhu","given":"Hongwei","family":"Zhu","affiliation":"Department of Operations and Information Systems , University of Massachusetts Lowell"}],"abstract":"Financial markets are a complex dynamical system. The complexity comes from the interaction between a market and its participants, in other words, the integrated outcome of activities of the entire participants determines the markets trend, while the markets trend affects activities of participants. These interwoven interactions make financial markets keep evolving. Inspired by stochastic recurrent models that successfully capture variability observed in natural sequential data such as speech and video, we propose CLVSA, a hybrid model that consists of stochastic recurrent networks, the sequence-to-sequence architecture, the self- and inter-attention mechanism, and convolutional LSTM units to capture variationally underlying features in raw financial trading data. Our model outperforms basic models, such as convolutional neural network, vanilla LSTM network, and sequence-to-sequence model with attention, based on backtesting results of six futures from January 2010 to December 2017. Our experimental results show that, by introducing an approximate posterior, CLVSA takes advantage of an extra regularizer based on the Kullback-Leibler divergence to prevent itself from overfitting traps.","doi":"10.24963/ijcai.2019/514","url":"https://ijcai.org/proceedings/2019/514","pdf":"https://ijcai.org/proceedings/2019/0514.pdf","markdown":"https://ijcai.org/proceedings/2019/514.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/514","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":514,"pages":{"first":3705,"last":3711},"keywords":["Multidisciplinary Topics and Applications: Finance","Machine Learning: Semi-Supervised Learning","Machine Learning: Deep Learning","Machine Learning: Time-series;Data Streams"],"videos":[],"references":[{"position":1,"raw":"Achelis, 2001","title":null,"authors":"Achelis","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Steven B Achelis. Technical Analysis from A to Z. McGraw Hill New York, 2001.","title":"Technical Analysis from A to Z","authors":"Steven B Achelis","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bacoyannis et al., 2018] Vangelis Bacoyannis, Vacslav Glukhov, Tom Jin, Jonathan Kochems, and Doo Re Song. Idiosyncrasies and challenges of data driven learning in electronic trading. arXiv preprint arXiv:1811.09549, 2018.","title":"Idiosyncrasies and challenges of data driven learning in electronic trading","authors":"Vangelis Bacoyannis, Vacslav Glukhov, Tom Jin, Jonathan Kochems, Doo Re Song","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bayer and Osendorfer, 2014] Justin Bayer and Christian Osendorfer. Learning stochastic recurrent networks. arXiv preprint arXiv:1411.7610, 2014.","title":"Learning stochastic recurrent networks","authors":"Osendorfer Bayer","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bowman et al., 2015] Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349, 2015.","title":"Generating sentences from a continuous space","authors":"Luke Samuel R Bowman, Oriol Vilnis, Andrew M Vinyals, Rafal Dai, Samy Jozefowicz, Bengio","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2016] Jianpeng Cheng, Li Dong, and Mirella Lapata. Long short-term memory-networks for machine reading. arXiv preprint arXiv:1601.06733, 2016.","title":"Long short-term memory-networks for machine reading","authors":"Cheng","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ding et al., 2015] Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan. Deep learning for event-driven stock pre- diction. In Ijcai, pages 2327-2333, 2015.","title":"Deep learning for event-driven stock prediction","authors":"Xiao Ding, Yue Zhang, Ting Liu, Junwen Duan","year":"2015","venue":"Ijcai","doi":null,"url":null,"internal_url":"/proceedings/2015/329"},{"position":9,"raw":"[Dixon et al., 2016] Matthew Dixon, Diego Klabjan, and Jin Hoon Bang. Classification-based financial markets pre- diction using deep neural networks. Algorithmic Finance, (Preprint):1-11, 2016.","title":"Classification-based financial markets prediction using deep neural networks","authors":"Matthew Dixon, Diego Klabjan, Jin Hoon Bang","year":"2016","venue":"Algorithmic Finance","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fernández-Rodríguez et al., 2003] Fernando Fernández- Rodríguez, Simon Sosvilla-Rivero, and Julian Andrada- Felix. Technical analysis in foreign exchange markets: evidence from the ems. Applied Financial Economics, 13(2):113-122, 2003.","title":"Technical analysis in foreign exchange markets: evidence from the ems","authors":"Fernando Fernández-Rodríguez, Simon Sosvilla-Rivero, Julian Andrada-Felix","year":"2003","venue":"Applied Financial Economics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goyal et al., 2017] Anirudh Goyal ALIAS PARTH Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rose- mary Ke, and Yoshua Bengio. Z-forcing: Training stochastic recurrent networks. In Advances in neural in- formation processing systems, pages 6713-6723, 2017.","title":"Z-forcing: Training stochastic recurrent networks","authors":"Anirudh Goyal, Alias Parth Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kaiser and Sutskever, 2015] Łukasz Kaiser and Ilya Sutskever. Neural gpus learn algorithms. arXiv preprint arXiv:1511.08228, 2015.","title":"Neural gpus learn algorithms","authors":"Łukasz Kaiser, Ilya Sutskever","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kim, 2003] Kyoung-jae Kim. Financial time series fore- casting using support vector machines. Neurocomputing, 55(1):307-319, 2003.","title":"Financial time series forecasting using support vector machines","authors":"Kyoung-Jae Kim","year":"2003","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lo, 2004] Andrew W Lo. The adaptive markets hypothesis. The Journal of Portfolio Management, 30(5):15-29, 2004.","title":"The adaptive markets hypothesis","authors":"Andrew W Lo","year":"2004","venue":"The Journal of Portfolio Management","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rechenthin, 2014] Michael David Rechenthin. Machine- learning classification techniques for the analysis and pre- diction of high-frequency stock direction. The University of Iowa, 2014.","title":"Machinelearning classification techniques for the analysis and prediction of high-frequency stock direction","authors":"Michael David, Rechenthin","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rezende et al., 2014] Danilo Jimenez Rezende, Shakir Mo- hamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082, 2014.","title":"Stochastic backpropagation and approximate inference in deep generative models","authors":"Danilo Jimenez Rezende, Shakir Mohamed, Daan Wierstra","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Re- search, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sun et al., 2017] Tong Sun, Jia Wang, Pengfei Zhang, Yu Cao, Benyuan Liu, and Degang Wang. Predicting stock price returns using microblog sentiment for chinese stock market. In 2017 3rd International Conference on Big Data Computing and Communications (BIGCOM), pages 87- 96. IEEE, 2017.","title":"Predicting stock price returns using microblog sentiment for chinese stock market","authors":"Tong Sun, Jia Wang, Pengfei Zhang, Yu Cao, Benyuan Liu, Degang Wang","year":"2017","venue":"3rd International Conference on Big Data Computing and Communications (BIGCOM)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Toderici et al., 2015] George Toderici, Sean M O'Malley, Sung Jin Hwang, Damien Vincent, David Minnen, Shumeet Baluja, Michele Covell, and Rahul Sukthankar. Variable rate image compression with recurrent neural net- works. arXiv preprint arXiv:1511.06085, 2015.","title":"Variable rate image compression with recurrent neural networks","authors":"George Toderici, M O' Sean, Sung Malley, Jin Hwang, Damien Vincent, David Minnen, Shumeet Baluja, Michele Covell, Rahul Sukthankar","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2018] Jia Wang, Tong Sun, Benyuan Liu, Yu Cao, and Degang Wang. Financial markets prediction with deep learning. In 2018 17th IEEE International Con- ference on Machine Learning and Applications (ICMLA), pages 97-104. IEEE, 2018.","title":"Financial markets prediction with deep learning","authors":"Jia Wang, Tong Sun, Benyuan Liu, Yu Cao, Degang Wang","year":"2018","venue":"17th IEEE International Conference on Machine Learning and Applications (ICMLA)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xingjian et al., 2015] SHI Xingjian, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional lstm network: A machine learning ap- proach for precipitation nowcasting. In Advances in neural information processing systems, pages 802-810, 2015.","title":"Convolutional lstm network: A machine learning approach for precipitation nowcasting","authors":"Xingjian","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu and Cohen, 2018] Yumo Xu and Shay B Cohen. Stock movement prediction from tweets and historical prices. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1970-1979, 2018.","title":"Stock movement prediction from tweets and historical prices","authors":"Cohen ; ] Yumo Xu, Shay B Xu, Cohen","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2017] Liheng Zhang, Charu Aggarwal, and Guo-Jun Qi. Stock price prediction via discovering multi- frequency trading patterns. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2141-2149. ACM, 2017.","title":"Stock price prediction via discovering multifrequency trading patterns","authors":"Zhang","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}