{"type":"ScholarlyArticle","title":"Spotlight News Driven Quantitative Trading Based on Trajectory Optimization","authors":[{"name":"Mengyuan Yang","slug":"mengyuan-yang","given":"Mengyuan","family":"Yang","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"Mengying Zhu","slug":"mengying-zhu","given":"Mengying","family":"Zhu","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"Qianqiao Liang","slug":"qianqiao-liang","given":"Qianqiao","family":"Liang","affiliation":"MYbank , Ant Group , Hangzhou , China"},{"name":"Xiaolin Zheng","slug":"xiaolin-zheng","given":"Xiaolin","family":"Zheng","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"MengHan Wang","slug":"menghan-wang","given":"Menghan","family":"Wang","affiliation":"eBay Inc. , Shanghai , China"}],"abstract":"News-driven quantitative trading (NQT) has been popularly studied in recent years. Most existing NQT methods are performed in a two-step paradigm, i.e., first analyzing markets by a financial prediction task and then making trading decisions, which is doomed to failure due to the nearly futile financial prediction task. To bypass the financial prediction task, in this paper, we focus on reinforcement learning (RL) based NQT paradigm, which leverages news to make profitable trading decisions directly. In this paper, we propose a novel NQT framework SpotlightTrader based on decision trajectory optimization, which can effectively stitch together a continuous and flexible sequence of trading decisions to maximize profits. In addition, we enhance this framework by constructing a spotlight-driven state trajectory that obeys a stochastic process with irregular abrupt jumps caused by spotlight news. Furthermore, in order to adapt to non-stationary financial markets, we propose an effective training pipeline for this framework, which blends offline pretraining with online finetuning to balance exploration and exploitation effectively during online tradings. Extensive experiments on three real-world datasets demonstrate our proposed model’s superiority over the state-of-the-art NQT methods.","doi":"10.24963/ijcai.2023/548","url":"https://ijcai.org/proceedings/2023/548","pdf":"https://ijcai.org/proceedings/2023/0548.pdf","markdown":"https://ijcai.org/proceedings/2023/548.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/548","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":548,"pages":{"first":4930,"last":4939},"keywords":["Machine Learning: ML: Deep reinforcement learning","Multidisciplinary Topics and Applications: MDA: Finance"],"videos":[],"references":[{"position":1,"raw":"[An et al., 2022] Bo An, Shuo Sun, and Rundong Wang. Deep reinforcement learning for quantitative trading: Challenges and opportunities. IEEE Intelligent Systems, 37(2):23-26, 2022.","title":"Deep reinforcement learning for quantitative trading: Challenges and opportunities","authors":"Bo An, Shuo Sun, Rundong Wang","year":"2022","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ang and Lim, 2022] Gary Ang and Ee-Peng Lim. Guided attention multimodal multitask financial forecasting with inter-company relationships and global and local news. In Proc. of ACL, pages 6313-6326, 2022.","title":"Guided attention multimodal multitask financial forecasting with inter-company relationships and global and local news","authors":"Gary Ang, Ee-Peng Lim","year":"2022","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Araci, 2019] Dogu Araci. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063, 2019.","title":"Finbert: Financial sentiment analysis with pre-trained language models","authors":"Dogu Araci","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Asodekar et al., 2022] Eeshaan Asodekar, Arpan Nookala, Sayali Ayre, and Anant V Nimkar. Deep reinforcement learning for automated stock trading: Inclusion of short selling. In International Symposium on Methodologies for Intelligent Systems, pages 187-197, 2022.","title":"Deep reinforcement learning for automated stock trading: Inclusion of short selling","authors":"Asodekar","year":"2022","venue":"International Symposium on Methodologies for Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen and Huang, 2021] Yu-Fu Chen and Szu-Hao Huang. Sentiment-influenced trading system based on multimodal deep reinforcement learning. Applied Soft Computing, page 107788, 2021.","title":"Sentiment-influenced trading system based on multimodal deep reinforcement learning","authors":"Yu-Fu Chen, Szu-Hao Huang","year":"2021","venue":"Applied Soft Computing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2018] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differ- ential equations. In Proc. of NeurIPS, pages 6572-6583, 2018.","title":"Neural ordinary differential equations","authors":"Yulia Ricky Tq Chen, Jesse Rubanova, David Bettencourt, Duvenaud","year":"2018","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2021] Lili Chen, Kevin Lu, Aravind Ra- jeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence model- ing. In Proc. of NeurIPS, pages 15084-15097, 2021.","title":"Decision transformer: Reinforcement learning via sequence modeling","authors":"Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch","year":"2021","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ding et al., 2014] Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan. Using structured events to predict stock price movement: An empirical investigation. In Proc. of EMNLP, pages 1415-1425, 2014.","title":"Using structured events to predict stock price movement: An empirical investigation","authors":"Xiao Ding, Yue Zhang, Ting Liu, Junwen Duan","year":"2014","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Du and Tanaka-Ishii, 2020] Xin Du and Kumiko Tanaka- Ishii. Stock embeddings acquired from news articles and price history, and an application to portfolio optimization. In Proc. of ACL, pages 3353-3363, 2020.","title":"Stock embeddings acquired from news articles and price history, and an application to portfolio optimization","authors":"Tanaka-Ishii ; Du, Xin Du, Kumiko Tanaka-Ishii","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2018] Ziniu Hu, Weiqing Liu, Jiang Bian, Xu- anzhe Liu, and Tie-Yan Liu. Listening to chaotic whispers: A deep learning framework for news-oriented stock trend prediction. In Proc. of WSDM, pages 261-269, 2018.","title":"Listening to chaotic whispers: A deep learning framework for news-oriented stock trend prediction","authors":"Ziniu Hu, Weiqing Liu, Jiang Bian, Xuanzhe Liu, Tie-Yan Liu","year":"2018","venue":"Proc. of WSDM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2022] Shengchao Hu, Li Shen, Ya Zhang, Yixin Chen, and Dacheng Tao. On transforming reinforce- ment learning by transformer: The development trajectory. arXiv preprint arXiv:2212.14164, pages 1-8, 2022.","title":"On transforming reinforcement learning by transformer: The development trajectory","authors":"Shengchao Hu, Li Shen, Ya Zhang, Yixin Chen, Dacheng Tao","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jiang et al., 2017] Zhengyao Jiang, Dixing Xu, and Jinjun Liang. A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059, pages 1-8, 2017.","title":"A deep reinforcement learning framework for the financial portfolio management problem","authors":"Jiang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kabbani and Duman, 2022] Taylan Kabbani and Ekrem Duman. Deep reinforcement learning approach for trad- ing automation in the stock market. IEEE Access, pages 93564-93574, 2022.","title":"Deep reinforcement learning approach for trading automation in the stock market","authors":"Duman ; ] Taylan Kabbani, Ekrem Kabbani, Duman","year":"2022","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kenton and Toutanova, 2019] Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. In Proceedings of NAACL-HLT, pages 4171-4186, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton, Lee Kristina, Toutanova","year":"2019","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Levine et al., 2020] Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tu- torial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, pages 1-8, 2020.","title":"Offline reinforcement learning: Tutorial, review, and perspectives on open problems","authors":"Sergey Levine, Aviral Kumar, George Tucker, Justin Fu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2020] Yang Liu, Qi Liu, Hongke Zhao, Zhen Pan, and Chuanren Liu. Adaptive quantitative trading: An im- itative deep reinforcement learning approach. In Proc. of AAAI, pages 2128-2135, 2020.","title":"Adaptive quantitative trading: An imitative deep reinforcement learning approach","authors":"Yang Liu, Qi Liu, Hongke Zhao, Zhen Pan, Chuanren Liu","year":"2020","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Martins and Astudillo, 2016] Andre Martins and Ramon Astudillo. From softmax to sparsemax: A sparse model of attention and multi-label classification. In Proc. of ICML, pages 1614-1623, 2016.","title":"From softmax to sparsemax: A sparse model of attention and multi-label classification","authors":"Astudillo Martins, Andre Martins, Ramon Astudillo","year":"2016","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Millea, 2021] Adrian Millea. Deep reinforcement learning for trading-a critical survey. Data, 6(11):119, 2021.","title":"Deep reinforcement learning for trading-a critical survey","authors":"Adrian Millea","year":"2021","venue":"Data","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nan et al., 2022] Abhishek Nan, Anandh Perumal, and Os- mar R Zaiane. Sentiment and knowledge based algorith- mic trading with deep reinforcement learning. In Interna- tional Conference on Database and Expert Systems Appli- cations, pages 167-180, 2022.","title":"Sentiment and knowledge based algorithmic trading with deep reinforcement learning","authors":"Abhishek Nan, Anandh Perumal, Osmar R Zaiane","year":"2022","venue":"International Conference on Database and Expert Systems Applications","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Radford et al., 2019] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):1-24, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI Blog","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sawhney et al., 2021a] Ramit Sawhney, Shivam Agarwal, Megh Thakkar, Arnav Wadhwa, and Rajiv Ratn Shah. Hy- perbolic online time stream modeling. In Proc. of SIGIR, pages 1682-1686, 2021.","title":"Hyperbolic online time stream modeling","authors":"Sawhney","year":"2021","venue":"Proc. of SIGIR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sawhney et al., 2021b] Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, and Rajiv Shah. Fast: Financial news and tweet based time aware network for stock trading. In Proc. of ACL, pages 2164-2175, 2021.","title":"Fast: Financial news and tweet based time aware network for stock trading","authors":"Sawhney","year":"2021","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sawhney et al., 2021c] Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, and Rajiv Shah. Quantitative day trad- ing from natural language using reinforcement learning. In Proc. of ACL, pages 4018-4030, 2021.","title":"Quantitative day trading from natural language using reinforcement learning","authors":"Sawhney","year":"2021","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Schaul et al., 2016] Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience re- play. International Conference on Learning Representa- tions, pages 1-8, 2016.","title":"Prioritized experience replay","authors":"Schaul","year":"2016","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Schumaker and Chen, 2009] Robert P Schumaker and Hsinchun Chen. Textual analysis of stock market predic- tion using breaking financial news: The azfin text system. ACM Transactions on Information Systems (TOIS), pages 1-19, 2009.","title":"Textual analysis of stock market prediction using breaking financial news: The azfin text system","authors":"P Robert, Hsinchun Schumaker, Chen","year":"2009","venue":"ACM Transactions on Information Systems (TOIS)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sun et al., 2021] Shuo Sun, Rundong Wang, and Bo An. Reinforcement learning for quantitative trading. arXiv preprint arXiv:2109.13851, pages 1-8, 2021.","title":"Reinforcement learning for quantitative trading","authors":"Shuo Sun, Rundong Wang, Bo An","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2021] Rundong Wang, Hongxin Wei, Bo An, Zhouyan Feng, and Jun Yao. Commission fee is not enough: A hierarchical reinforced framework for portfo- lio management. In Proc. of AAAI, pages 626-633, 2021.","title":"Commission fee is not enough: A hierarchical reinforced framework for portfolio management","authors":"Wang","year":"2021","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2022] Heyuan Wang, Tengjiao Wang, Shun Li, Shijie Guan, Jiayi Zheng, and Wei Chen. Heterogeneous interactive snapshot network for review-enhanced stock profiling and recommendation. In Proc. of IJCAI, pages 3962-3969, 2022.","title":"Heterogeneous interactive snapshot network for review-enhanced stock profiling and recommendation","authors":"Heyuan Wang, Tengjiao Wang, Shun Li, Shijie Guan, Jiayi Zheng, Wei Chen","year":"2022","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/550"},{"position":29,"raw":"[Wu, 2020] Xianchao Wu. Event-driven learning of system- atic behaviours in stock markets. In Proc. of ACL, pages 2434-2444, 2020.","title":"Event-driven learning of systematic behaviours in stock markets","authors":"Xianchao Wu, Wu","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu and Cohen, 2018] Yumo Xu and Shay B Cohen. Stock movement prediction from tweets and historical prices. In Proc. of ACL, pages 1970-1979, 2018.","title":"Stock movement prediction from tweets and historical prices","authors":"Cohen ; ] Yumo Xu, Shay B Xu, Cohen","year":"2018","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2021] Ke Xu, Yifan Zhang, Deheng Ye, Peilin Zhao, and Mingkui Tan. Relation-aware transformer for portfolio policy learning. In Proc. of IJCAI, pages 4647- 4653, 2021.","title":"Relation-aware transformer for portfolio policy learning","authors":"Ke Xu, Yifan Zhang, Deheng Ye, Peilin Zhao, Mingkui Tan","year":"2021","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/641"},{"position":32,"raw":"[Yang et al., 2020] Hongyang Yang, Xiao-Yang Liu, Shan Zhong, and Anwar Walid. Deep reinforcement learning for automated stock trading: An ensemble strategy. In Pro- ceedings of the First ACM International Conference on AI in Finance, pages 1-8, 2020.","title":"Deep reinforcement learning for automated stock trading: An ensemble strategy","authors":"Hongyang Yang, Xiao-Yang Liu, Shan Zhong, Anwar Walid","year":"2020","venue":"Proceedings of the First ACM International Conference on AI in Finance","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ye et al., 2020] Yunan Ye, Hengzhi Pei, Boxin Wang, Pin- Yu Chen, Yada Zhu, Ju Xiao, and Bo Li. Reinforcement- learning based portfolio management with augmented as- set movement prediction states. In Proc. of AAAI, pages 1112-1119, 2020.","title":"Reinforcementlearning based portfolio management with augmented asset movement prediction states","authors":"Yunan Ye, Hengzhi Pei, Boxin Wang, Pin-Yu Chen, Yada Zhu, Ju Xiao, Bo Li","year":"2020","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2022] Yifan Zhang, Peilin Zhao, Qingyao Wu, Bin Li, Junzhou Huang, and Mingkui Tan. Cost-sensitive portfolio selection via deep reinforcement learning. IEEE Transactions on Knowledge and Data Engineering, pages 236-248, 2022.","title":"Cost-sensitive portfolio selection via deep reinforcement learning","authors":"Yifan Zhang, Peilin Zhao, Qingyao Wu, Bin Li, Junzhou Huang, Mingkui Tan","year":"2022","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zheng et al., 2022] Qinqing Zheng, Amy Zhang, and Aditya Grover. Online decision transformer. In Proc. of ICML, pages 27042-27059, 2022.","title":"Online decision transformer","authors":"Zheng","year":"2022","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhou et al., 2021] Zhihan Zhou, Liqian Ma, and Han Liu. Trade the event: Corporate events detection for news- based event-driven trading. In Proc. of ACL, pages 2114- 2124, 2021.","title":"Trade the event: Corporate events detection for newsbased event-driven trading","authors":"Zhou","year":"2021","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}