{"type":"ScholarlyArticle","title":"A Two-level Reinforcement Learning Algorithm for Ambiguous Mean-variance Portfolio Selection Problem","authors":[{"name":"Xin Huang","slug":"xin-huang","given":"Xin","family":"Huang","affiliation":"Department of Systems Engineering and Engineering Management , The Chinese University of Hong Kong , Hong Kong"},{"name":"Duan Li","slug":"duan-li","given":"Duan","family":"Li","affiliation":"School of Data Science , City University of Hong Kong , Hong Kong"}],"abstract":"Traditional modeling on the mean-variance portfolio selection often assumes a full knowledge on statistics of assets' returns. It is, however, not always the case in real financial markets. This paper deals with an ambiguous mean-variance portfolio selection problem with a mixture model on the returns of risky assets, where the proportions of different component distributions are assumed to be unknown to the investor, but being constants (in any time instant). Taking into consideration the updates of proportions from future observations is essential to find an optimal policy with active learning feature, but makes the problem intractable when we adopt the classical methods. Using reinforcement learning, we derive an investment policy with a learning feature in a two-level framework. In the lower level, the time-decomposed approach (dynamic programming) is adopted to solve a family of scenario subcases where in each case the series of component distributions along multiple time periods is specified. At the upper level, a scenario-decomposed approach (progressive hedging algorithm) is applied in order to iteratively aggregate the scenario solutions from the lower layer based on the current knowledge on proportions, and this two-level solution framework is repeated in a manner of rolling horizon. We carry out experimental studies to illustrate the execution of our policy scheme.","doi":"10.24963/ijcai.2020/624","url":"https://ijcai.org/proceedings/2020/624","pdf":"https://ijcai.org/proceedings/2020/0624.pdf","markdown":"https://ijcai.org/proceedings/2020/624.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/624","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Special Track on AI in FinTech","track":"Special Track on AI in FinTech","number":624,"pages":{"first":4527,"last":4533},"keywords":["AI for trading: AI for portfolio analytics"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24478/file_24478.m3u8","length_sec":297,"provider":"hls"}],"references":[{"position":1,"raw":"[Bertsekas, 2019] Dimitri P. Bertsekas. Reinforcement Learning and Optimal Control. Athena Scientific, 2019.","title":"Reinforcement Learning and Optimal Control","authors":"Dimitri P Bertsekas","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cui et al., 2014] Xiangyu Cui, Jianjun Gao, Xun Li, and Duan Li. Optimal multi-period mean-variance policy un- der no-shorting constraint. European Journal of Opera- tional Research, 234(2):459-468, 2014.","title":"Optimal multi-period mean-variance policy under no-shorting constraint","authors":"Xiangyu Cui, Jianjun Gao, Xun Li, Duan Li","year":"2014","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Li and Ng, 2000] Duan Li and Wan-Lung Ng. Optimal dy- namic portfolio selection: Multiperiod mean-variance for- mulation. Mathematical Finance, 10(3):387-406, 2000.","title":"Optimal dynamic portfolio selection: Multiperiod mean-variance formulation","authors":"Duan Li, Wan-Lung Ng","year":"2000","venue":"Mathematical Finance","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Li et al., 1998] Duan Li, Tsz-Fung Chan, and Wan-Lung Ng. Safety-first dynamic portfolio selection. Dynamics of Continuous, Discrete and Impulsive Systems Series B: Applications and Algorithms, 4(4):585-600, 12 1998.","title":"Safety-first dynamic portfolio selection","authors":"Duan Li, Tsz-Fung Chan, Wan-Lung Ng","year":"1998","venue":"Dynamics of Continuous, Discrete and Impulsive Systems Series B: Applications and Algorithms","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Markowitz, 1952] Harry Markowitz. Portfolio selection. The Journal of Finance, 7(1):77-91, 1952.","title":"Portfolio selection","authors":"Markowitz","year":"1952","venue":"The Journal of Finance","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[McLachlan and Peel, 2004] Geoffrey J McLachlan and David Peel. Finite Mixture Models. John Wiley & Sons, 2004.","title":"Finite Mixture Models","authors":"J Geoffrey, David Mclachlan, Peel","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Rockafellar and Wets, 1991] R Tyrrell Rockafellar and Roger J-B Wets. Scenarios and policy aggregation in op- timization under uncertainty. Mathematics of Operations Research, 16(1):119-147, 1991.","title":"Scenarios and policy aggregation in optimization under uncertainty","authors":"Rockafellar Tyrrell, J-B Roger, Wets","year":"1991","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Strub et al., 2019] Moris S Strub, Duan Li, Xiangyu Cui, and Jianjun Gao. Discrete-time mean-CVaR portfolio selection and time-consistency induced term structure of the CVaR. Journal of Economic Dynamics and Control, 108:103751, 2019.","title":"Discrete-time mean-CVaR portfolio selection and time-consistency induced term structure of the CVaR","authors":"Duan Moris S Strub, Xiangyu Li, Jianjun Cui, Gao","year":"2019","venue":"Journal of Economic Dynamics and Control","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement Learning: An Introduction. MIT Press, 2018.","title":"Reinforcement Learning: An Introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Zhu et al., 2014] Shushang Zhu, Minjie Fan, and Duan Li. Portfolio management with robustness in both prediction and decision: A mixture model based learning approach. Journal of Economic Dynamics and Control, 48:1-25, 2014.","title":"Portfolio management with robustness in both prediction and decision: A mixture model based learning approach","authors":"Zhu","year":"2014","venue":"Journal of Economic Dynamics and Control","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}