{"type":"ScholarlyArticle","title":"Deep Hashing-based Dynamic Stock Correlation Estimation via Normalizing Flow","authors":[{"name":"Xiaolin Zheng","slug":"xiaolin-zheng","given":"Xiaolin","family":"Zheng","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"Mengpu Liu","slug":"mengpu-liu","given":"Mengpu","family":"Liu","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"Mengying Zhu","slug":"mengying-zhu","given":"Mengying","family":"Zhu","affiliation":"Zhejiang University , Hangzhou , China"}],"abstract":"In financial scenarios, influenced by common factors such as global macroeconomic and sector-specific factors, stocks exhibit varying degrees of correlations with each other, which is essential in risk-averse portfolio allocation. Because the real risk matrix is unobservable, the covariance-based correlation matrix is widely used for constructing diversified stock portfolios. However, studies have seldom focused on dynamic correlation matrix estimation under the non-stationary financial market. Moreover, as the number of stocks in the market grows, existing correlation matrix estimation methods face more serious challenges with regard to efficiency and effectiveness. In this paper, we propose a novel hash-based dynamic correlation forecasting model (HDCF) to estimate dynamic stock correlations. Under structural assumptions on the correlation matrix, HDCF learns the hash representation based on normalizing flows instead of the real-valued representation, which performs extremely efficiently in high-dimensional settings. Experiments show that our proposed model outperforms baselines on portfolio decisions in terms of effectiveness and efficiency.","doi":"10.24963/ijcai.2023/555","url":"https://ijcai.org/proceedings/2023/555","pdf":"https://ijcai.org/proceedings/2023/0555.pdf","markdown":"https://ijcai.org/proceedings/2023/555.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/555","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":555,"pages":{"first":4993,"last":5001},"keywords":["Multidisciplinary Topics and Applications: MDA: Finance","Machine Learning: ML: Representation learning"],"videos":[],"references":[{"position":1,"raw":"[Bickel and Levina, 2008] Peter J Bickel and Elizaveta Lev- ina. Covariance regularization by thresholding. The An- nals of statistics, 36(6):2577-2604, 2008.","title":"Covariance regularization by thresholding","authors":"J Peter, Elizaveta Bickel, Levina","year":"2008","venue":"The Annals of statistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bollerslev et al., 1988] Tim Bollerslev, Robert F. Engle, and Jeffrey M. Wooldridge. A capital asset pricing model with time-varying covariances. Journal of Political Economy, 96:116 -131, 1988.","title":"A capital asset pricing model with time-varying covariances","authors":"Tim Bollerslev, Robert F Engle, Jeffrey M Wooldridge","year":"1988","venue":"Journal of Political Economy","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2017] Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. Deep visual-semantic quantiza- tion for efficient image retrieval. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 916-925, 2017.","title":"Deep visual-semantic quantization for efficient image retrieval","authors":"Yue Cao, Mingsheng Long, Jianmin Wang, Shichen Liu","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chandrasekaran et al., 2010] Venkat Chandrasekaran, Pablo A. Parrilo, and Alan S. Willsky. Latent variable graphical model selection via convex optimization. 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1610-1613, 2010.","title":"Latent variable graphical model selection via convex optimization","authors":"Venkat Chandrasekaran, Pablo A Parrilo, Alan S Willsky","year":"2010","venue":"48th Annual Allerton Conference on Communication, Control, and Computing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2013] Xiaohui Chen, Mengyu Xu, and Wei Biao Wu. Covariance and precision matrix estimation for high-dimensional time series. Annals of Statistics, 41:2994-3021, 2013.","title":"Covariance and precision matrix estimation for high-dimensional time series","authors":"Xiaohui Chen, Mengyu Xu, Wei Biao Wu","year":"2013","venue":"Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2019] Jia Chen, Degui Li, and Oliver Lin- ton. A new semiparametric estimation approach for large dynamic covariance matrices with multiple conditioning variables. Journal of Econometrics, 212(1):155-176, 2019. Big Data in Dynamic Predictive Econometric Mod- eling.","title":"A new semiparametric estimation approach for large dynamic covariance matrices with multiple conditioning variables","authors":"Jia Chen, Degui Li, Oliver Linton","year":"2019","venue":"Big Data in Dynamic Predictive Econometric Modeling","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Elton and Gruber, 1973] Edwin J. Elton and Martin J. Gru- ber. Estimating the dependence structure of share prices -implications for portfolio selection. Journal of Finance, 28:1203-1232, 1973.","title":"Estimating the dependence structure of share prices -implications for portfolio selection","authors":"Edwin J Elton, Martin J Gruber","year":"1973","venue":"Journal of Finance","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Elton et al., 1978] Edwin J. Elton, Martin J. Gruber, and Thomas J. Urich. Are betas best. Journal of Finance, 33:1375-1384, 1978.","title":"Are betas best","authors":"Edwin J Elton, Martin J Gruber, Thomas J Urich","year":"1978","venue":"Journal of Finance","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Engle et al., 2017] Robert F. Engle, Olivier Ledoit, and Michael Wolf. Large dynamic covariance matrices. Jour- nal of Business & Economic Statistics, 37:363 -375, 2017.","title":"Large dynamic covariance matrices","authors":"Robert F Engle, Olivier Ledoit, Michael Wolf","year":"2017","venue":"Journal of Business & Economic Statistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Engle, 2002] Robert Engle. Dynamic conditional correla- tion: A simple class of multivariate generalized autore- gressive conditional heteroskedasticity models. Journal of Business & Economic Statistics, 20(3):339-350, 2002.","title":"Dynamic conditional correlation: A simple class of multivariate generalized autoregressive conditional heteroskedasticity models","authors":"Robert Engle","year":"2002","venue":"Journal of Business & Economic Statistics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Engle, 2009] Robert Engle. Anticipating Correlations: A New Paradigm for Risk Management. Princeton University Press, 2009.","title":"Anticipating Correlations: A New Paradigm for Risk Management","authors":"Robert Engle","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fama and French, 1989] Eugene F. Fama and Kenneth R. French. Business conditions and expected returns on stocks and bonds. Journal of Financial Economics, 25:23- 49, 1989.","title":"Business conditions and expected returns on stocks and bonds","authors":"Eugene F Fama, Kenneth R French","year":"1989","venue":"Journal of Financial Economics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Fama and French, 1993] Eugene F. Fama and Kenneth R. French. Common risk factors in the returns on stocks and bonds. Journal of Financial Economics, 33:3-56, 1993.","title":"Common risk factors in the returns on stocks and bonds","authors":"Eugene F Fama, Kenneth R French","year":"1993","venue":"Journal of Financial Economics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Fan et al., 2007] Jianqing Fan, Yingying Fan, and Jinchi Lv. High dimensional covariance matrix estimation using a factor model. Journal of Econometrics, 147:186-197, 2007.","title":"High dimensional covariance matrix estimation using a factor model","authors":"Jianqing Fan, Yingying Fan, Jinchi Lv","year":"2007","venue":"Journal of Econometrics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Fan et al., 2013a] Jianqing Fan, Fang Han, and Han Liu. Challenges of big data analysis. National science review, 1 2:293-314, 2013.","title":"Challenges of big data analysis","authors":"Fan","year":"2013","venue":"National science review","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Fan et al., 2013b] Jianqing Fan, Yuan Liao, and Martina Mincheva. Large Covariance Estimation by Threshold- ing Principal Orthogonal Complements. Journal of the Royal Statistical Society Series B: Statistical Methodol- ogy, 75(4):603-680, 08 2013.","title":"Large Covariance Estimation by Thresholding Principal Orthogonal Complements","authors":"Fan","year":"2013","venue":"Journal of the Royal Statistical Society Series B: Statistical Methodology","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Friedman et al., 2008] Jerome H. Friedman, Trevor J. Hastie, and Robert Tibshirani. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9 3:432- 41, 2008.","title":"Sparse inverse covariance estimation with the graphical lasso","authors":"Jerome H Friedman, Trevor J Hastie, Robert Tibshirani","year":"2008","venue":"Biostatistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Gong et al., 2013] Yunchao Gong, Svetlana Lazebnik, Al- bert Gordo, and Florent Perronnin. Iterative quantization: A procrustean approach to learning binary codes for large- scale image retrieval. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 35:2916-2929, 2013.","title":"Iterative quantization: A procrustean approach to learning binary codes for largescale image retrieval","authors":"Gong","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jiang and Li, 2016] Qing-Yuan Jiang and Wu-Jun Li. Deep cross-modal hashing. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3270- 3278, 2016.","title":"Deep cross-modal hashing","authors":"Li ; Qing-Yuan Jiang, Wu-Jun Jiang, Li","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Jordan et al., 1999] Michael I. Jordan, Zoubin Ghahramani, T. Jaakkola, and Lawrence K. Saul. An introduction to variational methods for graphical models. Machine Learn- ing, 37:183-233, 1999.","title":"An introduction to variational methods for graphical models","authors":"I Michael, Zoubin Jordan, T Ghahramani, Lawrence K Jaakkola, Saul","year":"1999","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ke et al., 2020] Yuan Ke, Heng Lian, and Wenyang Zhang. High-dimensional dynamic covariance matrices with ho- mogeneous structure. Journal of Business & Economic Statistics, 40:96 -110, 2020.","title":"High-dimensional dynamic covariance matrices with homogeneous structure","authors":"Yuan Ke, Heng Lian, Wenyang Zhang","year":"2020","venue":"Journal of Business & Economic Statistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980:1-15, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lan et al., 2017] Shiwei Lan, Andrew J. Holbrook, Gabriel A. Elias, Norbert J. Fortin, Hernando C. Ombao, and Babak Shahbaba. Flexible bayesian dynamic mod- eling of correlation and covariance matrices. Bayesian analysis, 15 4:1199-1228, 2017.","title":"Flexible bayesian dynamic modeling of correlation and covariance matrices","authors":"Shiwei Lan, Andrew J Holbrook, Gabriel A Elias, Norbert J Fortin, Hernando C Ombao, Babak Shahbaba","year":"2017","venue":"Bayesian analysis","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liang et al., 2021] Qianqiao Liang, Mengying Zhu, Xi- aolin Zheng, and Yan Wang. An adaptive news-driven method for cvar-sensitive online portfolio selection in non- stationary financial markets. In Zhi-Hua Zhou, editor, Pro- ceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 2708-2715. Inter- national Joint Conferences on Artificial Intelligence Orga- nization, 8 2021. Main Track.","title":"An adaptive news-driven method for cvar-sensitive online portfolio selection in nonstationary financial markets","authors":"Qianqiao Liang, Mengying Zhu, Xiaolin Zheng, Yan Wang","year":"2021","venue":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liu et al., 2017] X. Liu, Lili Zhao, Dajun Ding, and Yajiao Dong. Deep hashing with category mask for fast video retrieval. ArXiv, abs/1712.08315, 2017.","title":"Deep hashing with category mask for fast video retrieval","authors":"Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Markowitz, 1952] Harry Markowitz. Harry m. markowitz. Portfolio selection, Journal of Finance, 7(1):77-91, 1952.","title":"Harry m. markowitz. Portfolio selection","authors":"Harry Markowitz","year":"1952","venue":"Journal of Finance","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Rangel and Engle, 2012] José Gonzalo Rangel and Robert F Engle. The factor-spline-garch model for high and low frequency correlations. Journal of Business & Economic Statistics, 30(1):109-124, 2012.","title":"José Gonzalo Rangel and Robert F Engle. The factor-spline-garch model for high and low frequency correlations","authors":"Engle Rangel","year":"2012","venue":"Journal of Business & Economic Statistics","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sak et al., 2014] Hasim Sak, Andrew W. Senior, and Franc ¸oise Beaufays. Long short-term memory based re- current neural network architectures for large vocabulary speech recognition. ArXiv, abs/1402.1128:1-5, 2014.","title":"Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition","authors":"Hasim Sak, Andrew W Senior, Franc, Beaufays","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Shen and Wang, 2017] Weiwei Shen and Jun Wang. Portfo- lio selection via subset resampling. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI'17, page 1517-1523. AAAI Press, 2017.","title":"Portfolio selection via subset resampling","authors":"Wang ; Shen, Weiwei Shen, Jun Wang","year":"2017","venue":"Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI'17","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xia et al., 2014] Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. Supervised hashing for image re- trieval via image representation learning. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intel- ligence, AAAI'14, page 2156-2162. AAAI Press, 2014.","title":"Supervised hashing for image retrieval via image representation learning","authors":"Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, Shuicheng Yan","year":"2014","venue":"Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, AAAI'14","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2017] Erkun Yang, Cheng Deng, Wei Liu, Xi- anglong Liu, Dacheng Tao, and Xinbo Gao. Pairwise re- lationship guided deep hashing for cross-modal retrieval. AAAI'17, page 1618-1625. AAAI Press, 2017.","title":"Pairwise relationship guided deep hashing for cross-modal retrieval","authors":"Erkun Yang, Cheng Deng, Wei Liu, Xianglong Liu, Dacheng Tao, Xinbo Gao","year":"2017","venue":"AAAI'17","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yuan et al., 2019] Li Yuan, Tao Wang, Xiaopeng Zhang, Francis E. H. Tay, Zequn Jie, Wei Liu, and Jiashi Feng. Central similarity quantization for efficient image and video retrieval. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3080- 3089, 2019.","title":"Central similarity quantization for efficient image and video retrieval","authors":"Li Yuan, Tao Wang, Xiaopeng Zhang, Francis E H Tay, Zequn Jie, Wei Liu, Jiashi Feng","year":"2019","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhu et al., 2016] Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. Deep hashing network for efficient similarity retrieval. AAAI'16, page 2415-2421. AAAI Press, 2016.","title":"Deep hashing network for efficient similarity retrieval","authors":"Han Zhu, Mingsheng Long, Jianmin Wang, Yue Cao","year":"2016","venue":"AAAI'16","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhu et al., 2020] Mengying Zhu, Xiaolin Zheng, Yan Wang, Qianqiao Liang, and Wenfang Zhang. Online portfolio selection with cardinality constraint and trans- action costs based on contextual bandit. In Christian Bessiere, editor, Proceedings of the Twenty-Ninth Interna- tional Joint Conference on Artificial Intelligence, IJCAI- 20, pages 4682-4689. International Joint Conferences on Artificial Intelligence Organization, 7 2020. Special Track on AI in FinTech.","title":"Online portfolio selection with cardinality constraint and transaction costs based on contextual bandit","authors":"Mengying Zhu, Xiaolin Zheng, Yan Wang, Qianqiao Liang, Wenfang Zhang","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20","doi":null,"url":null,"internal_url":"/proceedings/2020/646"},{"position":35,"raw":"[Zhu et al., 2022] Haoren Zhu, Shih-Yang Liu, Pengfei Zhao, Yingying Chen, and Dik Lun Lee. Forecasting asset dependencies to reduce portfolio risk. 36:4397-4404, Jun. 2022.","title":"Forecasting asset dependencies to reduce portfolio risk","authors":"Haoren Zhu, Shih-Yang Liu, Pengfei Zhao, Yingying Chen, Dik Lun, Lee","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}