{"type":"ScholarlyArticle","title":"PoRank: A Practical Framework for Learning to Rank Policies","authors":[{"name":"Pengjie Gu","slug":"pengjie-gu","given":"Pengjie","family":"Gu","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Mengchen Zhao","slug":"mengchen-zhao","given":"Mengchen","family":"Zhao","affiliation":"School of Software Engineering , South China University of Technology , China"},{"name":"Xu He","slug":"xu-he","given":"Xu","family":"He","affiliation":"Huawei Noah' Ark Lab"},{"name":"Yi Cai","slug":"yi-cai","given":"Yi","family":"Cai","affiliation":"School of Software Engineering , South China University of Technology , China"},{"name":"Bo An","slug":"bo-an","given":"Bo","family":"An","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore; Skywork AI , Singapore"}],"abstract":"In many real-world scenarios, we need to select from a set of candidate policies before online deployment. Although existing Off-policy evaluation (OPE) methods can be used to estimate the online performance, they suffer from high variance. Fortunately, we care only about the ranking of the candidate policies, rather than their exact online rewards. Based on this, we propose a novel framework PoRank for learning to rank policies. In practice, learning to rank policies faces two main challenges: 1) generalization over the huge policy space and 2) lack of supervision signals. To overcome the first challenge, PoRank uses a Policy Comparison Transformer (PCT) for learning cross-policy representations, which capture the core discrepancies between policies and generalizes well across the whole policy space. The second challenge arises because learning to rank requires online comparisons of policies as ground-truth labels, whereas deploying policies online might be highly expensive. To overcome this, PoRank adopts a crowdsourcing based learning-to-rank (LTR) framework, where a set of OPE algorithms are employed to provide weak comparison labels. Experimental results show that PoRank not only outperforms baselines when the ground-truth labels are provided, but also achieves competitive performance when the ground-truth labels are unavailable.","doi":"10.24963/ijcai.2024/447","url":"https://ijcai.org/proceedings/2024/447","pdf":"https://ijcai.org/proceedings/2024/0447.pdf","markdown":"https://ijcai.org/proceedings/2024/447.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/447","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":447,"pages":{"first":4044,"last":4052},"keywords":["Machine Learning: ML: Reinforcement learning"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2023] Tianchi Cai, Jiyan Jiang, Wenpeng Zhang, Shiji Zhou, Xierui Song, Li Yu, Lihong Gu, Xiaodong Zeng, Jinjie Gu, and Guannan Zhang. Marketing bud- get allocation with offline constrained deep reinforcement learning. In Proceedings of the Sixteenth ACM Interna- tional Conference on Web Search and Data Mining, pages 186-194, 2023.","title":"Marketing budget allocation with offline constrained deep reinforcement learning","authors":"Tianchi Cai, Jiyan Jiang, Wenpeng Zhang, Shiji Zhou, Xierui Song, Li Yu, Lihong Gu, Xiaodong Zeng, Jinjie Gu, Guannan Zhang","year":"2023","venue":"Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Deng et al., 2023a] Tianchen Deng, Guole Shen, Tong Qin, Jianyu Wang, Wentao Zhao, Jingchuan Wang, Danwei Wang, and Weidong Chen. Plgslam: Progressive neu- ral scene represenation with local to global bundle adjust- ment. arXiv preprint arXiv:2312.09866, 2023.","title":"Plgslam: Progressive neural scene represenation with local to global bundle adjustment","authors":"Deng","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2023b] Tianchen Deng, Hongle Xie, Jingchuan Wang, and Weidong Chen. Long-term visual simultane- ous localization and mapping: Using a bayesian persis- tence filter-based global map prediction. IEEE Robotics & Automation Magazine, 30(1):36-49, 2023.","title":"Long-term visual simultaneous localization and mapping: Using a bayesian persistence filter-based global map prediction","authors":"Deng","year":"2023","venue":"IEEE Robotics & Automation Magazine","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dosovitskiy et al., 2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Trans- formers for image recognition at scale. In International Conference on Learning Representations, 2021.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Filipe and Pereira, 2018] Rodrigues Filipe and Francisco Pereira. Deep learning from crowds. In Proceedings of the AAAI Conference on Artificial Intelligence, page 1611-1618, 2018.","title":"Deep learning from crowds","authors":"Rodrigues Filipe, Francisco Pereira","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fu et al., 2020] Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4RL: Datasets for deep data-driven reinforcement learning. arXiv, 2020.","title":"D4RL: Datasets for deep data-driven reinforcement learning","authors":"Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, Sergey Levine","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[G. et al., 2017] Bellemare Marc G., Will Dabney, and Rémi Munos. A distributional perspective on reinforcement learning. In International Conference on Machine Learn- ing, pages 449-458, 2017.","title":"A distributional perspective on reinforcement learning","authors":"Bellemare Marc, G, Will Dabney, Rémi Munos","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guan et al., 2018] Melody Guan, Varun Gulshan, Andrew Dai, and Geoffrey Hinton. Who said what: Modeling in- dividual labelers improves classification. In Proceedings of the AAAI Conference on Artificial Intelligence, page 3109-3118, 2018.","title":"Who said what: Modeling individual labelers improves classification","authors":"Melody Guan, Varun Gulshan, Andrew Dai, Geoffrey Hinton","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Haarnoja et al., 2018] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning, pages 1861-1870, 2018.","title":"Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor","authors":"Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, Sergey Levine","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hanna et al., 2019] Josiah Hanna, Niekum Scott, and Peter Stone. Importance sampling policy evaluation with an es- timated behavior policy. In International Conference on Machine Learning, pages 2605-2613, 2019.","title":"Importance sampling policy evaluation with an estimated behavior policy","authors":"Hanna","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Harb et al., 2020] Jean Harb, Tom Schaul, Doina Precup, and Pierre-Luc Bacon. Policy evaluation networks. arXiv, 2020.","title":"Policy evaluation networks","authors":"Jean Harb, Tom Schaul, Doina Precup, Pierre-Luc Bacon","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hoang et al., 2019] Le Hoang, Cameron Voloshin, and Yisong Yue. Batch policy learning under constraints. In International Conference on Machine Learning, pages 3703-3712, 2019.","title":"Batch policy learning under constraints","authors":"Le Hoang, Cameron Voloshin, Yisong Yue","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Isac et al., 2019] Arnekvist Isac, Danica Kragic, and Jo- hannes A. Stork. VPE: Variational policy embedding for transfer reinforcement learning. In ICRA, pages 36-42, 2019.","title":"Arnekvist Isac, Danica Kragic, and Johannes A. Stork. VPE: Variational policy embedding for transfer reinforcement learning","authors":"Isac","year":"2019","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jiang and Li, 2016] Nan Jiang and Lihong Li. Doubly ro- bust off-policy value evaluation for reinforcement learn- ing. In International Conference on Machine Learning, pages 652-661, 2016.","title":"Doubly robust off-policy value evaluation for reinforcement learning","authors":"Nan Jiang, Lihong Li","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jin et al., 2022] Yue Jin, Yue Zhang, Tao Qin, Xudong Zhang, Jian Yuan, Houqiang Li, and Tie-Yan Liu. Super- vised off-policy ranking. In International Conference on Machine Learning, pages 10323-10339, 2022.","title":"Supervised off-policy ranking","authors":"Jin Yue, Yue Zhang, Tao Qin, Xudong Zhang, Jian Yuan, Houqiang Li, Tie-Yan Liu","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jing et al., 2014] Zhang Jing, Xindong Wu, and Victor S. Sheng. Imbalanced multiple noisy labeling. Transac- tions on Knowledge and Data Engineering, 27(2):489- 503, 2014.","title":"Imbalanced multiple noisy labeling","authors":"Zhang Jing, Xindong Wu, Victor S Sheng","year":"2014","venue":"Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jing et al., 2015] Zhang Jing, Victor S. Sheng, Jian Wu, and Xindong Wu. Multi-class ground truth inference in crowd- sourcing with clustering. Transactions on Knowledge and Data Engineering, 28(4):1080-1085, 2015.","title":"Multi-class ground truth inference in crowdsourcing with clustering","authors":"Zhang Jing, Victor S Sheng, Jian Wu, Xindong Wu","year":"2015","venue":"Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kumar et al., 2019] Aviral Kumar, Justin Fu, George Tucker, and Sergey Levine. Stabilizing off-policy Q- Learning via bootstrapping error reduction. In Advances in Neural Information Processing Systems, pages 11761- 11771, 2019.","title":"Stabilizing off-policy Q-Learning via bootstrapping error reduction","authors":"Aviral Kumar, Justin Fu, George Tucker, Sergey Levine","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kumar et al., 2020] Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative Q-Learning for offline reinforcement learning. In Advances in Neural In- formation Processing Systems, pages 1179-1191, 2020.","title":"Conservative Q-Learning for offline reinforcement learning","authors":"Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2015] Lihong Li, Rémi Munos, and Csaba Szepesvári. Toward minimax off-policy value estimation. In Artificial Intelligence and Statistics, pages 608-616, 2015.","title":"Toward minimax off-policy value estimation","authors":"Lihong Li, Rémi Munos, Csaba Szepesvári","year":"2015","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2022] Junbing Li, Changqing Zhang, Joey Tianyi Zhou, Huazhu Fu, Shuyin Xia, and Qinghua Hu. Deep- lift: Deep label-specific feature learning for image anno- tation. IEEE Transactions on Cybernetics, 52(8):7732- 7741, 2022.","title":"Deeplift: Deep label-specific feature learning for image annotation","authors":"Junbing Li, Changqing Zhang, Joey Tianyi Zhou, Huazhu Fu, Shuyin Xia, Qinghua Hu","year":"2022","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lili et al., 2021] Chen Lili, Lu Kevin, Rajeswaran Aravind, Lee Kimin, Grover Aditya, Laskin Michael, Abbeel Pieter, Srinivas Aravind, and Mordatch Igor. Decision trans- former: Reinforcement learning via sequence modeling. In Advances in Neural Information Processing Systems, pages 15084-15097, 2021.","title":"Decision transformer: Reinforcement learning via sequence modeling","authors":"Chen Lili, Lu Kevin, Rajeswaran Aravind, Lee Kimin, Grover Aditya, Laskin Michael, Abbeel Pieter, Srinivas Aravind, Mordatch Igor","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2018] Qiang Liu, Lihong Li, Ziyang Tang, and Dengyong Zhou. Breaking the curse of horizon: Infinite- horizon off-policy estimation. In Advances in Neural In- formation Processing Systems, page 5361-5371, 2018.","title":"Breaking the curse of horizon: Infinitehorizon off-policy estimation","authors":"Qiang Liu, Lihong Li, Ziyang Tang, Dengyong Zhou","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu, 2009] Tie-Yan Liu. Learning to rank for information retrieval. Information Retrieval, 3(3):225-331, 2009.","title":"Learning to rank for information retrieval","authors":"Tie-Yan Liu, Liu","year":"2009","venue":"Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mahmood et al., 2014] A. Rupam Mahmood, Hado P van Hasselt, and Richard S Sutton. Weighted importance sam- pling for off-policy learning with linear function approx- imation. In Advances in Neural Information Processing Systems, 2014.","title":"Weighted importance sampling for off-policy learning with linear function approximation","authors":"A Rupam Mahmood, Richard S Hado P Van Hasselt, Sutton","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Maystre and Grossglauser, 2017] Lucas Maystre and Matthias Grossglauser. Just sort it! a simple and effective approach to active preference learning. In International Conference on Machine Learning, pages 2344-2353.","title":"Just sort it! a simple and effective approach to active preference learning","authors":"Lucas Maystre, Matthias Grossglauser","year":null,"venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"PMLR, 2017.","title":null,"authors":"Pmlr","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Mengjiao et al., 2022] Yang Mengjiao, Bo Dai, Ofir Nachum, George Tucker, and Dale Schuurmans. Offline policy selection under uncertainty. In International Conference on Artificial Intelligence and Statistics, pages 4376-4396, 2022.","title":"Offline policy selection under uncertainty","authors":"Yang Mengjiao, Bo Dai, Ofir Nachum, George Tucker, Dale Schuurmans","year":"2022","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Michael et al., 2021] Janner Michael, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. In Advances in Neural Infor- mation Processing Systems, pages 1273-1286, 2021.","title":"Offline reinforcement learning as one big sequence modeling problem","authors":"Janner Michael, Qiyang Li, Sergey Levine","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Nachum et al., 2019] Ofir Nachum, Yinlam Chow, Bo Dai, and Lihong Li. DualDICE: Behavior-agnostic estimation of discounted stationary distribution corrections. In Ad- vances in Neural Information Processing Systems, 2019.","title":"DualDICE: Behavior-agnostic estimation of discounted stationary distribution corrections","authors":"Ofir Nachum, Yinlam Chow, Bo Dai, Lihong Li","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Paduraru, 2013] Cosmin Paduraru. Off-policy evaluation in Markov decision processes. PhD thesis, McGill University Libraries, 2013.","title":"Off-policy evaluation in Markov decision processes","authors":"Cosmin Paduraru","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Rodrigo et al., 2019] Nogueira Rodrigo, Wei Yang, Kyunghyun Cho, and Jimmy Lin. Multi-stage document ranking with bert. arXiv preprint arXiv:1910.14424, 2019.","title":"Multi-stage document ranking with bert","authors":"Nogueira Rodrigo, Wei Yang, Kyunghyun Cho, Jimmy Lin","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Rodrigues et al., 2013] Filipe Rodrigues, Francisco Pereira, and Bernardete Ribeiro. Learning from multiple annota- tors: Distinguishing good from random labelers. Pattern Recognition Letters, 34(12):1428-1436, 2013.","title":"Learning from multiple annotators: Distinguishing good from random labelers","authors":"Filipe Rodrigues, Francisco Pereira, Bernardete Ribeiro","year":"2013","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[S. and Zhang, 2019] Sheng Victor S. and Jing Zhang. Ma- chine learning with crowdsourcing: A brief summary of the past research and future directions. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 9837-9843, 2019.","title":"Machine learning with crowdsourcing: A brief summary of the past research and future directions","authors":"Sheng Victor, S, Jing Zhang","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Shi et al., 2021] Tianyu Shi, Dong Chen, Kaian Chen, and Zhaojian Li. Offline reinforcement learning for au- tonomous driving with safety and exploration enhance- ment. arXiv preprint arXiv:2110.07067, 2021.","title":"Offline reinforcement learning for autonomous driving with safety and exploration enhancement","authors":"Tianyu Shi, Dong Chen, Kaian Chen, Zhaojian Li","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Tang et al., 2022] Hongyao Tang, Zhaopeng Meng, Jianye Hao, Chen Chen, Daniel Graves, Dong Li, Changmin Yu, Hangyu Mao, Wulong Liu, Yaodong Yang, Wenyuan Tao, and Li Wang. What about inputting policy in value function: Policy representation and policy-extended value function approximator. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 8441-8449, 2022.","title":"What about inputting policy in value function: Policy representation and policy-extended value function approximator","authors":"Hongyao Tang, Zhaopeng Meng, Jianye Hao, Chen Chen, Daniel Graves, Dong Li, Changmin Yu, Hangyu Mao, Wulong Liu, Yaodong Yang, Wenyuan Tao, Li Wang","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Tian and Zhu, 2015] Tian Tian and Jun Zhu. Max-margin majority voting for learning from crowds. In Advances in Neural Information Processing Systems, page 1621-1629, 2015.","title":"Max-margin majority voting for learning from crowds","authors":"Zhu ; ] Tian Tian, Jun Tian, Zhu","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang et al., 2020] Ziyu Wang, Alexander Novikov, Konrad Zolna, Josh S Merel, Jost Tobias Springenberg, Scott E Reed, Bobak Shahriari, Noah Siegel, Caglar Gulcehre, Nicolas Heess, and Nando de Freitas. Critic regularized regression. In Advances in Neural Information Processing Systems, pages 7768-7778, 2020.","title":"Critic regularized regression","authors":"Ziyu Wang, Alexander Novikov, Konrad Zolna, Josh S Merel, Jost Tobias Springenberg, Scott E Reed, Bobak Shahriari, Noah Siegel, Caglar Gulcehre, Nicolas Heess, Nando De Freitas","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yang et al., 2023] Chan-Yun Yang, Chamani Shiranthika, Chung-Yih Wang, Kuo-Wei Chen, and Sagara Sumath- ipala. Reinforcement learning strategies in cancer chemotherapy treatments: A review. Computer Methods and Programs in Biomedicine, 229:107280, 2023.","title":"Reinforcement learning strategies in cancer chemotherapy treatments: A review","authors":"Chan-Yun Yang, Chamani Shiranthika, Chung-Yih Wang, Kuo-Wei Chen, Sagara Sumathipala","year":"2023","venue":"Computer Methods and Programs in Biomedicine","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yuchen et al., 2016] Zhang Yuchen, Xi Chen, Dengyong Zhou, and Michael I. Jordan. Spectral methods meet em: A provably optimal algorithm for crowdsourcing. JMLR, 17(1):3537-3580, 2016.","title":"Spectral methods meet em: A provably optimal algorithm for crowdsourcing","authors":"Xi Zhang Yuchen, Dengyong Chen, Michael I Zhou, Jordan","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhang and Jiang, 2021] Siyuan Zhang and Nan Jiang. To- wards hyperparameter-free policy selection for offline re- inforcement learning. In Advances in Neural Information Processing Systems, 2021.","title":"Towards hyperparameter-free policy selection for offline reinforcement learning","authors":"Jiang ; ] Siyuan Zhang, Nan Zhang, Jiang","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2020] Zihao Zhang, Stefan Zohren, and Roberts Stephen. Deep reinforcement learning for trading. The Journal of Financial Data Science, 2020.","title":"Deep reinforcement learning for trading","authors":"Zhang","year":"2020","venue":"The Journal of Financial Data Science","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhu et al., 2021] Chen Zhu, Wei Ping, Chaowei Xiao, Mo- hammad Shoeybi, Tom Goldstein, Anima Anandkumar, and Bryan Catanzaro. Long-short transformer: Efficient transformers for language and vision. In Advances in Neu- ral Information Processing Systems, pages 17723-17736, 2021.","title":"Long-short transformer: Efficient transformers for language and vision","authors":"Chen Zhu, Wei Ping, Chaowei Xiao, Mohammad Shoeybi, Tom Goldstein, Anima Anandkumar, Bryan Catanzaro","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}