{"type":"ScholarlyArticle","title":"X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learner","authors":[{"name":"Haoyuan Jiang","slug":"haoyuan-jiang","given":"Haoyuan","family":"Jiang","affiliation":"Baidu Inc. , China"},{"name":"Ziyue Li","slug":"ziyue-li","given":"Ziyue","family":"Li","affiliation":"University of Cologne , Germany"},{"name":"Hua Wei","slug":"hua-wei","given":"Hua","family":"Wei","affiliation":"Arizona State University , U.S.A"},{"name":"Xuantang Xiong","slug":"xuantang-xiong","given":"Xuantang","family":"Xiong","affiliation":"Institute of Automation , Chinese Academy of Sciences , China"},{"name":"Jingqing Ruan","slug":"jingqing-ruan","given":"Jingqing","family":"Ruan","affiliation":"Institute of Automation , Chinese Academy of Sciences , China"},{"name":"Jiaming Lu","slug":"jiaming-lu","given":"Jiaming","family":"Lu","affiliation":"Fudan University , China"},{"name":"Hangyu Mao","slug":"hangyu-mao","given":"Hangyu","family":"Mao","affiliation":"SenseTime Research , China"},{"name":"Rui Zhao","slug":"rui-zhao","given":"Rui","family":"Zhao","affiliation":"SenseTime Research , China"}],"abstract":"The effectiveness of traffic light control has been significantly improved by current reinforcement learning-based approaches via better cooperation among multiple traffic lights. However, a persisting issue remains: how to obtain a multi-agent traffic signal control algorithm with remarkable transferability across diverse cities? In this paper, we propose a Transformer on Transformer (TonT) model for cross-city meta multi-agent traffic signal control, named as X-Light: We input the full Markov Decision Process trajectories, and the Lower Transformer aggregates the states, actions, rewards among the target intersection and its neighbors within a city, and the Upper Transformer learns the general decision trajectories across different cities. This dual-level approach bolsters the model's robust generalization and transferability. Notably, when directly transferring to unseen scenarios, ours surpasses all baseline methods with +7.91% on average, and even +16.3% in some cases, yielding the best results.","doi":"10.24963/ijcai.2024/11","url":"https://ijcai.org/proceedings/2024/11","pdf":"https://ijcai.org/proceedings/2024/0011.pdf","markdown":"https://ijcai.org/proceedings/2024/11.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/11","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":11,"pages":{"first":94,"last":102},"keywords":["Machine Learning: ML: Meta-learning","Agent-based and Multi-agent Systems: MAS: Applications","Machine Learning: ML: Multi-task and transfer learning","Multidisciplinary Topics and Applications: MTA: Transportation"],"videos":[],"references":[{"position":1,"raw":"[Ault and Sharon, 2021] James Ault and Guni Sharon. Rein- forcement learning benchmarks for traffic signal control. In Thirty-fifth Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track (Round 1), 2021.","title":"Reinforcement learning benchmarks for traffic signal control","authors":"Sharon ; Ault, James Ault, Guni Sharon","year":"2021","venue":"Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhenhui Li. Toward a thousand lights: Decentralized deep rein- forcement learning for large-scale traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 34, pages 3414-3421, 2020.","title":"Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control","authors":"Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, Zhenhui Li","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021] Lili Chen, Kevin Lu, Aravind Ra- jeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence model- ing. Advances in neural information processing systems, 34:15084-15097, 2021.","title":"Decision transformer: Reinforcement learning via sequence modeling","authors":"Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch","year":"2021","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chu et al., 2019] Tianshu Chu, Jie Wang, Lara Codecà, and Zhaojian Li. Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Transactions on In- telligent Transportation Systems, 21(3):1086-1095, 2019.","title":"Multi-agent deep reinforcement learning for large-scale traffic signal control","authors":"Tianshu Chu, Jie Wang, Lara Codecà, Zhaojian Li","year":"2019","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Du et al., 2024] Xinqi Du, Ziyue Li, Cheng Long, Yongheng Xing, S Yu Philip, and Hechang Chen. Felight: Fairness-aware traffic signal control via sample-efficient reinforcement learning. IEEE Transactions on Knowledge and Data Engineering, 2024.","title":"Felight: Fairness-aware traffic signal control via sample-efficient reinforcement learning","authors":"Du","year":"2024","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duan et al., 2017] Yan Duan, John Schulman, Xi Chen, Pe- ter L. Bartlett, Ilya Sutskever, and Pieter Abbeel. RLˆ2: Fast reinforcement learning via slow reinforcement learn- ing, 2017.","title":"RLˆ2: Fast reinforcement learning via slow reinforcement learning","authors":"Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, Pieter Abbeel","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. Advances in neural information processing systems, 27, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guo et al., 2014] Haifeng Guo, Jun Cheng, Qitao Peng, Chao Zhu, and Yuanjie Mu. Dynamic division of traffic control sub-area methods based on the similarity of adja- cent intersections. In 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), pages 2208- 2213. IEEE, 2014.","title":"Dynamic division of traffic control sub-area methods based on the similarity of adjacent intersections","authors":"Haifeng Guo, Jun Cheng, Qitao Peng, Chao Zhu, Yuanjie Mu","year":"2014","venue":"17th International IEEE Conference on Intelligent Transportation Systems (ITSC)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2021] Xin Guo, Zhengxu Yu, Pengfei Wang, Zhongming Jin, Jianqiang Huang, Deng Cai, Xiaofei He, and Xiansheng Hua. Urban traffic light control via ac- tive multi-agent communication and supply-demand mod- eling. IEEE Transactions on Knowledge and Data Engi- neering, 2021.","title":"Urban traffic light control via active multi-agent communication and supply-demand modeling","authors":"Xin Guo, Zhengxu Yu, Pengfei Wang, Zhongming Jin, Jianqiang Huang, Deng Cai, Xiaofei He, Xiansheng Hua","year":"2021","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing sys- tems, 30, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Han et al., 2021] Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, and Yunhe Wang. Transformer in transformer. Advances in Neural Information Processing Systems, 34:15908-15919, 2021.","title":"Transformer in transformer","authors":"Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, Yunhe Wang","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Humplik et al., 2019] Jan Humplik, Alexandre Galashov, Leonard Hasenclever, Pedro A Ortega, Yee Whye Teh, and Nicolas Heess. Meta reinforcement learning as task infer- ence. arXiv preprint arXiv:1905.06424, 2019.","title":"Meta reinforcement learning as task inference","authors":"Humplik","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hunt et al., 1982] PB Hunt, DI Robertson, RD Bretherton, and M Cr Royle. The SCOOT on-line traffic signal opti- misation technique. Traffic Engineering & Control, 23(4), 1982.","title":"The SCOOT on-line traffic signal optimisation technique","authors":"Hunt","year":"1982","venue":"Traffic Engineering & Control","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Janner et al., 2021] Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big se- quence modeling problem. Advances in neural informa- tion processing systems, 34:1273-1286, 2021.","title":"Offline reinforcement learning as one big sequence modeling problem","authors":"Michael Janner, Qiyang Li, Sergey Levine","year":"2021","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jiang et al., 2024] Haoyuan Jiang, Ziyue Li, Zhishuai Li, Lei Bai, Hangyu Mao, Wolfgang Ketter, and Rui Zhao. A general scenario-agnostic reinforcement learning for traf- fic signal control. IEEE Transactions on Intelligent Trans- portation Systems, 2024.","title":"A general scenario-agnostic reinforcement learning for traffic signal control","authors":"Haoyuan Jiang, Ziyue Li, Zhishuai Li, Lei Bai, Hangyu Mao, Wolfgang Ketter, Rui Zhao","year":"2024","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kouvelas et al., 2014] Anastasios Kouvelas, Jennie Lioris, S Alireza Fayazi, and Pravin Varaiya. Maximum pres- sure controller for stabilizing queues in signalized arterial networks. Transportation Research Record, 2421(1):133- 141, 2014.","title":"Maximum pressure controller for stabilizing queues in signalized arterial networks","authors":"Anastasios Kouvelas, Jennie Lioris, S Alireza Fayazi, Pravin Varaiya","year":"2014","venue":"Transportation Research Record","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liang et al., 2019] Xiaoyuan Liang, Xunsheng Du, Guiling Wang, and Zhu Han. A deep reinforcement learning net- work for traffic light cycle control. IEEE Transactions on Vehicular Technology, 68(2):1243-1253, 2019.","title":"A deep reinforcement learning network for traffic light cycle control","authors":"Xiaoyuan Liang, Xunsheng Du, Guiling Wang, Zhu Han","year":"2019","venue":"IEEE Transactions on Vehicular Technology","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liang et al., 2022] Yuxuan Liang, Pan Zhou, Roger Zim- mermann, and Shuicheng Yan. Dualformer: Local-global stratified transformer for efficient video recognition. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Gio- vanni Maria Farinella, and Tal Hassner, editors, Com- puter Vision -ECCV 2022, pages 577-595, Cham, 2022. Springer Nature Switzerland.","title":"Dualformer: Local-global stratified transformer for efficient video recognition","authors":"Yuxuan Liang, Pan Zhou, Roger Zimmermann, Shuicheng Yan","year":"2022","venue":"Computer Vision -ECCV 2022","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lou et al., 2022] Yican Lou, Jia Wu, and Yunchuan Ran. Meta-reinforcement learning for multiple traffic signals control. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 4264-4268, 2022.","title":"Meta-reinforcement learning for multiple traffic signals control","authors":"Lou","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lowrie, 1990] PR Lowrie. SCATS, Sydney co-ordinated adaptive traffic system: A traffic responsive method of controlling urban traffic. 1990.","title":"SCATS, Sydney co-ordinated adaptive traffic system: A traffic responsive method of controlling urban traffic","authors":"; Pr Lowrie, Lowrie","year":"1990","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lu et al., 2023] Jiaming Lu, Jingqing Ruan, Haoyuan Jiang, Ziyue Li, Hangyu Mao, and Rui Zhao. Dualight: En- hancing traffic signal control by leveraging scenario- specific and scenario-shared knowledge. arXiv preprint arXiv:2312.14532, 2023.","title":"Dualight: Enhancing traffic signal control by leveraging scenariospecific and scenario-shared knowledge","authors":"Jiaming Lu, Jingqing Ruan, Haoyuan Jiang, Ziyue Li, Hangyu Mao, Rui Zhao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mao et al., 2023] Hangyu Mao, Rui Zhao, Ziyue Li, Zhi- wei Xu, Hao Chen, Yiqun Chen, Bin Zhang, Zhen Xiao, Junge Zhang, and Jiangjin Yin. Pdit: Interleaving percep- tion and decision-making transformers for deep reinforce- ment learning. arXiv preprint arXiv:2312.15863, 2023.","title":"Pdit: Interleaving perception and decision-making transformers for deep reinforcement learning","authors":"Hangyu Mao, Rui Zhao, Ziyue Li, Zhiwei Xu, Hao Chen, Yiqun Chen, Bin Zhang, Zhen Xiao, Junge Zhang, Jiangjin Yin","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Melo, 2022] Luckeciano C Melo. Transformers are meta- reinforcement learners. In international conference on ma- chine learning, pages 15340-15359. PMLR, 2022.","title":"Transformers are metareinforcement learners","authors":"C Luckeciano, Melo","year":"2022","venue":"international conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Oroojlooy et al., 2020] Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, and Jorge Silva. Attendlight: Universal attention-based reinforcement learning model for traffic signal control. Advances in Neural Information Processing Systems, 33:4079-4090, 2020.","title":"Attendlight: Universal attention-based reinforcement learning model for traffic signal control","authors":"Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, Jorge Silva","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Prashanth and Bhatnagar, 2011] LA Prashanth and Shalabh Bhatnagar. Reinforcement learning with average cost for adaptive control of traffic lights at intersections. In 2011 14th International IEEE Conference on Intelligent Trans- portation Systems (ITSC), pages 1640-1645. IEEE, 2011.","title":"Reinforcement learning with average cost for adaptive control of traffic lights at intersections","authors":"Bhatnagar ; La Prashanth, Shalabh Prashanth, Bhatnagar","year":"2011","venue":"14th International IEEE Conference on Intelligent Transportation Systems (ITSC)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Roess et al., 2004] Roger P Roess, Elena S Prassas, and William R McShane. Traffic engineering. Pear- son/Prentice Hall, 2004.","title":"Traffic engineering","authors":"Elena S Roger P Roess, William R Prassas, Mcshane","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Rostami et al., 2020] Mohammad Rostami, David Isele, and Eric Eaton. Using task descriptions in lifelong machine learning for improved performance and zero- shot transfer. Journal of Artificial Intelligence Research, 67:673-704, 2020.","title":"Using task descriptions in lifelong machine learning for improved performance and zeroshot transfer","authors":"Mohammad Rostami, David Isele, Eric Eaton","year":"2020","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Smith et al., 2013] Stephen F Smith, Gregory Barlow, Xiao-Feng Xie, and Zachary B Rubinstein. Surtrac: Scal- able urban traffic control. 2013.","title":"Surtrac: Scalable urban traffic control","authors":"F Stephen, Gregory Smith, Xiao-Feng Barlow, Zachary B Xie, Rubinstein","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wei et al., 2019a] Hua Wei, Chacha Chen, Guanjie Zheng, Kan Wu, Vikash Gayah, Kai Xu, and Zhenhui Li. Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, pages 1290-1298, 2019.","title":"Presslight: Learning max pressure control to coordinate traffic signals in arterial network","authors":"Wei","year":"2019","venue":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wei et al., 2019b] Hua Wei, Nan Xu, Huichu Zhang, Guan- jie Zheng, Xinshi Zang, Chacha Chen, Weinan Zhang, Yanmin Zhu, Kai Xu, and Zhenhui Li. Colight: Learn- ing network-level cooperation for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1913- 1922, 2019.","title":"Colight: Learning network-level cooperation for traffic signal control","authors":"Wei","year":"2019","venue":"Proceedings of the 28th ACM International Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wei et al., 2021] Hua Wei, Guanjie Zheng, Vikash Gayah, and Zhenhui Li. Recent advances in reinforcement learn- ing for traffic signal control: A survey of models and eval- uation. ACM SIGKDD Explorations Newsletter, 22(2):12- 18, 2021.","title":"Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation","authors":"Hua Wei, Guanjie Zheng, Vikash Gayah, Zhenhui Li","year":"2021","venue":"ACM SIGKDD Explorations Newsletter","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wolshon et al., 2016] Brian Wolshon, Anurag Pande, et al. Traffic engineering handbook. John Wiley & Sons, 2016.","title":"Traffic engineering handbook","authors":"Brian Wolshon, Anurag Pande","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wu et al., 2023] Qiang Wu, Mingyuan Li, Jun Shen, Linyuan Lü, Bo Du, and Ke Zhang. Transformerlight: A novel sequence modeling based traffic signaling mecha- nism via gated transformer. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '23, page 2639-2647, New York, NY, USA, 2023. Association for Computing Machinery.","title":"Transformerlight: A novel sequence modeling based traffic signaling mechanism via gated transformer","authors":"Qiang Wu, Mingyuan Li, Jun Shen, Linyuan Lü, Bo Du, Ke Zhang","year":"2023","venue":"Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '23","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu et al., 2013] Lun-Hui Xu, Xin-Hai Xia, and Qiang Luo. The study of reinforcement learning for traffic self- adaptive control under multiagent markov game environ- ment. Mathematical Problems in Engineering, 2013, 2013.","title":"The study of reinforcement learning for traffic selfadaptive control under multiagent markov game environment","authors":"Lun-Hui Xu, Xin-Hai Xia, Qiang Luo","year":"2013","venue":"Mathematical Problems in Engineering","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Xu et al., 2022] Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Joshua Tenenbaum, and Chuang Gan. Prompting decision transformer for few-shot pol- icy generalization. In international conference on machine learning, pages 24631-24645. PMLR, 2022.","title":"Prompting decision transformer for few-shot policy generalization","authors":"Xu","year":"2022","venue":"international conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yu et al., 2022a] Chao Yu, Akash Velu, Eugene Vinitsky, Ji- axuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Sys- tems, 35:24611-24624, 2022.","title":"The surprising effectiveness of ppo in cooperative multi-agent games","authors":"Yu","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yu et al., 2022b] Rui Yu, Dawei Du, Rodney LaLonde, Daniel Davila, Christopher Funk, Anthony Hoogs, and Brian Clipp. Cascade transformers for end-to-end person search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7267-7276, June 2022.","title":"Cascade transformers for end-to-end person search","authors":"Yu","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zang et al., 2020] Xinshi Zang, Huaxiu Yao, Guanjie Zheng, Nan Xu, Kai Xu, and Zhenhui Li. Metalight: Value-based meta-reinforcement learning for traffic signal control. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, volume 34, pages 1153-1160, 2020.","title":"Metalight: Value-based meta-reinforcement learning for traffic signal control","authors":"Xinshi Zang, Huaxiu Yao, Guanjie Zheng, Nan Xu, Kai Xu, Zhenhui Li","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2020] Huichu Zhang, Chang Liu, Weinan Zhang, Guanjie Zheng, and Yong Yu. Generalight: Im- proving environment generalization of traffic signal con- trol via meta reinforcement learning. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 1783-1792, 2020.","title":"Generalight: Improving environment generalization of traffic signal control via meta reinforcement learning","authors":"Zhang","year":"2020","venue":"Proceedings of the 29th ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zheng et al., 2019] Guanjie Zheng, Yuanhao Xiong, Xinshi Zang, Jie Feng, Hua Wei, Huichu Zhang, Yong Li, Kai Xu, and Zhenhui Li. Learning phase competition for traffic sig- nal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1963-1972, 2019.","title":"Learning phase competition for traffic signal control","authors":"Zheng","year":"2019","venue":"Proceedings of the 28th ACM International Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhu et al., 2023] Liwen Zhu, Peixi Peng, Zongqing Lu, and Yonghong Tian. Metavim: Meta variationally intrinsic motivated reinforcement learning for decentralized traffic signal control. IEEE Transactions on Knowledge and Data Engineering, 2023.","title":"Metavim: Meta variationally intrinsic motivated reinforcement learning for decentralized traffic signal control","authors":"Liwen Zhu, Peixi Peng, Zongqing Lu, Yonghong Tian","year":"2023","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}