{"type":"ScholarlyArticle","title":"DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback","authors":[{"name":"Junfan Lin","slug":"junfan-lin","given":"Junfan","family":"Lin","affiliation":"The School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China."},{"name":"Yuying Zhu","slug":"yuying-zhu","given":"Yuying","family":"Zhu","affiliation":"The School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China."},{"name":"Lingbo Liu","slug":"lingbo-liu","given":"Lingbo","family":"Liu","affiliation":"Department Land Surveying and Geo-Informatic , The Hong Kong Polytechnic University , Hongkong.; Smart Cities Research Institute , The Hong Kong Polytechnic University , Hong Kong"},{"name":"Yang Liu","slug":"yang-liu","given":"Yang","family":"Liu","affiliation":"The School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China."},{"name":"Guanbin Li","slug":"guanbin-li","given":"Guanbin","family":"Li","affiliation":"The School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China."},{"name":"Liang Lin","slug":"liang-lin","given":"Liang","family":"Lin","affiliation":"The School of Computer Science and Engineering , Sun Yat-sen University , Guangzhou , China."}],"abstract":"Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network, which in turn enhances fuel utilization efficiency, air quality, and road safety, benefiting society as a whole. Due to the complexity of long-horizon control and coordination, most prior TSC methods leverage deep reinforcement learning (RL) to search for a control policy and have witnessed great success. However, TSC still faces two significant challenges. 1) The travel time of a vehicle is delayed feedback on the effectiveness of TSC policy at each traffic intersection since it is obtained after the vehicle has left the road network. Although several heuristic reward functions have been proposed as substitutes for travel time, they are usually biased and not leading the policy to improve in the correct direction. 2) The traffic condition of each intersection is influenced by the non-local intersections since vehicles traverse multiple intersections over time. Therefore, the TSC agent is required to leverage both the local observation and the non-local traffic conditions to predict the long-horizontal traffic conditions of each intersection comprehensively. To address these challenges, we propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness and a non-local enhanced TSC agent to better predict future traffic conditions for more precise traffic control. Extensive experiments and ablation studies demonstrate that DenseLight can consistently outperform advanced baselines on various road networks with diverse traffic flows. The code is available at https://github.com/junfanlin/DenseLight.","doi":"10.24963/ijcai.2023/672","url":"https://ijcai.org/proceedings/2023/672","pdf":"https://ijcai.org/proceedings/2023/0672.pdf","markdown":"https://ijcai.org/proceedings/2023/672.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/672","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":672,"pages":{"first":6058,"last":6066},"keywords":["AI for Good: Agent-based and Multi-agent Systems","AI for Good: Planning and Scheduling"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2020] Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhenhui Li. Toward a thousand lights: Decentralized deep rein- forcement learning for large-scale traffic signal control. In AAAI, volume 34, pages 3414-3421, 2020.","title":"Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control","authors":"Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, Zhenhui Li","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2022] Changlu Chen, Yanbin Liu, Ling Chen, and Chengqi Zhang. Bidirectional spatial-temporal adap- tive transformer for urban traffic flow forecasting. TNNLS, 2022.","title":"Bidirectional spatial-temporal adaptive transformer for urban traffic flow forecasting","authors":"Chen","year":"2022","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chu et al., 2019] Tianshu Chu, Jie Wang, Lara Codecà, and Zhaojian Li. Multi-agent deep reinforcement learning for large-scale traffic signal control. TITS, 21(3):1086-1095, 2019.","title":"Multi-agent deep reinforcement learning for large-scale traffic signal control","authors":"Tianshu Chu, Jie Wang, Lara Codecà, Zhaojian Li","year":"2019","venue":"TITS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cools et al., 2013] Seung-Bae Cools, Carlos Gershenson, and Bart D'Hooghe. Self-organizing traffic lights: A re- alistic simulation. In Advances in applied self-organizing systems, pages 45-55. Springer, 2013.","title":"Self-organizing traffic lights: A realistic simulation","authors":"Seung-Bae Cools, Carlos Gershenson, Bart D' Hooghe","year":"2013","venue":"Advances in applied self-organizing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[D'Almeida et al., 2021] Marcelo D'Almeida, Aline Paes, and Daniel Mossé. Designing reinforcement learning agents for traffic signal control with the right goals: a time- loss based approach. In ITSC, pages 1412-1418. IEEE, 2021.","title":"Designing reinforcement learning agents for traffic signal control with the right goals: a timeloss based approach","authors":"D' Marcelo, Aline Almeida, Daniel Paes, Mossé","year":"2021","venue":"ITSC","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fujimoto et al., 2018] Scott Fujimoto, Herke Hoof, and David Meger. Addressing function approximation error in actor-critic methods. In ICML, pages 1587-1596. PMLR, 2018.","title":"Addressing function approximation error in actor-critic methods","authors":"Scott Fujimoto, Herke Hoof, David Meger","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2021] Xin Guo, Zhengxu Yu, Pengfei Wang, Zhongming Jin, Jianqiang Huang, Deng Cai, Xiaofei He, and Xiansheng Hua. Urban traffic light control via ac- tive multi-agent communication and supply-demand mod- eling. TKDE, pages 1-1, 2021.","title":"Urban traffic light control via active multi-agent communication and supply-demand modeling","authors":"Xin Guo, Zhengxu Yu, Pengfei Wang, Zhongming Jin, Jianqiang Huang, Deng Cai, Xiaofei He, Xiansheng Hua","year":"2021","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hong et al., 2022] Wanshi Hong, Gang Tao, Hong Wang, and Chieh Wang. Traffic signal control with adaptive online-learning scheme using multiple-model neural net- works. TNNLS, 2022.","title":"Traffic signal control with adaptive online-learning scheme using multiple-model neural networks","authors":"Hong","year":"2022","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2021] Hao Huang, Zhiqun Hu, Zhaoming Lu, and Xiangming Wen. Network-scale traffic signal con- trol via multiagent reinforcement learning with deep spa- tiotemporal attentive network. IEEE trans cybern, 2021.","title":"Network-scale traffic signal control via multiagent reinforcement learning with deep spatiotemporal attentive network","authors":"Hao Huang, Zhiqun Hu, Zhaoming Lu, Xiangming Wen","year":"2021","venue":"IEEE trans cybern","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hunt et al., 1981] PB Hunt, DI Robertson, RD Bretherton, and RI Winton. Scoot-a traffic responsive method of co- ordinating signals. Publication of: Transport and Road Research Laboratory, LR 1014 Monograph, Jan 1981.","title":"Scoot-a traffic responsive method of coordinating signals","authors":"Hunt","year":"1014","venue":"Monograph","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Koonce and Rodegerdts, 2008] Peter Koonce and Lee Rodegerdts. Traffic signal timing manual. United States. Federal Highway Administration, 2008.","title":"Traffic signal timing manual","authors":"Rodegerdts Koonce, Peter Koonce, Lee Rodegerdts","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- offrey Hinton. Deep learning. nature, 521(7553):436-444, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liang et al., 2021] Enming Liang, Kexin Wen, William HK Lam, Agachai Sumalee, and Renxin Zhong. An integrated reinforcement learning and centralized programming ap- proach for online taxi dispatching. TNNLS, 2021.","title":"An integrated reinforcement learning and centralized programming approach for online taxi dispatching","authors":"Liang","year":"2021","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Little et al., 1981] John DC Little, Mark D Kelson, and Nathan H Gartner. Maxband: A versatile program for set- ting signals on arteries and triangular networks. Trans- portation Research Record 795, pages 40-46, 1981.","title":"Maxband: A versatile program for setting signals on arteries and triangular networks","authors":"John Dc Little, Nathan H Mark D Kelson, Gartner","year":"1981","venue":"Transportation Research Record","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2020] Lingbo Liu, Jiajie Zhen, Guanbin Li, Geng Zhan, Zhaocheng He, Bowen Du, and Liang Lin. Dy- namic spatial-temporal representation learning for traffic flow prediction. TITS, 22(11):7169-7183, 2020.","title":"Dynamic spatial-temporal representation learning for traffic flow prediction","authors":"Lingbo Liu, Jiajie Zhen, Guanbin Li, Geng Zhan, Zhaocheng He, Bowen Du, Liang Lin","year":"2020","venue":"TITS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2022] Yang Liu, Yu-Shen Wei, Hong Yan, Guan- Bin Li, and Liang Lin. Causal reasoning meets visual rep- resentation learning: A prospective study. Machine Intel- ligence Research, pages 1-27, 2022.","title":"Causal reasoning meets visual representation learning: A prospective study","authors":"Yang Liu, Yu-Shen Wei, Hong Yan, Guan-Bin Li, Liang Lin","year":"2022","venue":"Machine Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lou et al., 2020] Jungang Lou, Yunliang Jiang, Qing Shen, Ruiqin Wang, and Zechao Li. Probabilistic regularized ex- treme learning for robust modeling of traffic flow forecast- ing. TNNLS, 2020.","title":"Probabilistic regularized extreme learning for robust modeling of traffic flow forecasting","authors":"Jungang Lou, Yunliang Jiang, Qing Shen, Ruiqin Wang, Zechao Li","year":"2020","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Luk et al., 1982] JY Luk, AG Sims, and PR Lowrie. Scats- application and field comparison with a transyt optimised fixed time system. In International Conference on Road Traffic Signalling, number 207, 1982.","title":"Scatsapplication and field comparison with a transyt optimised fixed time system","authors":"Luk","year":"1982","venue":"International Conference on Road Traffic Signalling","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mirchandani and Head, 2001] Pitu Mirchandani and Larry Head. A real-time traffic signal control system: architec- ture, algorithms, and analysis. Transportation Research Part C: Emerging Technologies, 9(6):415-432, 2001.","title":"A real-time traffic signal control system: architecture, algorithms, and analysis","authors":"Pitu Mirchandani, Larry Head","year":"2001","venue":"Transportation Research Part C: Emerging Technologies","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidje- land, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518(7540):529-533, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski","year":"2015","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Oroojlooy et al., 2020] Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, and Jorge Silva. Attendlight: Universal attention-based reinforcement learning model for traffic signal control. NeurIPS, 33:4079-4090, 2020.","title":"Attendlight: Universal attention-based reinforcement learning model for traffic signal control","authors":"Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, Jorge Silva","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Prashanth and Bhatnagar, 2010] LA Prashanth and Shalabh Bhatnagar. Reinforcement learning with function approx- imation for traffic signal control. TITS, 12(2):412-421, 2010.","title":"Reinforcement learning with function approximation for traffic signal control","authors":"Bhatnagar ; La Prashanth, Shalabh Prashanth, Bhatnagar","year":"2010","venue":"TITS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rashid et al., 2018] Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning. In ICML, volume 80, pages 4295-4304. PMLR, 2018.","title":"QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning","authors":"Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, Shimon Whiteson","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Roess et al., 2004] Roger P Roess, Elena S Prassas, and William R McShane. Traffic engineering. Pear- son/Prentice Hall, 2004.","title":"Traffic engineering","authors":"Elena S Roger P Roess, William R Prassas, Mcshane","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Son et al., 2019] Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. QTRAN: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In Kamalika Chaud- huri and Ruslan Salakhutdinov, editors, ICML, volume 97 of PMLR, pages 5887-5896. PMLR, 09-15 Jun 2019.","title":"QTRAN: Learning to factorize with transformation for cooperative multi-agent reinforcement learning","authors":"Son","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tan et al., 2019] Tian Tan, Feng Bao, Yue Deng, Alex Jin, Qionghai Dai, and Jie Wang. Cooperative deep reinforce- ment learning for large-scale traffic grid signal control. IEEE trans cybern, 50(6):2687-2700, 2019.","title":"Cooperative deep reinforcement learning for large-scale traffic grid signal control","authors":"Tian Tan, Feng Bao, Yue Deng, Alex Jin, Qionghai Dai, Jie Wang","year":"2019","venue":"IEEE trans cybern","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Taylor, 2002] Brian D Taylor. Rethinking traffic congestion. Access Magazine, 1(21):8-16, 2002.","title":"Rethinking traffic congestion","authors":"Brian D Taylor","year":"2002","venue":"Access Magazine","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Van der Pol and Oliehoek, 2016] Elise Van der Pol and Frans A Oliehoek. Coordinated deep reinforcement learn- ers for traffic light control. NeurIPS, 2016.","title":"Coordinated deep reinforcement learners for traffic light control","authors":"Pol Van Der, Elise Oliehoek, Frans A Van Der Pol, Oliehoek","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Van Hasselt et al., 2016] Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with dou- ble q-learning. In AAAI, volume 30, 2016.","title":"Deep reinforcement learning with double q-learning","authors":"Van Hasselt","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Varaiya, 2013] Pravin Varaiya. Max pressure control of a network of signalized intersections. Transportation Re- search Part C: Emerging Technologies, 36:177-195, 2013.","title":"Max pressure control of a network of signalized intersections","authors":"Pravin Varaiya","year":"2013","venue":"Transportation Research Part C: Emerging Technologies","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2020] Xiaoqiang Wang, Liangjun Ke, Zhimin Qiao, and Xinghua Chai. Large-scale traffic signal control using a novel multiagent reinforcement learning. IEEE transactions on cybernetics, 51(1):174-187, 2020.","title":"Large-scale traffic signal control using a novel multiagent reinforcement learning","authors":"Xiaoqiang Wang, Liangjun Ke, Zhimin Qiao, Xinghua Chai","year":"2020","venue":"IEEE transactions on cybernetics","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2022] Tian Wang, Jiahui Chen, Jinhu Lü, Kexin Liu, Aichun Zhu, Hichem Snoussi, and Baochang Zhang. Synchronous spatiotemporal graph transformer: A new framework for traffic data prediction. TNNLS, 2022.","title":"Synchronous spatiotemporal graph transformer: A new framework for traffic data prediction","authors":"Tian Wang, Jiahui Chen, Jinhu Lü, Kexin Liu, Aichun Zhu, Hichem Snoussi, Baochang Zhang","year":"2022","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wei et al., 2018] Hua Wei, Guanjie Zheng, Huaxiu Yao, and Zhenhui Li. Intellilight: A reinforcement learning ap- proach for intelligent traffic light control. In CKDDM, pages 2496-2505, 2018.","title":"Intellilight: A reinforcement learning approach for intelligent traffic light control","authors":"Hua Wei, Guanjie Zheng, Huaxiu Yao, Zhenhui Li","year":"2018","venue":"CKDDM","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wei et al., 2019a] Hua Wei, Chacha Chen, Guanjie Zheng, Kan Wu, Vikash Gayah, Kai Xu, and Zhenhui Li. Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In CKDDM, pages 1290-1298, 2019.","title":"Presslight: Learning max pressure control to coordinate traffic signals in arterial network","authors":"Wei","year":"2019","venue":"CKDDM","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wei et al., 2019b] Hua Wei, Nan Xu, Huichu Zhang, Guan- jie Zheng, Xinshi Zang, Chacha Chen, Weinan Zhang, Yanmin Zhu, Kai Xu, and Zhenhui Li. Colight: Learn- ing network-level cooperation for traffic signal control. In CIKM, pages 1913-1922, 2019.","title":"Colight: Learning network-level cooperation for traffic signal control","authors":"Wei","year":"2019","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Weng et al., 2021] Jiayi Weng, Huayu Chen, Dong Yan, Kaichao You, Alexis Duburcq, Minghao Zhang, Hang Su, and Jun Zhu. Tianshou: A highly modularized deep reinforcement learning library. arXiv preprint arXiv:2107.14171, 2021.","title":"Tianshou: A highly modularized deep reinforcement learning library","authors":"Jiayi Weng, Huayu Chen, Dong Yan, Kaichao You, Alexis Duburcq, Minghao Zhang, Hang Su, Jun Zhu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wu et al., 2021] Qiang Wu, Liang Zhang, Jun Shen, Linyuan Lü, Bo Du, and Jianqing Wu. Efficient pressure: Improving efficiency for signalized intersections. arXiv preprint arXiv:2112.02336, 2021.","title":"Efficient pressure: Improving efficiency for signalized intersections","authors":"Qiang Wu, Liang Zhang, Jun Shen, Linyuan Lü, Bo Du, Jianqing Wu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Xu et al., 2021] Bingyu Xu, Yaowei Wang, Zhaozhi Wang, Huizhu Jia, and Zongqing Lu. Hierarchically and cooper- atively learning traffic signal control. In AAAI, volume 35, pages 669-677, 2021.","title":"Hierarchically and cooperatively learning traffic signal control","authors":"Xu","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yang et al., 2016] Hao-Fan Yang, Tharam S Dillon, and Yi- Ping Phoebe Chen. Optimized structure of the traffic flow forecasting model with a deep learning approach. TNNLS, 28(10):2371-2381, 2016.","title":"Optimized structure of the traffic flow forecasting model with a deep learning approach","authors":"Hao-Fan, Tharam S Yang, Yi-Ping Phoebe Dillon, Chen","year":"2016","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zang et al., 2020] Xinshi Zang, Huaxiu Yao, Guanjie Zheng, Nan Xu, Kai Xu, and Zhenhui Li. Metalight: Value-based meta-reinforcement learning for traffic signal control. In AAAI, volume 34, pages 1153-1160, 2020.","title":"Metalight: Value-based meta-reinforcement learning for traffic signal control","authors":"Xinshi Zang, Huaxiu Yao, Guanjie Zheng, Nan Xu, Kai Xu, Zhenhui Li","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zeng, 2021] Zheng Zeng. Graphlight: Graph-based rein- forcement learning for traffic signal control. In ICCCS, pages 645-650. IEEE, 2021.","title":"Graphlight: Graph-based reinforcement learning for traffic signal control","authors":"Zheng Zeng","year":"2021","venue":"ICCCS","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhang and Batterman, 2013] Kai Zhang and Stuart Batter- man. Air pollution and health risks due to vehicle traffic. Science of the total Environment, 450:307-316, 2013.","title":"Air pollution and health risks due to vehicle traffic","authors":"Batterman Zhang, Kai Zhang, Stuart Batterman","year":"2013","venue":"Science of the total Environment","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhang et al., 2019] Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In W3C, pages 3620-3624, 2019.","title":"Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario","authors":"Zhang","year":"2019","venue":"W3C","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Zhang et al., 2021] Liang Zhang, Qiang Wu, Jun Shen, Linyuan Lü, Jianqing Wu, and Bo Du. Expres- sion is enough: Improving traffic signal control with advanced traffic state representation. arXiv preprint arXiv:2112.10107, 2021.","title":"Expression is enough: Improving traffic signal control with advanced traffic state representation","authors":"Liang Zhang, Qiang Wu, Jun Shen, Linyuan Lü, Jianqing Wu, Bo Du","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zheng et al., 2019] Guanjie Zheng, Yuanhao Xiong, Xinshi Zang, Jie Feng, Hua Wei, Huichu Zhang, Yong Li, Kai Xu, and Zhenhui Li. Learning phase competition for traffic signal control. In CIKM, pages 1963-1972, 2019.","title":"Learning phase competition for traffic signal control","authors":"Zheng","year":"2019","venue":"CIKM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}