{"type":"ScholarlyArticle","title":"InitLight: Initial Model Generation for Traffic Signal Control Using Adversarial Inverse Reinforcement Learning","authors":[{"name":"Yutong Ye","slug":"yutong-ye","given":"Yutong","family":"Ye","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University"},{"name":"Yingbo Zhou","slug":"yingbo-zhou","given":"Yingbo","family":"Zhou","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University"},{"name":"Jiepin Ding","slug":"jiepin-ding","given":"Jiepin","family":"Ding","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University"},{"name":"Ting Wang","slug":"ting-wang","given":"Ting","family":"Wang","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University"},{"name":"Mingsong Chen","slug":"mingsong-chen","given":"Mingsong","family":"Chen","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University; Shanghai Institute of Intelligent Science and Technology , Tongji University"},{"name":"Xiang Lian","slug":"xiang-lian","given":"Xiang","family":"Lian","affiliation":"Department of Computer Science , Kent State University"}],"abstract":"Due to repetitive trial-and-error style interactions between agents and a fixed traffic environment during the policy learning, existing Reinforcement Learning (RL)-based Traffic Signal Control (TSC) methods greatly suffer from long RL training time and poor adaptability of RL agents to other complex traffic environments. To address these problems, we propose a novel Adversarial Inverse Reinforcement Learning (AIRL)-based pre-training method named InitLight, which enables effective initial model generation for TSC agents. Unlike traditional RL-based TSC approaches that train a large number of agents simultaneously for a specific multi-intersection environment, InitLight pre-trains only one single initial model based on multiple single-intersection environments together with their expert trajectories. Since the reward function learned by InitLight can recover ground-truth TSC rewards for different intersections at optimality, the pre-trained agent can be deployed at intersections of any traffic environments as initial models to accelerate subsequent overall global RL training. Comprehensive experimental results show that, the initial model generated by InitLight can not only significantly accelerate the convergence with much fewer episodes, but also own superior generalization ability to accommodate various kinds of complex traffic environments.","doi":"10.24963/ijcai.2023/550","url":"https://ijcai.org/proceedings/2023/550","pdf":"https://ijcai.org/proceedings/2023/0550.pdf","markdown":"https://ijcai.org/proceedings/2023/550.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/550","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":550,"pages":{"first":4949,"last":4958},"keywords":["Machine Learning: ML: Deep reinforcement learning","Multidisciplinary Topics and Applications: MDA: Transportation","Multidisciplinary Topics and Applications: MDA: Sensor networks and smart cities"],"videos":[],"references":[{"position":1,"raw":"[Abbeel and Ng, 2004] Pieter Abbeel and Andrew Y Ng. Apprenticeship learning via inverse reinforcement learn- ing. In Proceedings of the International Conference on Machine Learning (ICML), pages 1-8, 2004.","title":"Apprenticeship learning via inverse reinforcement learning","authors":"Abbeel, ; Ng, Pieter Abbeel, Andrew Y Ng","year":"2004","venue":"Proceedings of the International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boularias et al., 2011] Abdeslam Boularias, Jens Kober, and Jan Peters. Relative entropy inverse reinforcement learning. In Proceedings of the International Confer- ence on Artificial Intelligence and Statistics (ICAIS), pages 182-189, 2011.","title":"Relative entropy inverse reinforcement learning","authors":"Abdeslam Boularias, Jens Kober, Jan Peters","year":"2011","venue":"Proceedings of the International Conference on Artificial Intelligence and Statistics (ICAIS)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2020a] Chacha Chen, Hua Wei, Nan Xu, Guan- jie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhen- hui Li. Toward a thousand lights: Decentralized deep re- inforcement learning for large-scale traffic signal control. In Proceedings of the AAAI Conference on Artificial Intel- ligence (AAAI), pages 3414-3421, 2020.","title":"Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control","authors":"Chen","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020b] Lisi Chen, Shuo Shang, Bin Yao, and Jing Li. Pay your trip for traffic congestion: Dynamic pric- ing in traffic-aware road networks. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 582-589, 2020.","title":"Pay your trip for traffic congestion: Dynamic pricing in traffic-aware road networks","authors":"Chen","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022] Rex Chen, Fei Fang, and Norman Sadeh. The real deal: A review of challenges and opportunities in moving reinforcement learning-based traffic signal control systems towards reality. In Proceedings of the IJCAI Work- shop on Agents in Traffic and Transportation Co-located, pages 14-31, 2022.","title":"The real deal: A review of challenges and opportunities in moving reinforcement learning-based traffic signal control systems towards reality","authors":"Rex Chen, Fei Fang, Norman Sadeh","year":"2022","venue":"Proceedings of the IJCAI Workshop on Agents in Traffic and Transportation Co-located","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi and Kim, 2013] Jaedeug Choi and Kee-Eung Kim. Bayesian nonparametric feature construction for inverse reinforcement learning. In Proceedings of the Interna- tional Joint Conference on Artificial Intelligence (IJCAI), pages 1287-1293, 2013.","title":"Bayesian nonparametric feature construction for inverse reinforcement learning","authors":"Kim ; Choi, Jaedeug Choi, Kee-Eung Kim","year":"2013","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2013/194"},{"position":7,"raw":"[Cools et al., 2013] Seung-Bae Cools, Carlos Gershenson, and Bart D'Hooghe. Self-organizing traffic lights: A re- alistic simulation. In Advances in applied self-organizing systems, pages 45-55. Springer, 2013.","title":"Self-organizing traffic lights: A realistic simulation","authors":"Seung-Bae Cools, Carlos Gershenson, Bart D' Hooghe","year":"2013","venue":"Advances in applied self-organizing systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[de Almeida et al., 2022] Vicente N de Almeida, Ana LC Bazzan, and Monireh Abdoos. Multiagent reinforcement learning for traffic signal control: a k-nearest neighbors based approach. In Proceedings of the IJCAI Workshop on Agents in Traffic and Transportation Co-located, pages 32-46, 2022.","title":"Multiagent reinforcement learning for traffic signal control: a k-nearest neighbors based approach","authors":"Vicente N De Almeida, Ana Lc Bazzan, Monireh Abdoos","year":"2022","venue":"Proceedings of the IJCAI Workshop on Agents in Traffic and Transportation Co-located","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 2022] Zekuan Fang, Fan Zhang, Ting Wang, Xi- ang Lian, and Mingsong Chen. Monitorlight: Reinforce- ment learning-based traffic signal control using mixed pressure monitoring. In Proceedings of International Conference on Information & Knowledge Management (CIKM), pages 478-487, 2022.","title":"Monitorlight: Reinforcement learning-based traffic signal control using mixed pressure monitoring","authors":"Zekuan Fang, Fan Zhang, Ting Wang, Xiang Lian, Mingsong Chen","year":"2022","venue":"Proceedings of International Conference on Information & Knowledge Management (CIKM)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Finn et al., 2016a] Chelsea Finn, Paul Christiano, Pieter Abbeel, and Sergey Levine. A connection between generative adversarial networks, inverse reinforcement learning, and energy-based models. arXiv preprint arXiv:1611.03852, 2016.","title":"A connection between generative adversarial networks, inverse reinforcement learning, and energy-based models","authors":"Finn","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Finn et al., 2016b] Chelsea Finn, Sergey Levine, and Pieter Abbeel. Guided cost learning: Deep inverse optimal con- trol via policy optimization. In Proceedings of the Inter- national Conference on Machine Learning (ICML), pages 49-58, 2016.","title":"Guided cost learning: Deep inverse optimal control via policy optimization","authors":"Finn","year":"2016","venue":"Proceedings of the International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fu et al., 2018] Justin Fu, Katie Luo, and Sergey Levine. Learning robust rewards with adverserial inverse rein- forcement learning. In Proceedings of the International Conference on Learning Representations (ICLR), pages 1- 15, 2018.","title":"Learning robust rewards with adverserial inverse reinforcement learning","authors":"Justin Fu, Katie Luo, Sergey Levine","year":"2018","venue":"Proceedings of the International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ho and Ermon, 2016] Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), pages 4565-4573, 2016.","title":"Generative adversarial imitation learning","authors":"Jonathan Ho, Stefano Ermon","year":"2016","venue":"Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hunt et al., 1982] PB Hunt, DI Robertson, RD Bretherton, and M Cr Royle. The scoot on-line traffic signal optimi- sation technique. Traffic Engineering & Control, 23(4), 1982.","title":"The scoot on-line traffic signal optimisation technique","authors":"Hunt","year":"1982","venue":"Traffic Engineering & Control","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jiang et al., 2021] Qize Jiang, Jingze Li, Weiwei Sun SUN, and Baihua Zheng. Dynamic lane traffic signal control with group attention and multi-timescale reinforcement learning. In Proceedings of the International Joint Confer- ence on Artificial Intelligence (IJCAI), pages 3642-3648, 2021.","title":"Dynamic lane traffic signal control with group attention and multi-timescale reinforcement learning","authors":"Jiang","year":"2021","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2021/501"},{"position":16,"raw":"[Jiang et al., 2022] Qize Jiang, Minhao Qin, Shengmin Shi, Weiwei Sun, and Baihua Zheng. Multi-agent reinforce- ment learning for traffic signal control through universal communication method. In Proceedings of the Interna- tional Joint Conference on Artificial Intelligence (IJCAI), pages 3854-3860, 2022.","title":"Multi-agent reinforcement learning for traffic signal control through universal communication method","authors":"Jiang","year":"2022","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2022/535"},{"position":17,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representa- tions (ICLR), pages 1-15, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":"Proceedings of the International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Koohy et al., 2022] Behrad Koohy, Sebastian Stein, Enrico Gerding, and Ghaithaa Manla. Reward function design in multi-agent reinforcement learning for traffic signal con- trol. In Proceedings of the IJCAI Workshop on Agents in Traffic and Transportation Co-located, pages 1-13, 2022.","title":"Reward function design in multi-agent reinforcement learning for traffic signal control","authors":"Behrad Koohy, Sebastian Stein, Enrico Gerding, Ghaithaa Manla","year":"2022","venue":"Proceedings of the IJCAI Workshop on Agents in Traffic and Transportation Co-located","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Koonce and Rodegerdts, 2008] Peter Koonce and Lee Rodegerdts. Traffic signal timing manual. Technical report, United States. Federal Highway Administration, 2008.","title":"Traffic signal timing manual","authors":"Rodegerdts Koonce, Peter Koonce, Lee Rodegerdts","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kullback, 1997] Solomon Kullback. Information theory and statistics. Courier Corporation, 1997.","title":"Information theory and statistics","authors":"Kullback","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ng et al., 2000] Andrew Y Ng, Stuart Russell, et al. Al- gorithms for inverse reinforcement learning. In Proceed- ings of the International Conference on Machine Learning (ICML), pages 663-670, 2000.","title":"Algorithms for inverse reinforcement learning","authors":"Y Andrew, Stuart Ng, Russell","year":"2000","venue":"Proceedings of the International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"Lowrie PR. Scats: A traffic responsive method of controlling urban traffic control/pr lowrie. Roads and Traffic Authority, 1992.","title":"Scats: A traffic responsive method of controlling urban traffic control/pr lowrie","authors":"P R Lowrie","year":"1992","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ramachandran and Amir, 2007] Deepak Ramachandran and Eyal Amir. Bayesian inverse reinforcement learning. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pages 2586-2591, 2007.","title":"Bayesian inverse reinforcement learning","authors":"Deepak Ramachandran, Eyal Amir","year":"2007","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2007/416"},{"position":24,"raw":"[Ratliff et al., 2006] Nathan D Ratliff, J Andrew Bagnell, and Martin A Zinkevich. Maximum margin planning. In Proceedings of the International Conference on Machine Learning (ICML), pages 729-736, 2006.","title":"Maximum margin planning","authors":"J Nathan D Ratliff, Martin A Andrew Bagnell, Zinkevich","year":"2006","venue":"Proceedings of the International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rizzo et al., 2019] Stefano Giovanni Rizzo, Giovanna Van- tini, and Sanjay Chawla. Time critic policy gradient meth- ods for traffic signal control in complex and congested sce- narios. In Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD), pages 1654-1664, 2019.","title":"Time critic policy gradient methods for traffic signal control in complex and congested scenarios","authors":"Giovanni Stefano, Giovanna Rizzo, Sanjay Vantini, Chawla","year":"2019","venue":"Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Russell, 1998] Stuart Russell. Learning agents for uncertain environments. In Proceedings of the Annual Conference on Computational Learning Theory (ACCLT), pages 101- 103, 1998.","title":"Learning agents for uncertain environments","authors":"Stuart Russell","year":"1998","venue":"Proceedings of the Annual Conference on Computational Learning Theory (ACCLT)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schulman et al., 2015] John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High- dimensional continuous control using generalized advan- tage estimation. Proceedings of the International Con- ference on Learning Representations (ICLR), pages 1-14, 2015.","title":"Highdimensional continuous control using generalized advantage estimation","authors":"John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, Pieter Abbeel","year":"2015","venue":"Proceedings of the International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tesauro and others, 1995] Gerald Tesauro et al. Temporal difference learning and td-gammon. Communications of the ACM, 38(3):58-68, 1995.","title":"Temporal difference learning and td-gammon","authors":"Gerald Tesauro","year":"1995","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Török and Kertész, 1996] János Török and János Kertész. The green wave model of two-dimensional traffic: Tran- sitions in the flow properties and in the geometry of the traffic jam. Physica A: Statistical Mechanics and its Ap- plications, 231(4):515-533, 1996.","title":"The green wave model of two-dimensional traffic: Transitions in the flow properties and in the geometry of the traffic jam","authors":"Kertész Török, János Török, János Kertész","year":"1996","venue":"Physica A: Statistical Mechanics and its Applications","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Varaiya, 2013] Pravin Varaiya. The max-pressure controller for arbitrary networks of signalized intersections. In Ad- vances in dynamic network modeling in complex trans- portation systems, pages 27-66. Springer, 2013.","title":"The max-pressure controller for arbitrary networks of signalized intersections","authors":"Pravin Varaiya","year":"2013","venue":"Advances in dynamic network modeling in complex transportation systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wei et al., 2019a] Hua Wei, Chacha Chen, Guanjie Zheng, Kan Wu, Vikash Gayah, Kai Xu, and Zhenhui Li. Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In Proceedings of the In- ternational Conference on Knowledge Discovery & Data Mining (SIGKDD), pages 1290-1298, 2019.","title":"Presslight: Learning max pressure control to coordinate traffic signals in arterial network","authors":"Wei","year":"2019","venue":"Proceedings of the International Conference on Knowledge Discovery & Data Mining (SIGKDD)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wei et al., 2019b] Hua Wei, Guanjie Zheng, Vikash Gayah, and Zhenhui Li. A survey on traffic signal control meth- ods. arXiv preprint arXiv:1904.08117, 2019.","title":"A survey on traffic signal control methods","authors":"Wei","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xing et al., 2022] Dong Xing, Qian Zheng, Qianhui Liu, and Gang Pan. Tinylight: Adaptive traffic signal control on devices with extremely limited resources. In Proceed- ings of the International Joint Conference on Artificial In- telligence (IJCAI), pages 3999-4005, 2022.","title":"Tinylight: Adaptive traffic signal control on devices with extremely limited resources","authors":"Dong Xing, Qian Zheng, Qianhui Liu, Gang Pan","year":"2022","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2022/555"},{"position":35,"raw":"[Xiong et al., 2019] Yuanhao Xiong, Guanjie Zheng, Kai Xu, and Zhenhui Li. Learning traffic signal control from demonstrations. In Proceedings of the International Conference on Information & Knowledge Management (CIKM), pages 2289-2292, 2019.","title":"Learning traffic signal control from demonstrations","authors":"Xiong","year":"2019","venue":"Proceedings of the International Conference on Information & Knowledge Management (CIKM)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ye et al., 2021] Yutong Ye, Wupan Zhao, Tongquan Wei, Shiyan Hu, and Mingsong Chen. Fedlight: Federated rein- forcement learning for autonomous multi-intersection traf- fic signal control. In Proceedings of the Design Automa- tion Conference (DAC), pages 847-852, 2021.","title":"Fedlight: Federated reinforcement learning for autonomous multi-intersection traffic signal control","authors":"Yutong Ye, Wupan Zhao, Tongquan Wei, Shiyan Hu, Mingsong Chen","year":"2021","venue":"Proceedings of the Design Automation Conference (DAC)","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Ye et al., 2022] Yutong Ye, Jiepin Ding, Ting Wang, Jun- long Zhou, Xian Wei, and Mingsong Chen. Fairlight: Fairness-aware autonomous traffic signal control with hi- erarchical action space. IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems, 2022.","title":"Fairlight: Fairness-aware autonomous traffic signal control with hierarchical action space","authors":"Yutong Ye, Jiepin Ding, Ting Wang, Junlong Zhou, Xian Wei, Mingsong Chen","year":"2022","venue":"IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zang et al., 2020] Xinshi Zang, Huaxiu Yao, Guanjie Zheng, Nan Xu, Kai Xu, and Zhenhui Li. Metalight: Value-based meta-reinforcement learning for traffic signal control. In Proceedings of the AAAI Conference on Artifi- cial Intelligence (AAAI), pages 1153-1160, 2020.","title":"Metalight: Value-based meta-reinforcement learning for traffic signal control","authors":"Xinshi Zang, Huaxiu Yao, Guanjie Zheng, Nan Xu, Kai Xu, Zhenhui Li","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhang et al., 2019] Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In Proceedings of the World Wide Web Conference (WWW), pages 3620-3624, 2019.","title":"Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario","authors":"Zhang","year":"2019","venue":"Proceedings of the World Wide Web Conference (WWW)","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang et al., 2020] Huichu Zhang, Chang Liu, Weinan Zhang, Guanjie Zheng, and Yong Yu. Generalight: Im- proving environment generalization of traffic signal con- trol via meta reinforcement learning. In Proceedings of the International Conference on Information & Knowl- edge Management (CIKM), pages 1783-1792, 2020.","title":"Generalight: Improving environment generalization of traffic signal control via meta reinforcement learning","authors":"Zhang","year":"2020","venue":"Proceedings of the International Conference on Information & Knowledge Management (CIKM)","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhang et al., 2021] Liang Zhang, Qiang Wu, and Jianming Deng. Knowledge intensive state design for traffic signal control. arXiv preprint arXiv:2201.00006, 2021.","title":"Knowledge intensive state design for traffic signal control","authors":"Liang Zhang, Qiang Wu, Jianming Deng","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhao et al., 2022] Wupan Zhao, Yutong Ye, Jiepin Ding, Ting Wang, Tongquan Wei, and Mingsong Chen. Ip- dalight: Intensity-and phase duration-aware traffic signal control based on reinforcement learning. Journal of Sys- tems Architecture, 123:102374, 2022.","title":"Ipdalight: Intensity-and phase duration-aware traffic signal control based on reinforcement learning","authors":"Zhao","year":"2022","venue":"Journal of Systems Architecture","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zheng et al., 2021] Guanjie Zheng, Hanyang Liu, Kai Xu, and Zhenhui Li. Objective-aware traffic simulation via in- verse reinforcement learning. In Proceedings of the In- ternational Joint Conference on Artificial Intelligence (IJ- CAI), pages 3771-3777, 2021.","title":"Objective-aware traffic simulation via inverse reinforcement learning","authors":"Zheng","year":"2021","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJ-CAI)","doi":null,"url":null,"internal_url":"/proceedings/2021/519"},{"position":44,"raw":"[Zhu et al., 2021] Liwen Zhu, Peixi Peng, Zongqing Lu, Xiangqian Wang, and Yonghong Tian. Variationally and intrinsically motivated reinforcement learning for decentralized traffic signal control. arXiv preprint arXiv:2101.00746, 2021.","title":"Variationally and intrinsically motivated reinforcement learning for decentralized traffic signal control","authors":"Liwen Zhu, Peixi Peng, Zongqing Lu, Xiangqian Wang, Yonghong Tian","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Ziebart et al., 2008] Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey, et al. Maximum entropy inverse reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 1433-1438, 2008.","title":"Maximum entropy inverse reinforcement learning","authors":"Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey","year":"2008","venue":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}