{"type":"ScholarlyArticle","title":"TinyLight: Adaptive Traffic Signal Control on Devices with Extremely Limited Resources","authors":[{"name":"Dong Xing","slug":"dong-xing","given":"Dong","family":"Xing","affiliation":"College of Computer Science and Technology , Zhejiang University , Hangzhou , China"},{"name":"Qian Zheng","slug":"qian-zheng","given":"Qian","family":"Zheng","affiliation":"College of Computer Science and Technology , Zhejiang University , Hangzhou , China"},{"name":"Qianhui Liu","slug":"qianhui-liu","given":"Qianhui","family":"Liu","affiliation":"College of Computer Science and Technology , Zhejiang University , Hangzhou , China; Department of Electrical and Computer Engineering , National University of Singapore , Singapore"},{"name":"Gang Pan","slug":"gang-pan","given":"Gang","family":"Pan","affiliation":"College of Computer Science and Technology , Zhejiang University , Hangzhou , China"}],"abstract":"Recent advances in deep reinforcement learning (DRL) have largely promoted the performance of adaptive traffic signal control (ATSC). Nevertheless, regarding the implementation, most works are cumbersome in terms of storage and computation. This hinders their deployment on scenarios where resources are limited. In this work, we propose TinyLight, the first DRL-based ATSC model that is designed for devices with extremely limited resources. TinyLight first constructs a super-graph to associate a rich set of candidate features with a group of light-weighted network blocks. Then,  to diminish the model's resource consumption, we ablate edges in the super-graph automatically with a novel entropy-minimized objective function. This enables TinyLight to work on a standalone microcontroller with merely 2KB RAM and 32KB ROM. We evaluate TinyLight on multiple road networks with real-world traffic demands. Experiments show that even with extremely limited resources, TinyLight still achieves competitive performance. The source code and appendix of this work can be found at https://bit.ly/38hH8t8.","doi":"10.24963/ijcai.2022/555","url":"https://ijcai.org/proceedings/2022/555","pdf":"https://ijcai.org/proceedings/2022/0555.pdf","markdown":"https://ijcai.org/proceedings/2022/555.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/555","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":555,"pages":{"first":3999,"last":4005},"keywords":["Multidisciplinary Topics and Applications: Smart Cities","Multidisciplinary Topics and Applications: Transportation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986964","length_sec":81,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985107","length_sec":632,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Abdelgawad et al., 2015] Hossam Abdelgawad, Kasra Rezaee, Samah El-Tantawy, Baher Abdulhai, and Tamer Abdulazim. Assessment of adaptive traffic signal control using hardware in the loop simulation. In ITSC, 2015.","title":"Assessment of adaptive traffic signal control using hardware in the loop simulation","authors":"Hossam Abdelgawad, Kasra Rezaee, Samah El-Tantawy, Baher Abdulhai, Tamer Abdulazim","year":"2015","venue":"ITSC","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Branco et al., 2019] Sérgio Branco, André G Ferreira, and Jorge Cabral. Machine learning in resource-scarce embed- ded systems, FPGAs, and end-devices: A survey. Elec- tronics, 8(11), 2019.","title":"Machine learning in resource-scarce embedded systems, FPGAs, and end-devices: A survey","authors":"Sérgio Branco, André G Ferreira, Jorge Cabral","year":"2019","venue":"Electronics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chauhan et al., 2020] Sachin Chauhan, Kashish Bansal, and Rijurekha Sen. EcoLight: Intersection control in de- veloping regions under extreme budget and network con- straints. In NeurIPS, 2020.","title":"EcoLight: Intersection control in developing regions under extreme budget and network constraints","authors":"Sachin Chauhan, Kashish Bansal, Rijurekha Sen","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhenhui Li. Toward a thousand lights: Decentralized deep rein- forcement learning for large-scale traffic signal control. In AAAI, 2020.","title":"Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control","authors":"Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, Zhenhui Li","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cools et al., 2008] Seung-Bae Cools, Carlos Gershenson, and Bart D'Hooghe. Self-organizing traffic lights: A re- alistic simulation. In Advances in Applied Self-organizing Systems. 2008.","title":"Self-organizing traffic lights: A realistic simulation","authors":"Seung-Bae Cools, Carlos Gershenson, Bart D' Hooghe","year":"2008","venue":"Advances in Applied Self-organizing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Elefteriadou et al., 2019] Lily Elefteriadou, Pruthvi Man- junatha, Sivaramakrishnan Srinivasan, Scott Washburn, Yafeng Yin, Xi Duan, Marian Ankomah, Yinan Zheng, Thomas Chase, Tyler Valila, et al. Before and after- implementation studies of advanced signal control tech- nologies in Florida. Technical report, 2019.","title":"Before and afterimplementation studies of advanced signal control technologies in Florida","authors":"Elefteriadou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Elsken et al., 2019] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. JMLR, 20(1), 2019.","title":"Neural architecture search: A survey","authors":"Thomas Elsken, Jan Hendrik Metzen, Frank Hutter","year":"2019","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hornik et al., 1989] Kurt Hornik, Maxwell B. Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 1989.","title":"Multilayer feedforward networks are universal approximators","authors":"Kurt Hornik, Maxwell B Stinchcombe, Halbert White","year":"1989","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hunt et al., 1981] PB Hunt, DI Robertson, RD Bretherton, and RI Winton. SCOOT -A traffic responsive method of coordinating signals. Technical report, 1981.","title":"SCOOT -A traffic responsive method of coordinating signals","authors":"Hunt","year":"1981","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2021] Qize Jiang, Jingze Li, Weiwei Sun, and Baihua Zheng. Dynamic lane traffic signal control with group attention and multi-timescale reinforcement learn- ing. In IJCAI, 2021.","title":"Dynamic lane traffic signal control with group attention and multi-timescale reinforcement learning","authors":"Jiang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/501"},{"position":11,"raw":"[Liu et al., 2020] Qianhui Liu, Haibo Ruan, Dong Xing, Huajin Tang, and Gang Pan. Effective AER object classifi- cation using segmented probability-maximization learning in spiking neural networks. In AAAI, 2020.","title":"Effective AER object classification using segmented probability-maximization learning in spiking neural networks","authors":"Qianhui Liu, Haibo Ruan, Dong Xing, Huajin Tang, Gang Pan","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Miller, 1963] Alan J Miller. Settings for fixed-cycle traf- fic signals. Journal of the Operational Research Society, 14(4), 1963.","title":"Settings for fixed-cycle traffic signals","authors":"Alan J Miller","year":"1963","venue":"Journal of the Operational Research Society","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Oroojlooy et al., 2020] Afshin Oroojlooy, MohammadReza Nazari, Davood Hajinezhad, and Jorge Silva. AttendLight: Universal attention-based reinforcement learning model for traffic signal control. In NeurIPS, 2020.","title":"AttendLight: Universal attention-based reinforcement learning model for traffic signal control","authors":"Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, Jorge Silva","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rizzo et al., 2019] Stefano Giovanni Rizzo, Giovanna Van- tini, and Sanjay Chawla. Time critic policy gradient meth- ods for traffic signal control in complex and congested sce- narios. In SIGKDD, 2019.","title":"Time critic policy gradient methods for traffic signal control in complex and congested scenarios","authors":"Giovanni Stefano, Giovanna Rizzo, Sanjay Vantini, Chawla","year":"2019","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tang et al., 2019] Keshuang Tang, Manfred Boltze, Hideki Nakamura, and Zong Tian. Global Practices on Road Traffic Signal Control: Fixed-time Control at Isolated In- tersections. Elsevier, 2019.","title":"Global Practices on Road Traffic Signal Control: Fixed-time Control at Isolated Intersections","authors":"Tang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Varaiya, 2013] Pravin Varaiya. The max-pressure controller for arbitrary networks of signalized intersections. In Ad- vances in Dynamic Network Modeling in Complex Trans- portation Systems, volume 2 of Complex Networks and Dynamic Systems. Springer, 2013.","title":"The max-pressure controller for arbitrary networks of signalized intersections","authors":"Pravin Varaiya","year":"2013","venue":"Advances in Dynamic Network Modeling in Complex Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wei et al., 2019a] Hua Wei, Chacha Chen, Guanjie Zheng, Kan Wu, Vikash V. Gayah, Kai Xu, and Zhenhui Li. PressLight: Learning max pressure control to coordinate traffic signals in arterial network. In SIGKDD, 2019.","title":"PressLight: Learning max pressure control to coordinate traffic signals in arterial network","authors":"Wei","year":"2019","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wei et al., 2019b] Hua Wei, Nan Xu, Huichu Zhang, Guan- jie Zheng, Xinshi Zang, Chacha Chen, Weinan Zhang, Yanmin Zhu, Kai Xu, and Zhenhui Li. CoLight: Learn- ing network-level cooperation for traffic signal control. In CIKM, 2019.","title":"CoLight: Learning network-level cooperation for traffic signal control","authors":"Wei","year":"2019","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wei et al., 2020] Hua Wei, Guanjie Zheng, Vikash V. Gayah, and Zhenhui Li. Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation. SIGKDD Explor., 22(2), 2020.","title":"Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation","authors":"Hua Wei, Guanjie Zheng, V Vikash, Zhenhui Gayah, Li","year":"2020","venue":"SIGKDD Explor","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xing et al., 2021] Dong Xing, Qianhui Liu, Qian Zheng, and Gang Pan. Learning with generated teammates to achieve type-free ad-hoc teamwork. In IJCAI, 2021.","title":"Learning with generated teammates to achieve type-free ad-hoc teamwork","authors":"Dong Xing, Qianhui Liu, Qian Zheng, Gang Pan","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/66"},{"position":22,"raw":"[Yang et al., 2021] Long Yang, Qian Zheng, and Gang Pan. Sample complexity of policy gradient finding second- order stationary points. In AAAI, 2021.","title":"Sample complexity of policy gradient finding secondorder stationary points","authors":"Long Yang, Qian Zheng, Gang Pan","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu et al., 2020] Zhengxu Yu, Shuxian Liang, Long Wei, Zhongming Jin, Jianqiang Huang, Deng Cai, Xiaofei He, and Xian-Sheng Hua. MaCAR: Urban traffic light control via active multi-agent communication and action rectifica- tion. In IJCAI, 2020.","title":"MaCAR: Urban traffic light control via active multi-agent communication and action rectification","authors":"Yu","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/345"},{"position":24,"raw":"[Zhang et al., 2019] Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In WWW, 2019.","title":"CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario","authors":"Zhang","year":"2019","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zheng et al., 2019] Guanjie Zheng, Yuanhao Xiong, Xinshi Zang, Jie Feng, Hua Wei, Huichu Zhang, Yong Li, Kai Xu, and Zhenhui Li. Learning phase competition for traffic signal control. In CIKM, 2019.","title":"Learning phase competition for traffic signal control","authors":"Zheng","year":"2019","venue":"CIKM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}