{"type":"ScholarlyArticle","title":"An Online Learning Framework for Energy-Efficient Navigation of Electric Vehicles","authors":[{"name":"Niklas Åkerblom","slug":"niklas-akerblom","given":"Niklas","family":"Åkerblom","affiliation":"Volvo Car Corporation; Chalmers University of Technology"},{"name":"Yuxin Chen","slug":"yuxin-chen","given":"Yuxin","family":"Chen","affiliation":"The University of Chicago"},{"name":"Morteza Haghir Chehreghani","slug":"morteza-haghir-chehreghani","given":"Morteza","family":"Haghir Chehreghani","affiliation":"Chalmers University of Technology"}],"abstract":"Energy-efficient navigation constitutes an important challenge in electric vehicles, due to their limited battery capacity. We employ a Bayesian approach to model the energy consumption at road segments for efficient navigation. In order to learn the model parameters, we develop an online learning framework and investigate several exploration strategies such as Thompson Sampling and Upper Confidence Bound. We then extend our online learning framework to multi-agent setting, where multiple vehicles adaptively navigate and learn the parameters of the energy model. We analyze Thompson Sampling and establish rigorous regret bounds on its performance. Finally, we demonstrate the performance of our methods via several real-world experiments on Luxembourg SUMO Traffic dataset.","doi":"10.24963/ijcai.2020/284","url":"https://ijcai.org/proceedings/2020/284","pdf":"https://ijcai.org/proceedings/2020/0284.pdf","markdown":"https://ijcai.org/proceedings/2020/284.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/284","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":284,"pages":{"first":2051,"last":2057},"keywords":["Machine Learning Applications: Networks","Machine Learning: Online Learning","Multidisciplinary Topics and Applications: Transportation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24741/file_24741.m3u8","length_sec":272,"provider":"hls"}],"references":[{"position":1,"raw":"[Agrawal and Goyal, 2012] Shipra Agrawal and Navin Goyal. Analysis of thompson sampling for the multi- armed bandit problem. In Conference on learning theory, pages 39-1, 2012.","title":"Analysis of thompson sampling for the multiarmed bandit problem","authors":"Shipra Agrawal, Navin Goyal","year":"2012","venue":"Conference on learning theory","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Artmeier et al., 2010] Andreas Artmeier, Julian Haselmayr, Martin Leucker, and Martin Sachenbacher. The shortest path problem revisited: Optimal routing for electric vehi- cles. In Annual conference on artificial intelligence, pages 309-316. Springer, 2010.","title":"The shortest path problem revisited: Optimal routing for electric vehicles","authors":"Andreas Artmeier, Julian Haselmayr, Martin Leucker, Martin Sachenbacher","year":"2010","venue":"Annual conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Auer, 2002] Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res., 3:397-422, 2002.","title":"Using confidence bounds for exploitation-exploration trade-offs","authors":"Peter Auer","year":"2002","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Basso et al., 2019] Rafael Basso, Balázs Kulcsár, Bo Egardt, Peter Lindroth, and Ivan Sanchez-Diaz. Energy consumption estimation integrated into the elec- tric vehicle routing problem. Transportation Research Part D: Transport and Environment, 69:141-167, 2019.","title":"Energy consumption estimation integrated into the electric vehicle routing problem","authors":"Rafael Basso, Balázs Kulcsár, Bo Egardt, Peter Lindroth, Ivan Sanchez-Diaz","year":"2019","venue":"Transportation Research Part D: Transport and Environment","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Baum et al., 2017] Moritz Baum, Jonas Sauer, Dorothea Wagner, and Tobias Zündorf. Consumption Profiles in Route Planning for Electric Vehicles: Theory and Appli- cations. In 16th International Symposium on Experimen- tal Algorithms (SEA 2017), volume 75 of Leibniz Inter- national Proceedings in Informatics (LIPIcs), pages 19:1- 19:18, 2017.","title":"Consumption Profiles in Route Planning for Electric Vehicles: Theory and Applications","authors":"Baum","year":"2017","venue":"16th International Symposium on Experimental Algorithms (SEA 2017)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bubeck and Liu, 2013] Sébastien Bubeck and Che-Yu Liu. Prior-free and prior-dependent regret bounds for thompson sampling. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Informa- tion Processing Systems 2013, pages 638-646, 2013.","title":"Prior-free and prior-dependent regret bounds for thompson sampling","authors":"Liu Bubeck, Sébastien Bubeck, Che-Yu Liu","year":"2013","venue":"27th Annual Conference on Neural Information Processing Systems 2013","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chapelle and Li, 2011] Olivier Chapelle and Lihong Li. An empirical evaluation of thompson sampling. In NIPS, pages 2249-2257, 2011.","title":"An empirical evaluation of thompson sampling","authors":"Li ; Chapelle, Olivier Chapelle, Lihong Li","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2013] Wei Chen, Yajun Wang, and Yang Yuan. Combinatorial multi-armed bandit: General framework and applications. In International Conference on Machine Learning, pages 151-159, 2013.","title":"Combinatorial multi-armed bandit: General framework and applications","authors":"Wei Chen, Yajun Wang, Yang Yuan","year":"2013","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen et al., 2017] Yuxin Chen, Jean-Michel Renders, Morteza Haghir Chehreghani, and Andreas Krause. Effi- cient online learning for optimizing value of information: Theory and application to interactive troubleshoot- ing. In Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, UAI, 2017.","title":"Efficient online learning for optimizing value of information: Theory and application to interactive troubleshooting","authors":"Yuxin Chen, Jean-Michel Renders, Morteza Haghir Chehreghani, Andreas Krause","year":"2017","venue":"Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, UAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Codecá et al., 2017] Lara Codecá, Raphaël Frank, Sébastien Faye, and Thomas Engel. Luxembourg SUMO Traffic (LuST) Scenario: Traffic Demand Evalua- tion. IEEE Intelligent Transportation Systems Magazine, 9(2):52-63, 2017.","title":"Luxembourg SUMO Traffic (LuST) Scenario: Traffic Demand Evaluation","authors":"Lara Codecá, Raphaël Frank, Sébastien Faye, Thomas Engel","year":"2017","venue":"IEEE Intelligent Transportation Systems Magazine","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dijkstra, 1959] Edsger W Dijkstra. A note on two prob- lems in connexion with graphs. Numerische mathematik, 1(1):269-271, 1959.","title":"A note on two problems in connexion with graphs","authors":"W Edsger, Dijkstra","year":"1959","venue":"Numerische mathematik","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gai et al., 2012] Yi Gai, Bhaskar Krishnamachari, and Rahul Jain. Combinatorial network optimization with un- known variables: Multi-armed bandits with linear rewards and individual observations. IEEE/ACM Transactions on Networking (TON), 20(5):1466-1478, 2012.","title":"Combinatorial network optimization with unknown variables: Multi-armed bandits with linear rewards and individual observations","authors":"Yi Gai, Bhaskar Krishnamachari, Rahul Jain","year":"2012","venue":"IEEE/ACM Transactions on Networking (TON)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Graepel et al., 2010] Thore Graepel, Joaquin Quiñonero Candela, Thomas Borchert, and Ralf Herbrich. Web- scale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine. In ICML, pages 13-20, 2010.","title":"Webscale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine","authors":"Thore Graepel, Joaquin Quiñonero Candela, Thomas Borchert, Ralf Herbrich","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Joulani et al., 2013] Pooria Joulani, Andras Gyorgy, and Csaba Szepesvári. Online learning under delayed feed- back. In International Conference on Machine Learning, pages 1453-1461, 2013.","title":"Online learning under delayed feedback","authors":"Joulani","year":"2013","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kaufmann and others, 2018] Emilie Kaufmann et al. On bayesian index policies for sequential resource allocation. The Annals of Statistics, 46(2):842-865, 2018.","title":"On bayesian index policies for sequential resource allocation","authors":"Emilie Kaufmann","year":"2018","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kaufmann et al., 2012a] Emilie Kaufmann, Olivier Cappé, and Aurélien Garivier. On bayesian upper confidence bounds for bandit problems. In Artificial intelligence and statistics, pages 592-600, 2012.","title":"On bayesian upper confidence bounds for bandit problems","authors":"Kaufmann","year":"2012","venue":"Artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kaufmann et al., 2012b] Emilie Kaufmann, Nathaniel Ko- rda, and Rémi Munos. Thompson sampling: An asymptot- ically optimal finite-time analysis. In Algorithmic Learn- ing Theory -23rd International Conference, ALT, pages 199-213, 2012.","title":"Thompson sampling: An asymptotically optimal finite-time analysis","authors":"Kaufmann","year":"2012","venue":"Algorithmic Learning Theory -23rd International Conference, ALT","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Osband and Van Roy, 2017] Ian Osband and Benjamin Van Roy. Why is posterior sampling better than optimism for reinforcement learning? In Proceedings of the 34th In- ternational Conference on Machine Learning-Volume 70, pages 2701-2710. JMLR. org, 2017.","title":"Why is posterior sampling better than optimism for reinforcement learning?","authors":"Van Osband, Ian Roy, Benjamin Osband, Roy Van","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Reverdy et al., 2014] Paul B Reverdy, Vaibhav Srivastava, and Naomi Ehrich Leonard. Modeling human decision making in generalized gaussian multiarmed bandits. Pro- ceedings of the IEEE, 102(4):544-571, 2014.","title":"Modeling human decision making in generalized gaussian multiarmed bandits","authors":"Vaibhav Paul B Reverdy, Naomi Ehrich Srivastava, Leonard","year":"2014","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Russo et al., 2018] Daniel J. Russo, Benjamin Van Roy, Ab- bas Kazerouni, Ian Osband, and Zheng Wen. A tutorial on thompson sampling. Foundations and Trends R in Ma- chine Learning, 11(1):1-96, 2018.","title":"A tutorial on thompson sampling","authors":"J Daniel, Benjamin Russo, Abbas Van Roy, Ian Kazerouni, Zheng Osband, Wen","year":"2018","venue":"Foundations and Trends R in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sachenbacher et al., 2011] Martin Sachenbacher, Martin Leucker, Andreas Artmeier, and Julian Haselmayr. Ef- ficient energy-optimal routing for electric vehicles. In Twenty-fifth AAAI conference on artificial intelligence, 2011.","title":"Efficient energy-optimal routing for electric vehicles","authors":"Martin Sachenbacher, Martin Leucker, Andreas Artmeier, Julian Haselmayr","year":"2011","venue":"Twenty-fifth AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Thompson, 1933] W.R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285--294, 1933.","title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","authors":"W R Thompson","year":"1933","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang and Chen, 2018] Siwei Wang and Wei Chen. Thomp- son sampling for combinatorial semi-bandits. In Inter- national Conference on Machine Learning, pages 5101- 5109, 2018.","title":"Thompson sampling for combinatorial semi-bandits","authors":"Siwei Wang, Wei Chen","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2015] Xinkai Wu, David Freese, Alfredo Cabr- era, and William A Kitch. Electric vehicles' energy con- sumption measurement and estimation. Transportation Research Part D: Transport and Environment, 34:52-67, 2015.","title":"Electric vehicles' energy consumption measurement and estimation","authors":"Xinkai Wu, David Freese, Alfredo Cabrera, William A Kitch","year":"2015","venue":"Transportation Research Part D: Transport and Environment","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}