{"type":"ScholarlyArticle","title":"Randomized Adversarial Imitation Learning for Autonomous Driving","authors":[{"name":"MyungJae Shin","slug":"myungjae-shin","given":"MyungJae","family":"Shin","affiliation":"Chung-Ang University , Seoul , Republic of Korea"},{"name":"Joongheon Kim","slug":"joongheon-kim","given":"Joongheon","family":"Kim","affiliation":"Chung-Ang University , Seoul , Republic of Korea"}],"abstract":"With the evolution of various advanced driver assistance system (ADAS) platforms, the design of autonomous driving system is becoming more complex and safety-critical. The autonomous driving system simultaneously activates multiple ADAS functions; and thus it is essential to coordinate various ADAS functions. This paper proposes a randomized adversarial imitation learning (RAIL) method that imitates the coordination of autonomous vehicle equipped with advanced sensors. The RAIL policies are trained through derivative-free optimization for the decision maker that coordinates the proper ADAS functions, e.g., smart cruise control and lane keeping system. Especially, the proposed method is also able to deal with the LIDAR data and makes decisions in complex multi-lane highways and multi-agent environments.","doi":"10.24963/ijcai.2019/638","url":"https://ijcai.org/proceedings/2019/638","pdf":"https://ijcai.org/proceedings/2019/0638.pdf","markdown":"https://ijcai.org/proceedings/2019/638.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/638","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning Applications","track":"Main track","number":638,"pages":{"first":4590,"last":4596},"keywords":["Machine Learning Applications: Applications of Reinforcement Learning","Multidisciplinary Topics and Applications: Transportation"],"videos":[],"references":[{"position":1,"raw":"[Ahmed, 1999] Kazi I. Ahmed. Modeling drivers' accelera- tion and lane changing behavior. PhD thesis, MIT, 1999.","title":"Modeling drivers' acceleration and lane changing behavior","authors":"I Kazi, Ahmed","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Finn et al., 2016a] Chelsea Finn, Paul Christiano, Pieter Abbeel, and Sergey Levine. A connection between gener- ative adversarial networks, inverse reinforcement learning, and energy-based models. In NIPS Workshop on Adversar- ial Training, 2016.","title":"A connection between generative adversarial networks, inverse reinforcement learning, and energy-based models","authors":"Finn","year":"2016","venue":"NIPS Workshop on Adversarial Training","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Finn et al., 2016b] Chelsea Finn, Sergey Levine, and Pieter Abbeel. Guided cost learning: Deep inverse optimal con- trol via policy optimization. In ICML, 2016.","title":"Guided cost learning: Deep inverse optimal control via policy optimization","authors":"Finn","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gipps, 1986] Peter G Gipps. A model for the structure of lane-changing decisions. Transportation Research Part B: Methodological, 1986.","title":"A model for the structure of lane-changing decisions","authors":"G Peter, Gipps","year":"1986","venue":"Transportation Research Part B: Methodological","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guo et al., 2018] Yijie Guo, Junhyuk Oh, Satinder Singh, and Honglak Lee. Generative adversarial self-imitation learning. arXiv preprint arXiv:1812.00950, 2018.","title":"Generative adversarial self-imitation learning","authors":"Yijie Guo, Junhyuk Oh, Satinder Singh, Honglak Lee","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Henderson et al., 2017a] Peter Henderson, Wei-Di Chang, Pierre-Luc Bacon, David Meger, Joelle Pineau, and Doina Precup. OptionGAN: Learning joint reward-policy options using generative adversarial inverse reinforcement learn- ing. arXiv preprint arXiv:1709.06683, 2017.","title":"OptionGAN: Learning joint reward-policy options using generative adversarial inverse reinforcement learning","authors":"Henderson","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Henderson et al., 2017b] Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. Deep reinforcement learning that matters. arXiv preprint arXiv:1709.06560, 2017.","title":"Deep reinforcement learning that matters","authors":"Henderson","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ho and Ermon, 2016] Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In NIPS, 2016.","title":"Generative adversarial imitation learning","authors":"Jonathan Ho, Stefano Ermon","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hoel et al., 2018] Carl-Johan Hoel, Krister Wolff, and Leo Laine. Automated speed and lane change decision mak- ing using deep reinforcement learning. arXiv preprint arXiv:1803.10056, 2018.","title":"Automated speed and lane change decision making using deep reinforcement learning","authors":"Carl-Johan Hoel, Krister Wolff, Leo Laine","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Islam et al., 2017] Riashat Islam, Peter Henderson, Maziar Gomrokchi, and Doina Precup. Reproducibility of bench- marked deep reinforcement learning tasks for continuous control. arXiv preprint arXiv:1708.04133, 2017.","title":"Reproducibility of benchmarked deep reinforcement learning tasks for continuous control","authors":"Islam","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Korssen et al., 2018] Tim Korssen, Victor Dolk, Joanna van de Mortel-Fronczak, Michel Reniers, and Maurice Heemels. Systematic model-based design and implemen- tation of supervisors for advanced driver assistance sys- tems. IEEE TITS, 2018.","title":"Systematic model-based design and implementation of supervisors for advanced driver assistance systems","authors":"Tim Korssen, Victor Dolk, Joanna Van De Mortel-Fronczak, Michel Reniers, Maurice Heemels","year":"2018","venue":"IEEE TITS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kostrikov et al., 2019] Ilya Kostrikov, Kumar Krishna Agrawal, Debidatta Dwibedi, Sergey Levine, and Jonathan Tompson. Discriminator-actor-critic: Address- ing sample inefficiency and reward bias in adversarial imitation learning. In ICLR, 2019.","title":"Discriminator-actor-critic: Addressing sample inefficiency and reward bias in adversarial imitation learning","authors":"Ilya Kostrikov, Krishna Kumar, Debidatta Agrawal, Sergey Dwibedi, Jonathan Levine, Tompson","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mania et al., 2018] Horia Mania, Aurelia Guy, and Ben- jamin Recht. Simple random search provides a compet- itive approach to reinforcement learning. arXiv preprint arXiv:1803.07055, 2018.","title":"Simple random search provides a competitive approach to reinforcement learning","authors":"Horia Mania, Aurelia Guy, Benjamin Recht","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mao et al., 2017] Xudong Mao, Qing Li, Haoran Xie, Ray- mond Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In ICCV, 2017.","title":"Least squares generative adversarial networks","authors":"Xudong Mao, Qing Li, Haoran Xie, Raymond Lau, Zhen Wang, Stephen Paul Smolley","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Matyas, 1965] J Matyas. Random optimization. Automation and Remote Control, 26(2):246-253, 1965.","title":"Random optimization","authors":"; J Matyas, Matyas","year":"1965","venue":"Automation and Remote Control","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Min and Kim, 2018] Kyushik Min and Hayoung Kim. Deep Q-learning based high level driving policy determination. In IV, 2018.","title":"Deep Q-learning based high level driving policy determination","authors":"Min, Kim ;, Kyushik Min, Hayoung Kim","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidje- land, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mukadam et al., 2017] Mustafa Mukadam, Akansel Cos- gun, Alireza Nakhaei, and Kikuo Fujimura. Tactical de- cision making for lane changing with deep reinforcement learning. In NIPS Workshop MLITS, 2017.","title":"Tactical decision making for lane changing with deep reinforcement learning","authors":"Mustafa Mukadam, Akansel Cosgun, Alireza Nakhaei, Kikuo Fujimura","year":"2017","venue":"NIPS Workshop MLITS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nagabandi et al., 2018] Anusha Nagabandi, Gregory Kahn, Ronald Fearing, and Sergey Levine. Neural network dy- namics for model-based deep reinforcement learning with model-free fine-tuning. In ICRA, 2018.","title":"Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning","authors":"Anusha Nagabandi, Gregory Kahn, Ronald Fearing, Sergey Levine","year":"2018","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pan et al., 2018] Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos Theodorou, and Byron Boots. Agile autonomous driving using end-to- end deep imitation learning. In RSS, 2018.","title":"Agile autonomous driving using end-toend deep imitation learning","authors":"Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos Theodorou, Byron Boots","year":"2018","venue":"RSS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pomerleau, 1989] Dean Pomerleau. Alvinn: An au- tonomous land vehicle in a neural network. In NIPS, 1989.","title":"Alvinn: An autonomous land vehicle in a neural network","authors":"Dean Pomerleau","year":"1989","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pomerleau, 1991] Dean Pomerleau. Rapidly adapting artifi- cial neural networks for autonomous navigation. In NIPS, 1991.","title":"Rapidly adapting artificial neural networks for autonomous navigation","authors":"Dean Pomerleau","year":"1991","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rajeswaran et al., 2017] Aravind Rajeswaran, Kendall Lowrey, Emanuel V Todorov, and Sham Kakade. Towards generalization and simplicity in continuous control. In NIPS, 2017.","title":"Towards generalization and simplicity in continuous control","authors":"Aravind Rajeswaran, Kendall Lowrey, Sham Emanuel V Todorov, Kakade","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Salimans et al., 2017] Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864, 2017.","title":"Evolution strategies as a scalable alternative to reinforcement learning","authors":"Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, Ilya Sutskever","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sallab et al., 2016] Ahmad El Sallab, Mohammed Abdou, Etienne Perot, and Senthil Yogamani. End-to-end deep reinforcement learning for lane keeping assist. arXiv preprint arXiv:1612.04340, 2016.","title":"End-to-end deep reinforcement learning for lane keeping assist","authors":"Ahmad El Sallab, Mohammed Abdou, Etienne Perot, Senthil Yogamani","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schulman et al., 2015] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In ICML, 2015.","title":"Trust region policy optimization","authors":"John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, Philipp Moritz","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris Maddi- son, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Pan- neershelvam, Marc Lanctot, et al. Mastering the game of Go with deep neural networks and tree search. Nature, 2016.","title":"Mastering the game of Go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris Maddison, Arthur Guez, Laurent Sifre, George Van Den, Julian Driessche, Ioannis Schrittwieser, Veda Antonoglou, Marc Panneershelvam, Lanctot","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Ziebart et al., 2008] Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, and Anind K Dey. Maximum entropy inverse reinforcement learning. In AAAI, 2008.","title":"Maximum entropy inverse reinforcement learning","authors":"Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey","year":"2008","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}