{"type":"ScholarlyArticle","title":"Crowd-Steer: Realtime Smooth and Collision-Free Robot Navigation in Densely Crowded Scenarios Trained using High-Fidelity Simulation","authors":[{"name":"Jing Liang","slug":"jing-liang","given":"Jing","family":"Liang","affiliation":"University of Maryland , College Park , USA"},{"name":"Utsav Patel","slug":"utsav-patel","given":"Utsav","family":"Patel","affiliation":"University of Maryland , College Park , USA"},{"name":"Adarsh Jagan Sathyamoorthy","slug":"adarsh-jagan-sathyamoorthy","given":"Adarsh Jagan","family":"Sathyamoorthy","affiliation":"University of Maryland , College Park , USA"},{"name":"Dinesh Manocha","slug":"dinesh-manocha","given":"Dinesh","family":"Manocha","affiliation":"University of Maryland , College Park , USA"}],"abstract":"We present a novel high fidelity 3-D simulator that significantly reduces the sim-to-real gap for collision avoidance in dense crowds using  Deep Reinforcement Learning (DRL). Our simulator models realistic crowd and pedestrian behaviors, along with friction, sensor noise and delays in the simulated robot model. We also describe a technique to incrementally control the randomness and complexity of training scenarios to achieve better convergence and generalization capabilities. We demonstrate the effectiveness of our simulator by training a policy that fuses data from multiple  perception sensors such as a 2-D lidar and a depth camera to detect pedestrians and computes smooth, collision-free velocities. Our novel reward function  and multi-sensor formulation results in smooth and unobtrusive navigation. We have evaluated the learned policy on two differential drive robots and evaluate its performance in new dense crowd scenarios, narrow corridors, T and L-junctions, etc. We observe that our algorithm outperforms prior dynamic navigation techniques in terms of metrics such as success rate, trajectory length, mean time to goal, and smoothness.","doi":"10.24963/ijcai.2020/583","url":"https://ijcai.org/proceedings/2020/583","pdf":"https://ijcai.org/proceedings/2020/0583.pdf","markdown":"https://ijcai.org/proceedings/2020/583.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/583","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Robotics","track":"Main track","number":583,"pages":{"first":4221,"last":4228},"keywords":["Robotics: Motion and Path Planning","Robotics: Learning in Robotics","Planning and Scheduling: Robot Planning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25282/file_25282.m3u8","length_sec":282,"provider":"hls"}],"references":[{"position":1,"raw":"[Alonso-Mora et al., 2010] Javier Alonso-Mora, Andreas Breitenmoser, Martin Rufli, Paul A. Beardsley, and Roland Siegwart. Optimal reciprocal collision avoidance for mul- tiple non-holonomic robots. In DARS, 2010.","title":"Optimal reciprocal collision avoidance for multiple non-holonomic robots","authors":"Alonso-Mora","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alonso-Mora et al., 2018] J. Alonso-Mora, P. Beardsley, and R. Siegwart. Cooperative collision avoidance for nonholonomic robots. IEEE Transactions on Robotics, 34(2):404-420, 2018.","title":"Cooperative collision avoidance for nonholonomic robots","authors":"Alonso-Mora","year":"2018","venue":"IEEE Transactions on Robotics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bareiss and van den Berg, 2015] Daman Bareiss and Jur van den Berg. Generalized reciprocal collision avoid- ance. The International Journal of Robotics Research, 34(12):1501-1514, 2015.","title":"Generalized reciprocal collision avoidance","authors":"Van Bareiss, Den Berg ; Jur Van Den, Berg","year":"2015","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Braun et al., 2003] A. Braun, S. R. Musse, L. P. L. de Oliveira, and B. E. J. Bodmann. Modeling individual be- haviors in crowd simulation. In Proceedings 11th IEEE In- ternational Workshop on Program Comprehension, pages 143-148, May 2003.","title":"Modeling individual behaviors in crowd simulation","authors":"A Braun, S R Musse, L P L De Oliveira, B E J Bodmann","year":"2003","venue":"Proceedings 11th IEEE International Workshop on Program Comprehension","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2017] Yu Fan Chen, Miao Liu, Michael Everett, and Jonathan P How. Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In ICRA, pages 285-292. IEEE, 2017.","title":"Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning","authors":"Miao Yu Fan Chen, Michael Liu, Jonathan P Everett, How","year":"2017","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2019] Changan Chen, Yuejiang Liu, Sven Kreiss, and Alexandre Alahi. Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning. In ICRA, pages 6015-6022. IEEE, 2019.","title":"Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning","authors":"Changan Chen, Yuejiang Liu, Sven Kreiss, Alexandre Alahi","year":"2019","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Curtis et al., 2016] Sean Curtis, Andrew Best, and Dinesh Manocha. Menge: A modular framework for simulating crowd movement. Collective Dynamics, 1:1-40, 2016.","title":"Menge: A modular framework for simulating crowd movement","authors":"Sean Curtis, Andrew Best, Dinesh Manocha","year":"2016","venue":"Collective Dynamics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Everett et al., 2018] Michael Everett, Yu Fan Chen, and Jonathan P How. Motion planning among dynamic, decision-making agents with deep reinforcement learning. In IROS, pages 3052-3059. IEEE, 2018.","title":"Motion planning among dynamic, decision-making agents with deep reinforcement learning","authors":"Michael Everett, Yu, Fan Chen, Jonathan P How","year":"2018","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fan et al., 2018a] Tingxiang Fan, Xinjing Cheng, Jia Pan, Dinesh Manocha, and Ruigang Yang. CrowdMove: Au- tonomous Mapless Navigation in Crowded Scenarios. arXiv e-prints, page arXiv:1807.07870, Jul 2018.","title":"CrowdMove: Autonomous Mapless Navigation in Crowded Scenarios","authors":"Fan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fan et al., 2018b] Tingxiang Fan, Pinxin Long, Wenxi Liu, and Jia Pan. Fully Distributed Multi-Robot Collision Avoidance via Deep Reinforcement Learning for Safe and Efficient Navigation in Complex Scenarios. arXiv e-prints, page arXiv:1808.03841, Aug 2018.","title":"Tingxiang Fan, Pinxin Long","authors":"Fan","year":"2018","venue":"Fully Distributed Multi-Robot Collision Avoidance via Deep Reinforcement Learning for Safe and Efficient Navigation in Complex Scenarios","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fox et al., 1997] D. Fox, W. Burgard, and S. Thrun. The dynamic window approach to collision avoidance. IEEE Robotics Automation Magazine, 4(1):23-33, March 1997.","title":"The dynamic window approach to collision avoidance","authors":"D Fox, W Burgard, S Thrun","year":"1997","venue":"IEEE Robotics Automation Magazine","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Helbing and Molnar, 1995] Dirk Helbing and Peter Molnar. Social force model for pedestrian dynamics. Physical re- view E, 51(5):4282, 1995.","title":"Social force model for pedestrian dynamics","authors":"Molnar Helbing, Dirk Helbing, Peter Molnar","year":"1995","venue":"Physical review E","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kim et al., 2018] Y. Kim, J. Jang, and S. Yun. End-to- end deep learning for autonomous navigation of mobile robot. In 2018 IEEE International Conference on Con- sumer Electronics (ICCE), pages 1-6, Jan 2018.","title":"End-toend deep learning for autonomous navigation of mobile robot","authors":"Y Kim, J Jang, S Yun","year":"2018","venue":"IEEE International Conference on Consumer Electronics (ICCE)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Long et al., 2017] Pinxin Long, Tingxiang Fan, Xinyi Liao, Wenxi Liu, Hao Zhang, and Jia Pan. Towards Op- timally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning. arXiv e-prints, page arXiv:1709.10082, Sep 2017.","title":"Pinxin Long, Tingxiang Fan","authors":"Long","year":"2017","venue":"Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Narain et al., 2009] Rahul Narain, Abhinav Golas, Sean Curtis, and Ming C. Lin. Aggregate dynamics for dense crowd simulation. ACM Trans. Graph., 28(5):1-8, De- cember 2009.","title":"Aggregate dynamics for dense crowd simulation","authors":"Rahul Narain, Abhinav Golas, Sean Curtis, Ming C Lin","year":"2009","venue":"ACM Trans. Graph","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Narang et al., 2015] Sahil Narang, Andrew Best, Sean Cur- tis, and Dinesh Manocha. Generating pedestrian trajec- tories consistent with the fundamental diagram based on physiological and psychological factors. PLOS ONE, 10:e0117856, 04 2015.","title":"Generating pedestrian trajectories consistent with the fundamental diagram based on physiological and psychological factors","authors":"Narang","year":"2015","venue":"PLOS ONE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Neunert et al., 2016] M. Neunert, T. Boaventura, and J. Buchli. Why Off-The-Shelf Physics Simulators Fail In Evaluating Feedback Controller Performance -A Case Study For Quadrupedal Robots: Proceedings of the 19th International Conference on CLAWAR 2016, pages 464- 472. 10 2016.","title":"Why Off-The-Shelf Physics Simulators Fail In Evaluating Feedback Controller Performance -A Case Study For Quadrupedal Robots","authors":"M Neunert, T Boaventura, J Buchli","year":"2016","venue":"Proceedings of the 19th International Conference on CLAWAR 2016","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pelechano et al., 2007] N. Pelechano, J. M. Allbeck, and N. I. Badler. Controlling individual agents in high-density crowd simulation. In Proceedings of the 2007 ACM SIG- GRAPH/Eurographics Symposium on Computer Anima- tion, SCA '07, page 99-108, Goslar, DEU, 2007. Euro- graphics Association.","title":"Controlling individual agents in high-density crowd simulation","authors":"N Pelechano, J M Allbeck, N I Badler","year":"2007","venue":"Proceedings of the 2007 ACM SIG-GRAPH/Eurographics Symposium on Computer Animation, SCA '07","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pfeiffer et al., 2016] Mark Pfeiffer, Michael Schaeuble, Juan Nieto, Roland Siegwart, and Cesar Cadena. From Perception to Decision: A Data-driven Approach to End- to-end Motion Planning for Autonomous Ground Robots. arXiv e-prints, page arXiv:1609.07910, Sep 2016.","title":"From Perception to Decision: A Data-driven Approach to Endto-end Motion Planning for Autonomous Ground Robots","authors":"Mark Pfeiffer, Michael Schaeuble, Juan Nieto, Roland Siegwart, Cesar Cadena","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pfeiffer et al., 2017] M. Pfeiffer, M. Schaeuble, J. Nieto, R. Siegwart, and C. Cadena. From perception to deci- sion: A data-driven approach to end-to-end motion plan- ning for autonomous ground robots. In 2017 IEEE Inter- national Conference on Robotics and Automation (ICRA), pages 1527-1533, May 2017.","title":"From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots","authors":"M Pfeiffer, M Schaeuble, J Nieto, R Siegwart, C Cadena","year":"2017","venue":"2017 IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Proxi- mal Policy Optimization Algorithms. arXiv e-prints, page arXiv:1707.06347, Jul 2017.","title":"Proximal Policy Optimization Algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tai et al., 2018] L. Tai, J. Zhang, M. Liu, and W. Burgard. Socially compliant navigation through raw depth inputs with generative adversarial imitation learning. In ICRA, pages 1111-1117, May 2018.","title":"Socially compliant navigation through raw depth inputs with generative adversarial imitation learning","authors":"L Tai, J Zhang, M Liu, W Burgard","year":"2018","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tilove, 1990] R. B. Tilove. Local obstacle avoidance for mobile robots based on the method of artificial poten- tials. In Proceedings., IEEE International Conference on Robotics and Automation, pages 566-571 vol.1, May 1990.","title":"Local obstacle avoidance for mobile robots based on the method of artificial potentials","authors":"R B Tilove","year":"1990","venue":"IEEE International Conference on Robotics and Automation","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[van den Berg et al., 2009] Jur P. van den Berg, Stephen J. Guy, M. Chiao Lin, and Dinesh Manocha. Reciprocal n- body collision avoidance. In ISRR, 2009.","title":"Reciprocal nbody collision avoidance","authors":"Van Den Berg","year":"2009","venue":"ISRR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xie et al., 2017] Linhai Xie, Sen Wang, Andrew Markham, and Niki Trigoni. Towards Monocular Vision based Ob- stacle Avoidance through Deep Reinforcement Learning. arXiv e-prints, page arXiv:1706.09829, Jun 2017.","title":"Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning","authors":"Xie","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2019] Wenhao Yu, Visak C. V. Kumar, Greg Turk, and C. Karen Liu. Sim-to-real transfer for biped locomo- tion. ArXiv, abs/1903.01390, 2019.","title":"Sim-to-real transfer for biped locomotion","authors":"Wenhao Yu, C V Visak, Greg Kumar, C Turk, Karen Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}