{"type":"ScholarlyArticle","title":"Forecast the Plausible Paths in Crowd Scenes","authors":[{"name":"Hang Su","slug":"hang-su","given":"Hang","family":"Su","affiliation":"Tsinghua National Lab for Information Science and Technology; State Key Lab of Intelligent Technology and Systems Center for Bio-Inspired Computing Research Department of Computer Science and Technology , Tsinghua University , Beijing , China"},{"name":"Jun Zhu","slug":"jun-zhu","given":"Jun","family":"Zhu","affiliation":"Tsinghua National Lab for Information Science and Technology; State Key Lab of Intelligent Technology and Systems Center for Bio-Inspired Computing Research Department of Computer Science and Technology , Tsinghua University , Beijing , China"},{"name":"Yinpeng Dong","slug":"yinpeng-dong","given":"Yinpeng","family":"Dong","affiliation":"Tsinghua National Lab for Information Science and Technology; State Key Lab of Intelligent Technology and Systems Center for Bio-Inspired Computing Research Department of Computer Science and Technology , Tsinghua University , Beijing , China"},{"name":"Bo Zhang","slug":"bo-zhang","given":"Bo","family":"Zhang","affiliation":"Tsinghua National Lab for Information Science and Technology; State Key Lab of Intelligent Technology and Systems Center for Bio-Inspired Computing Research Department of Computer Science and Technology , Tsinghua University , Beijing , China"}],"abstract":"Forecasting the future plausible paths of pedestrians in crowd scenes is of wide applications, but it still remains as a challenging task due to the complexities and uncertainties of crowd motions. To address these issues, we propose to explore the inherent crowd dynamics via a social-aware recurrent Gaussian process model, which facilitates the path prediction by taking advantages of the interplay between the rich prior knowledge and motion uncertainties. Specifically, we derive a social-aware LSTM to explore the crowd dynamic, resulting in a hidden feature embedding the rich prior in massive data. Afterwards, we integrate the descriptor into deep Gaussian processes with motion uncertainties appropriately harnessed. Crowd motion forecasting is implemented by regressing relative motion against the current positions, yielding the predicted paths based on a functional object associated with a distribution. Extensive experiments on public datasets demonstrate that our method obtains the state-of-the-art performance in both structured and unstructured scenes by exploring the complex and uncertain motion patterns, even if the occlusion is serious or the observed trajectories are noisy.","doi":"10.24963/ijcai.2017/386","url":"https://ijcai.org/proceedings/2017/386","pdf":"https://ijcai.org/proceedings/2017/0386.pdf","markdown":"https://ijcai.org/proceedings/2017/386.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/386","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":386,"pages":{"first":2772,"last":2778},"keywords":["Machine Learning: Deep Learning","Robotics and Vision: Vision and Perception"],"videos":[],"references":[{"position":1,"raw":"[Alahi et al., 2016] Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Sil- vio Savarese. Social lstm: Human trajectory prediction in crowded spaces. In Computer Vision and Pattern Recog- nition (CVPR), IEEE Conference on, pages 1-7, 2016.","title":"Social lstm: Human trajectory prediction in crowded spaces","authors":"Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, Silvio Savarese","year":"2016","venue":"Computer Vision and Pattern Recognition (CVPR), IEEE Conference on","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dai et al., 2016] Zhenwen Dai, Andreas Damianou, Javier González, and Neil Lawrence. Variational auto-encoded deep Gaussian processes. In International Conference on Learning Representations (ICLR), pages 1-14, 2016.","title":"Variational auto-encoded deep Gaussian processes","authors":"Dai","year":"2016","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Damianou and Lawrence, 2013] Andreas Damianou and Neil Lawrence. Deep gaussian processes. In Proceedings of the Sixteenth International Conference on Artificial In- telligence and Statistics (AISTATS), pages 207-215, 2013.","title":"Deep gaussian processes","authors":"Lawrence ; Damianou, Andreas Damianou, Neil Lawrence","year":"2013","venue":"Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics (AISTATS)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ellis et al., 2009a] David Ellis, Eric Sommerlade, and Ian Reid. Modelling pedestrian trajectory patterns with gaus- sian processes. In Computer Vision Workshops (ICCVW), IEEE 12th International Conference on, pages 1229-1234. IEEE, 2009.","title":"Modelling pedestrian trajectory patterns with gaussian processes","authors":"Ellis","year":"2009","venue":"Computer Vision Workshops (ICCVW), IEEE 12th International Conference on","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ellis et al., 2009b] David Ellis, Eric Sommerlade, and Ian Reid. Modelling pedestrian trajectory patterns with gaus- sian processes. In Computer Vision Workshops (ICCVW), 2009 IEEE 12th International Conference on, pages 1229- 1234. IEEE, 2009.","title":"Modelling pedestrian trajectory patterns with gaussian processes","authors":"Ellis","year":"2009","venue":"IEEE 12th International Conference on","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Godoy et al., 2016] Julio Erasmo Godoy, Ioannis Karamouzas, Stephen J Guy, and Maria L Gini. Im- plicit coordination in crowded multi-agent navigation. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI 2016), pages 2487-2493, 2016.","title":"Implicit coordination in crowded multi-agent navigation","authors":"Julio Erasmo Godoy, Ioannis Karamouzas, Stephen J Guy, Maria L Gini","year":"2016","venue":"Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI 2016)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Helbing and Johansson, 2011] Dirk Helbing and Anders Jo- hansson. Pedestrian, crowd and evacuation dynamics. Springer, 2011.","title":"Pedestrian, crowd and evacuation dynamics","authors":"Johansson Helbing, Anders Johansson","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kitani et al., 2012] Kris Kitani, Brian Ziebart, James Bag- nell, and Martial Hebert. Activity forecasting. European Conference on Computer Vision (ECCV), pages 201-214, 2012.","title":"Activity forecasting","authors":"Kris Kitani, Brian Ziebart, James Bagnell, Martial Hebert","year":"2012","venue":"European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li and Fu, 2014] Kang Li and Yun Fu. Prediction of hu- man activity by discovering temporal sequence patterns. Pattern Analysis and Machine Intelligence, IEEE Trans- actions on (TPAMI), 36(8):1644-1657, 2014.","title":"Prediction of human activity by discovering temporal sequence patterns","authors":"Kang Li, Yun Fu","year":"2014","venue":"Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2014] Weixin Li, Vijay Mahadevan, and Nuno Vasconcelos. Anomaly detection and localization in crowded scenes. Pattern Analysis and Machine Intel- ligence, IEEE Transactions on (TPAMI), 36(1):18-32, 2014.","title":"Anomaly detection and localization in crowded scenes. Pattern Analysis and Machine Intelligence","authors":"Weixin Li, Vijay Mahadevan, Nuno Vasconcelos","year":"2014","venue":"IEEE Transactions on (TPAMI)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lipton, 2015] Zachary C Lipton. A critical review of recur- rent neural networks for sequence learning. arXiv preprint arXiv:1506.00019, 2015.","title":"A critical review of recurrent neural networks for sequence learning","authors":"Zachary C Lipton","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Long et al., 2016] Pinxin Long, Wenxi Liu, and Jia Pan. Deep-learned collision avoidance policy for distributed multi-agent navigation. arXiv preprint arXiv:1609.06838, 2016.","title":"Deep-learned collision avoidance policy for distributed multi-agent navigation","authors":"Long","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Robicquet et al., 2016] Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese. Learn- ing social etiquette: Human trajectory understanding in crowded scenes. In European Conference on Computer Vision (ECCV), pages 549-565, 2016.","title":"Learning social etiquette: Human trajectory understanding in crowded scenes","authors":"Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, Silvio Savarese","year":"2016","venue":"European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Shao et al., 2014] Jing Shao, Chen Change Loy, and Xiao- gang Wang. Scene-independent group profiling in crowd. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 2227-2234, 2014.","title":"Scene-independent group profiling in crowd","authors":"Jing Shao, Chen Change Loy, Xiaogang Wang","year":"2014","venue":"Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Srivastava et al., 2015] Nitish Srivastava, Elman Mansi- mov, and Ruslan Salakhudinov. Unsupervised learning of video representations using lstms. In Proceedings of the 32nd International Conference on Machine Learning (ICML), pages 843-852, 2015.","title":"Unsupervised learning of video representations using lstms","authors":"Nitish Srivastava, Elman Mansimov, Ruslan Salakhudinov","year":"2015","venue":"Proceedings of the 32nd International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Su et al., 2016] Hang Su, Yinpeng Dong, Jun Zhu, Haibin Ling, and Bo Zhang. Crowd scene understanding with co- herent recurrent neural networks. In International Joint Conference on Artificial Intelligence (IJCAI), pages 3469- 3476, 2016.","title":"Crowd scene understanding with coherent recurrent neural networks","authors":"Su","year":"2016","venue":"International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2016/490"},{"position":17,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems (NIPS), pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in neural information processing systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Titsias and Lawrence, 2010] Michalis K Titsias and Neil D Lawrence. Bayesian gaussian process latent variable model. In The 13th International Conference on Artifi- cial Intelligence and Statistics (AISTATS), volume 9, pages 844-851, 2010.","title":"Bayesian gaussian process latent variable model","authors":"K Michalis, Neil D Titsias, Lawrence","year":"2010","venue":"The 13th International Conference on Artificial Intelligence and Statistics (AISTATS)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Titsias, 2009] Michalis K Titsias. Variational learning of inducing variables in sparse gaussian processes. In The 12th International Conference on Artificial Intelligence and Statistics (AISTATS), volume 5, pages 567-574, 2009.","title":"Variational learning of inducing variables in sparse gaussian processes","authors":"K Michalis, Titsias","year":"2009","venue":"The 12th International Conference on Artificial Intelligence and Statistics (AISTATS)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yi et al., 2016] Shuai Yi, Hongsheng Li, and Xiaogang Wang. Pedestrian behavior understanding and prediction with deep neural networks. In European Conference on Computer Vision, pages 263-279. Springer, 2016.","title":"Pedestrian behavior understanding and prediction with deep neural networks","authors":"Shuai Yi, Hongsheng Li, Xiaogang Wang","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhou et al., 2011] Bolei Zhou, Xiaogang Wang, and Xiaoou Tang. Random field topic model for semantic region anal- ysis in crowded scenes from tracklets. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 3441-3448, 2011.","title":"Random field topic model for semantic region analysis in crowded scenes from tracklets","authors":"Zhou","year":"2011","venue":"Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhou et al., 2012] Bolei Zhou, Xiaoou Tang, and Xiaogang Wang. Coherent filtering: detecting coherent motions from crowd clutters. In European Conference on Computer Vi- sion (ECCV), pages 857-871, 2012.","title":"Coherent filtering: detecting coherent motions from crowd clutters","authors":"Zhou","year":"2012","venue":"European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhou et al., 2015] Bolei Zhou, Xiaoou Tang, and Xiaogang Wang. Learning collective crowd behaviors with dynamic pedestrian-agents. International Journal of Computer Vi- sion (IJCV), 111(1):50-68, 2015.","title":"Learning collective crowd behaviors with dynamic pedestrian-agents","authors":"Zhou","year":"2015","venue":"International Journal of Computer Vision (IJCV)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ziebart et al., 2008] Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, and Anind K Dey. Maximum entropy inverse reinforcement learning. In The Twenty-Third AAAI Conference on Artificial Intelligence (AAAI), pages 1433- 1438, 2008.","title":"Maximum entropy inverse reinforcement learning","authors":"Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey","year":"2008","venue":"The Twenty-Third AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}