{"type":"ScholarlyArticle","title":"Learning Transferable UAV for Forest Visual Perception","authors":[{"name":"Lyujie Chen","slug":"lyujie-chen","given":"Lyujie","family":"Chen","affiliation":"Beijing National Research Center for Information Science and Technology (BNRist ); Department of Computer Science and Technology , Tsinghua University , Bejing , China"},{"name":"Wufan Wang","slug":"wufan-wang","given":"Wufan","family":"Wang","affiliation":"Beijing National Research Center for Information Science and Technology (BNRist ); Department of Computer Science and Technology , Tsinghua University , Bejing , China"},{"name":"Jihong Zhu","slug":"jihong-zhu","given":"Jihong","family":"Zhu","affiliation":"Beijing National Research Center for Information Science and Technology (BNRist ); Department of Computer Science and Technology , Tsinghua University , Bejing , China"}],"abstract":"In this paper, we propose a new pipeline of training a monocular UAV to fly a collision-free trajectory along the dense forest trail. As gathering high-precision images in the real world is expensive and the off-the-shelf dataset has some deficiencies, we collect a new dense forest trail dataset in a variety of simulated environment in Unreal Engine. Then we formulate visual perception of forests as a classification problem. A ResNet-18 model is trained to decide the moving direction frame by frame. To transfer the learned strategy to the real world, we construct a ResNet-18 adaptation model via multi-kernel maximum mean discrepancies to leverage the relevant labelled data and alleviate the discrepancy between simulated and real environment. Simulation and real-world flight with a variety of appearance and environment changes are both tested. The ResNet-18 adaptation and its variant model achieve the best result of 84.08% accuracy in reality.","doi":"10.24963/ijcai.2018/678","url":"https://ijcai.org/proceedings/2018/678","pdf":"https://ijcai.org/proceedings/2018/0678.pdf","markdown":"https://ijcai.org/proceedings/2018/678.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/678","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Robotics","track":"Main track","number":678,"pages":{"first":4883,"last":4889},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Deep Learning","Robotics: Vision and Perception","Robotics: Learning in Robotics"],"videos":[],"references":[{"position":1,"raw":"[Barry and Tedrake, 2015] Andrew J Barry and Russ Tedrake. Pushbroom stereo for high-speed navigation in cluttered environments. In Robotics and automation (icra), 2015 ieee international conference on, pages 3046-3052. IEEE, 2015.","title":"Pushbroom stereo for high-speed navigation in cluttered environments","authors":"J Andrew, Russ Barry, Tedrake","year":"2015","venue":"Robotics and automation (icra), 2015 ieee international conference on","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Byrne et al., 2006] Jeffrey Byrne, Martin Cosgrove, and Raman Mehra. Stereo based obstacle detection for an un- manned air vehicle. In Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Con- ference on, pages 2830-2835. IEEE, 2006.","title":"Stereo based obstacle detection for an unmanned air vehicle","authors":"Jeffrey Byrne, Martin Cosgrove, Raman Mehra","year":"2006","venue":"Proceedings 2006 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Daftry et al., 2016a] Shreyansh Daftry, J Andrew Bagnell, and Martial Hebert. Learning transferable policies for monocular reactive mav control. In International Sym- posium on Experimental Robotics, pages 3-11. Springer, 2016.","title":"Learning transferable policies for monocular reactive mav control","authors":"Daftry","year":"2016","venue":"International Symposium on Experimental Robotics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Daftry et al., 2016b] Shreyansh Daftry, Sam Zeng, Arbaaz Khan, Debadeepta Dey, Narek Melik-Barkhudarov, J An- drew Bagnell, and Martial Hebert. Robust monocular flight in cluttered outdoor environments. arXiv preprint arXiv:1604.04779, 2016.","title":"Robust monocular flight in cluttered outdoor environments","authors":"Daftry","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dey et al., 2016] Debadeepta Dey, Kumar Shaurya Shankar, Sam Zeng, Rupesh Mehta, M Talha Agcayazi, Christopher Eriksen, Shreyansh Daftry, Martial Hebert, and J Andrew Bagnell. Vision and learning for delib- erative monocular cluttered flight. In Field and Service Robotics, pages 391-409. Springer, 2016.","title":"Vision and learning for deliberative monocular cluttered flight","authors":"Dey","year":"2016","venue":"Field and Service Robotics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Giusti et al., 2016] Alessandro Giusti, Jerome Guzzi, Dan Ciresan, Fang-Lin He, Juan Pablo Rodriguez, Flavio Fontana, Matthias Faessler, Christian Forster, Jurgen Schmidhuber, Gianni Di Caro, Davide Scaramuzza, and Luca Gambardella. A machine learning approach to vi- sual perception of forest trails for mobile robots. IEEE Robotics and Automation Letters, 2016.","title":"A machine learning approach to visual perception of forest trails for mobile robots","authors":"Alessandro Giusti, Jerome Guzzi, Dan Ciresan, Fang-Lin He, Juan, Pablo Rodriguez, Flavio Fontana, Matthias Faessler, Christian Forster, Jurgen Schmidhuber, Di Gianni, Davide Caro, Luca Scaramuzza, Gambardella","year":"2016","venue":"IEEE Robotics and Automation Letters","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gretton et al., 2012] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(Mar):723-773, 2012.","title":"A kernel two-sample test","authors":"Arthur Gretton, Karsten M Borgwardt, J Malte, Bernhard Rasch, Alexander Schölkopf, Smola","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpass- ing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026-1034, 2015.","title":"Delving deep into rectifiers: Surpassing human-level performance on imagenet classification","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"He","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jia et al., 2014] Yangqing Jia, Evan Shelhamer, Jeff Don- ahue, Sergey Karayev, Jonathan Long, Ross Girshick, Ser- gio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Jia","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Levine et al., 2016] Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. End-to-end training of deep visuomotor policies. Journal of Machine Learning Re- search, 17(39):1-40, 2016.","title":"End-to-end training of deep visuomotor policies","authors":"Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel","year":"2016","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Long et al., 2015] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International Confer- ence on Machine Learning, pages 97-105, 2015.","title":"Learning transferable features with deep adaptation networks","authors":"Mingsheng Long, Yue Cao, Jianmin Wang, Michael Jordan","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Long et al., 2016] Mingsheng Long, Jianmin Wang, and Michael I Jordan. Deep transfer learning with joint adap- tation networks. arXiv preprint arXiv:1605.06636, 2016.","title":"Deep transfer learning with joint adaptation networks","authors":"Mingsheng Long, Jianmin Wang, Michael I Jordan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ross et al., 2013] Stéphane Ross, Narek Melik- Barkhudarov, Kumar Shaurya Shankar, Andreas Wendel, Debadeepta Dey, J Andrew Bagnell, and Martial Hebert. Learning monocular reactive uav control in cluttered natural environments. In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages 1765-1772. IEEE, 2013.","title":"Learning monocular reactive uav control in cluttered natural environments","authors":"Stéphane Ross, Narek Melik-Barkhudarov, Shaurya Kumar, Andreas Shankar, Debadeepta Wendel, Andrew Dey, Martial Bagnell, Hebert","year":"2013","venue":"Robotics and Automation (ICRA), 2013 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shah et al., 2017] Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and Service Robotics, 2017.","title":"Airsim: High-fidelity visual and physical simulation for autonomous vehicles","authors":"Shital Shah, Debadeepta Dey, Chris Lovett, Ashish Kapoor","year":"2017","venue":"Field and Service Robotics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Szegedy et al., 2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabi- novich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.","title":"Going deeper with convolutions","authors":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Szegedy et al., 2017] Christian Szegedy, Sergey Ioffe, Vin- cent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI, pages 4278-4284, 2017.","title":"Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning","authors":"Szegedy","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Weiss et al., 2016] Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. A survey of transfer learning. Journal of Big Data, 3(1):9, 2016.","title":"A survey of transfer learning","authors":"Karl Weiss, Taghi M Khoshgoftaar, Dingding Wang","year":"2016","venue":"Journal of Big Data","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yang and Pollefeys, 2003] Ruigang Yang and Marc Polle- feys. Multi-resolution real-time stereo on commodity graphics hardware. In Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer So- ciety Conference on, volume 1, pages I-I. IEEE, 2003.","title":"Multi-resolution real-time stereo on commodity graphics hardware","authors":"Pollefeys Yang","year":"2003","venue":"Proceedings. 2003 IEEE Computer Society Conference on","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}