{"type":"ScholarlyArticle","title":"Image Gradient-based Joint Direct Visual Odometry for Stereo Camera","authors":[{"name":"Jianke Zhu","slug":"jianke-zhu","given":"Jianke","family":"Zhu","affiliation":"College of Computer Science , Zhejiang University , Hangzhou , China; Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies"}],"abstract":"Visual odometry is an important research problem for computer vision and robotics. In general, the feature-based visual odometry methods heavily rely on the accurate correspondences between local salient points, while the direct approaches could make full use of whole image and perform dense 3D reconstruction simultaneously. However, the direct visual odometry usually suffers from the drawback of getting stuck at local optimum especially with large displacement, which may lead to the inferior results. To tackle this critical problem, we propose a novel scheme for stereo odometry in this paper, which is able to improve the convergence with more accurate pose. The key of our approach is a dual Jacobian optimization that is fused into a multi-scale pyramid scheme. Moreover, we introduce a gradient-based feature representation, which enjoys the merit of being robust to illumination changes. Furthermore, a joint direct odometry approach is proposed to incorporate the information from the last frame and previous keyframes. We have conducted the experimental evaluation on the challenging KITTI odometry benchmark, whose promising results show that the proposed algorithm is very effective for stereo visual odometry.","doi":"10.24963/ijcai.2017/636","url":"https://ijcai.org/proceedings/2017/636","pdf":"https://ijcai.org/proceedings/2017/0636.pdf","markdown":"https://ijcai.org/proceedings/2017/636.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/636","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Robotics and Vision","track":"Main track","number":636,"pages":{"first":4558,"last":4564},"keywords":["Robotics and Vision: Robotics and Vision","Robotics and Vision: Vision and Perception"],"videos":[],"references":[{"position":1,"raw":"[Badino et al., 2013] Hernan Badino, Akihiro Yamamoto, and Takeo Kanade. Visual odometry by multi-frame fea- ture integration. In International Workshop on Computer Vision for Autonomous Driving (CVAD 13), 2013.","title":"Visual odometry by multi-frame feature integration","authors":"Hernan Badino, Akihiro Yamamoto, Takeo Kanade","year":"2013","venue":"International Workshop on Computer Vision for Autonomous Driving (CVAD 13)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baker and Matthews, 2004] Simon Baker and Iain Matthews. Lucas-kanade 20 years on: A unifying framework. Int'l J. Computer Vision, 56(3):221-255, March 2004.","title":"Lucas-kanade 20 years on: A unifying framework","authors":"Matthews Baker, Simon Baker, Iain Matthews","year":"2004","venue":"Int'l J. Computer Vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Benhimane and Malis, 2004] Selim Benhimane and Ezio Malis. Real-time image-based tracking of planes using ef- ficient second-order minimization. In 2004 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems, pages 943-948, 2004.","title":"Real-time image-based tracking of planes using efficient second-order minimization","authors":"Malis Benhimane, Selim Benhimane, Ezio Malis","year":"2004","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Black and Rangarajan, 1996] Michael J. Black and Anand Rangarajan. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int'l J. Computer Vision, 19(1):57-91, 1996.","title":"On the unification of line processes, outlier rejection, and robust statistics with applications in early vision","authors":"J Michael, Anand Black, Rangarajan","year":"1996","venue":"Int'l J. Computer Vision","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Comport et al., 2010] Andrew I. Comport, Ezio Malis, and Patrick Rives. Real-time quadrifocal visual odometry. I. J. Robotics Res., 29(2-3):245-266, 2010.","title":"Real-time quadrifocal visual odometry","authors":"Andrew I Comport, Ezio Malis, Patrick Rives","year":"2010","venue":"I. J. Robotics Res","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Corke et al., 2007] Peter Corke, Jorge Lobo, and Jorge Dias. An introduction to inertial and visual sensing. I. J. Robotics Res., 26(6):519-535, 2007.","title":"An introduction to inertial and visual sensing","authors":"Corke","year":"2007","venue":"I. J. Robotics Res","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Crivellaro and Lepetit, 2014] Alberto Crivellaro and Vin- cent Lepetit. Robust 3d tracking with descriptor fields. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, pages 3414-3421, 2014.","title":"Alberto Crivellaro and Vincent Lepetit. Robust 3d tracking with descriptor fields","authors":"Lepetit Crivellaro","year":"2014","venue":"IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cvisic and Petrovic, 2015] Igor Cvisic and Ivan Petrovic. Stereo odometry based on careful feature selection and tracking. In 2015 European Conference on Mobile Robots, ECMR 2015, pages 1-6, 2015.","title":"Igor Cvisic and Ivan Petrovic. Stereo odometry based on careful feature selection and tracking","authors":"Petrovic Cvisic","year":"2015","venue":"European Conference on Mobile Robots, ECMR 2015","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Engel et al., 2014] Jakob Engel, Thomas Schöps, and Daniel Cremers. LSD-SLAM: large-scale direct monoc- ular SLAM. In Computer Vision -ECCV 2014 -13th Eu- ropean Conference, pages 834-849, 2014.","title":"LSD-SLAM: large-scale direct monocular SLAM","authors":"Engel","year":"2014","venue":"Computer Vision -ECCV 2014 -13th European Conference","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Engel et al., 2015] Jakob Engel, Jörg Stückler, and Daniel Cremers. Large-scale direct SLAM with stereo cameras. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2015, pages 1935-1942, 2015.","title":"Large-scale direct SLAM with stereo cameras","authors":"Engel","year":"2015","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2015","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fischler and Bolles, 1981] Martin A. Fischler and Robert C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. CACM, 24(6):381-395, 1981.","title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","authors":"Martin A Fischler, Robert C Bolles","year":"1981","venue":"CACM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Forster et al., 2014] Christian Forster, Matia Pizzoli, and Davide Scaramuzza. SVO: fast semi-direct monocular vi- sual odometry. In 2014 IEEE International Conference on Robotics and Automation, ICRA 2014, pages 15-22, 2014.","title":"SVO: fast semi-direct monocular visual odometry","authors":"Forster","year":"2014","venue":"IEEE International Conference on Robotics and Automation, ICRA 2014","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Geiger et al., 2012] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on Com- puter Vision and Pattern Recognition (CVPR), 2012.","title":"Are we ready for autonomous driving? the kitti vision benchmark suite","authors":"Andreas Geiger, Philip Lenz, Raquel Urtasun","year":"2012","venue":"Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hirschmüller, 2008] Heiko Hirschmüller. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell., 30(2):328-341, 2008.","title":"Stereo processing by semiglobal matching and mutual information","authors":"Heiko Hirschmüller","year":"2008","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kalman, 1960] Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, 82(Series D):35-45, 1960.","title":"A new approach to linear filtering and prediction problems","authors":"Rudolph Emil, Kalman","year":"1960","venue":"Transactions of the ASME-Journal of Basic Engineering","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Klein and Murray, 2007] Georg Klein and David Murray. Parallel tracking and mapping for small AR workspaces. In Proc. Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'07), Nara, Japan, November 2007.","title":"Parallel tracking and mapping for small AR workspaces","authors":"Murray ; Klein, Georg Klein, David Murray","year":"2007","venue":"Proc. Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'07)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lucas and Kanade, 1981] Bruce D. Lucas and Takeo Kanade. An iterative image registration technique with an application to stereo vision. In IJCAI81, pages 674-679, 1981.","title":"An iterative image registration technique with an application to stereo vision","authors":"Bruce D Lucas, Takeo Kanade","year":"1981","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/1981/127"},{"position":18,"raw":"[Ma et al., 2004] Yi Ma, Stefano Soatto, Jana Kosecka, and S. Shankar Sastry. An Invitation to 3-D Vision: From Im- ages to Geometric Models. Springer-Verlag, New York, 2004.","title":"An Invitation to 3-D Vision: From Images to Geometric Models","authors":"Yi Ma, Stefano Soatto, Jana Kosecka, S Shankar Sastry","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mur-Artal et al., 2015] Raul Mur-Artal, J. M. M. Montiel, and Juan D. Tardós. ORB-SLAM: A versatile and ac- curate monocular SLAM system. IEEE Trans. Robotics, 31(5):1147-1163, 2015.","title":"ORB-SLAM: A versatile and accurate monocular SLAM system","authors":"Raul Mur-Artal, J M M Montiel, Juan D Tardós","year":"2015","venue":"IEEE Trans. Robotics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Newcombe et al., 2011] Richard A. Newcombe, Steven Lovegrove, and Andrew J. Davison. DTAM: dense track- ing and mapping in real-time. In IEEE International Con- ference on Computer Vision, pages 2320-2327, 2011.","title":"DTAM: dense tracking and mapping in real-time","authors":"Richard A Newcombe, Steven Lovegrove, Andrew J Davison","year":"2011","venue":"IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Persson et al., 2015] Mikael Persson, Tommaso Piccini, Michael Felsberg, and Rudolf Mester. Robust stereo vi- sual odometry from monocular techniques. In 2015 IEEE Intelligent Vehicles Symposium, IV 2015, pages 686-691, 2015.","title":"Robust stereo visual odometry from monocular techniques","authors":"Mikael Persson, Tommaso Piccini, Michael Felsberg, Rudolf Mester","year":"2015","venue":"IEEE Intelligent Vehicles Symposium","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Song et al., 2016] Shiyu Song, Manmohan Chandraker, and Clark C. Guest. High accuracy monocular SFM and scale correction for autonomous driving. IEEE Trans. Pattern Anal. Mach. Intell., 38(4):730-743, 2016.","title":"High accuracy monocular SFM and scale correction for autonomous driving","authors":"Shiyu Song, Manmohan Chandraker, Clark C Guest","year":"2016","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sturm et al., 2012] Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A bench- mark for the evaluation of RGB-D SLAM systems. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2012, pages 573-580, 2012.","title":"A benchmark for the evaluation of RGB-D SLAM systems","authors":"Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, Daniel Cremers","year":"2012","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2012","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Triggs et al., 1999] Bill Triggs, Philip F. McLauchlan, Richard I. Hartley, and Andrew W. Fitzgibbon. Bundle adjustment -A modern synthesis. In Vision Algorithms: Theory and Practice, International Workshop on Vision Algorithms, held during ICCV '99, pages 298-372, 1999.","title":"Bundle adjustment -A modern synthesis","authors":"Bill Triggs, Philip F Mclauchlan, Richard I Hartley, Andrew W Fitzgibbon","year":"1999","venue":"Vision Algorithms: Theory and Practice, International Workshop on Vision Algorithms, held during ICCV '99","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang and Singh, 2015] Ji Zhang and Sanjiv Singh. Visual- lidar odometry and mapping: low-drift, robust, and fast. In IEEE International Conference on Robotics and Automa- tion, ICRA 2015, pages 2174-2181, 2015.","title":"Visuallidar odometry and mapping: low-drift, robust, and fast","authors":"Singh Zhang, Ji Zhang, Sanjiv Singh","year":"2015","venue":"IEEE International Conference on Robotics and Automation, ICRA 2015","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2014] Ji Zhang, Michael Kaess, and Sanjiv Singh. Real-time depth enhanced monocular odometry. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4973-4980, 2014.","title":"Real-time depth enhanced monocular odometry","authors":"Ji Zhang, Michael Kaess, Sanjiv Singh","year":"2014","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}