{"type":"ScholarlyArticle","title":"BiCo-Net: Regress Globally, Match Locally for Robust 6D Pose Estimation","authors":[{"name":"Zelin Xu","slug":"zelin-xu","given":"Zelin","family":"Xu","affiliation":"South China University of Technology"},{"name":"Yichen Zhang","slug":"yichen-zhang","given":"Yichen","family":"Zhang","affiliation":"South China University of Technology"},{"name":"Ke Chen","slug":"ke-chen","given":"Ke","family":"Chen","affiliation":"South China University of Technology; Peng Cheng Laboratory"},{"name":"Kui Jia","slug":"kui-jia","given":"Kui","family":"Jia","affiliation":"South China University of Technology; Peng Cheng Laboratory"}],"abstract":"The challenges of learning a robust 6D pose function lie in 1) severe occlusion and 2) systematic noises in depth images. Inspired by the success of point-pair features, the goal of this paper is to recover the 6D pose of an object instance segmented from RGB-D images by locally matching pairs of oriented points between the model and camera space. To this end, we propose a novel Bi-directional Correspondence Mapping Network (BiCo-Net) to first generate point clouds guided by a typical pose regression, which can thus incorporate pose-sensitive information to optimize generation of local coordinates and their normal vectors. As pose predictions via geometric computation only rely on one single pair of local oriented points, our BiCo-Net can achieve robustness against sparse and occluded point clouds. An ensemble of redundant pose predictions from locally matching and direct pose regression further refines final pose output against noisy observations. Experimental results on three popularly benchmarking datasets can verify that our method can achieve state-of-the-art performance, especially for the more challenging severe occluded scenes. Source codes are available at https://github.com/Gorilla-Lab-SCUT/BiCo-Net.","doi":"10.24963/ijcai.2022/210","url":"https://ijcai.org/proceedings/2022/210","pdf":"https://ijcai.org/proceedings/2022/0210.pdf","markdown":"https://ijcai.org/proceedings/2022/210.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/210","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":210,"pages":{"first":1509,"last":1515},"keywords":["Robotics: Perception","Computer Vision: 3D Computer Vision"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986596","length_sec":90,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984739","length_sec":342,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Arun et al., 1987] K Somani Arun, Thomas S Huang, and Steven D Blostein. Least-squares fitting of two 3-d point sets. TPAMI, 1987.","title":"Least-squares fitting of two 3-d point sets","authors":"Thomas S Somani Arun, Steven D Huang, Blostein","year":"1987","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barron and Malik, 2013] Jonathan T Barron and Jitendra Malik. Intrinsic scene properties from a single rgb-d im- age. In CVPR, 2013.","title":"Intrinsic scene properties from a single rgb-d image","authors":"Jonathan T Barron, Jitendra Malik","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brachmann et al., 2014] Eric Brachmann, Alexander Krull, Frank Michel, Stefan Gumhold, Jamie Shotton, and Carsten Rother. Learning 6d object pose estimation using 3d object coordinates. In ECCV, 2014.","title":"Learning 6d object pose estimation using 3d object coordinates","authors":"Eric Brachmann, Alexander Krull, Frank Michel, Stefan Gumhold, Jamie Shotton, Carsten Rother","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Wei Chen, Xi Jia, Hyung Jin Chang, Jin- ming Duan, and Ales Leonardis. G2l-net: Global to local network for real-time 6d pose estimation with embedding vector features. In CVPR, 2020.","title":"G2l-net: Global to local network for real-time 6d pose estimation with embedding vector features","authors":"Wei Chen, Xi Jia, Jin Hyung, Jinming Chang, Ales Duan, Leonardis","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Doumanoglou et al., 2016] Andreas Doumanoglou, Rigas Kouskouridas, Sotiris Malassiotis, and Tae-Kyun Kim. Recovering 6d object pose and predicting next-best-view in the crowd. In CVPR, 2016.","title":"Recovering 6d object pose and predicting next-best-view in the crowd","authors":"Andreas Doumanoglou, Rigas Kouskouridas, Sotiris Malassiotis, Tae-Kyun Kim","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Drost et al., 2010] Bertram Drost, Markus Ulrich, Nassir Navab, and Slobodan Ilic. Model globally, match locally: Efficient and robust 3d object recognition. In CVPR, 2010.","title":"Model globally, match locally: Efficient and robust 3d object recognition","authors":"Bertram Drost, Markus Ulrich, Nassir Navab, Slobodan Ilic","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, 2017.","title":"Kaiming He, Georgia Gkioxari","authors":"He","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2020] Yisheng He, Wei Sun, Haibin Huang, Jian- ran Liu, Haoqiang Fan, and Jian Sun. Pvn3d: A deep point-wise 3d keypoints voting network for 6dof pose es- timation. In CVPR, 2020.","title":"Pvn3d: A deep point-wise 3d keypoints voting network for 6dof pose estimation","authors":"Yisheng He, Wei Sun, Haibin Huang, Jianran Liu, Haoqiang Fan, Jian Sun","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2021] Yisheng He, Haibin Huang, Haoqiang Fan, Qifeng Chen, and Jian Sun. Ffb6d: A full flow bidirec- tional fusion network for 6d pose estimation. CVPR, 2021.","title":"Ffb6d: A full flow bidirectional fusion network for 6d pose estimation","authors":"Yisheng He, Haibin Huang, Haoqiang Fan, Qifeng Chen, Jian Sun","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hinterstoisser et al., 2011] Stefan Hinterstoisser, Stefan Holzer, Cedric Cagniart, Slobodan Ilic, Kurt Konolige, Nassir Navab, and Vincent Lepetit. Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In ICCV, 2011.","title":"Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes","authors":"Stefan Hinterstoisser, Stefan Holzer, Cedric Cagniart, Slobodan Ilic, Kurt Konolige, Nassir Navab, Vincent Lepetit","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kehl et al., 2016] Wadim Kehl, Fausto Milletari, Federico Tombari, Slobodan Ilic, and Nassir Navab. Deep learning of local rgb-d patches for 3d object detection and 6d pose estimation. In ECCV, 2016.","title":"Deep learning of local rgb-d patches for 3d object detection and 6d pose estimation","authors":"Wadim Kehl, Fausto Milletari, Federico Tombari, Slobodan Ilic, Nassir Navab","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kehl et al., 2017] Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic, and Nassir Navab. Ssd-6d: Mak- ing rgb-based 3d detection and 6d pose estimation great again. In ICCV, 2017.","title":"Ssd-6d: Making rgb-based 3d detection and 6d pose estimation great again","authors":"Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic, Nassir Navab","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lepetit et al., 2008] Vincent Lepetit, Francesc Moreno- Noguer, and P. Fua. Epnp: An accurate o(n) solution to the pnp problem. IJCV, 2008.","title":"Epnp: An accurate o(n) solution to the pnp problem","authors":"Vincent Lepetit, Francesc Moreno-Noguer, P Fua","year":"2008","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2018] Chi Li, Jin Bai, and Gregory D Hager. A unified framework for multi-view multi-class object pose estimation. In ECCV, 2018.","title":"A unified framework for multi-view multi-class object pose estimation","authors":"Li","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liebelt et al., 2008] Joerg Liebelt, Cordelia Schmid, and Klaus Schertler. independent object class detection using 3d feature maps. In CVPR, 2008.","title":"independent object class detection using 3d feature maps","authors":"Liebelt","year":"2008","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Oberweger et al., 2018] Markus Oberweger, Mahdi Rad, and Vincent Lepetit. Making deep heatmaps robust to par- tial occlusions for 3d object pose estimation. In ECCV, 2018.","title":"Making deep heatmaps robust to partial occlusions for 3d object pose estimation","authors":"Markus Oberweger, Rad Mahdi, Vincent Lepetit","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Park et al., 2019] Kiru Park, Timothy Patten, and Markus Vincze. Pix2pose: Pixel-wise coordinate regression of ob- jects for 6d pose estimation. In ICCV, 2019.","title":"Pix2pose: Pixel-wise coordinate regression of objects for 6d pose estimation","authors":"Kiru Park, Timothy Patten, Markus Vincze","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pavlakos et al., 2017] Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G Derpanis, and Kostas Dani- ilidis. 6-dof object pose from semantic keypoints. In ICRA, 2017.","title":"6-dof object pose from semantic keypoints","authors":"Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G Derpanis, Kostas Daniilidis","year":"2017","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Peng et al., 2019] Sida Peng, Yuan Liu, Qixing Huang, Xi- aowei Zhou, and Hujun Bao. Pvnet: Pixel-wise voting network for 6dof pose estimation. In CVPR, 2019.","title":"Pvnet: Pixel-wise voting network for 6dof pose estimation","authors":"Sida Peng, Yuan Liu, Qixing Huang, Xiaowei Zhou, Hujun Bao","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rad and Lepetit, 2017] Mahdi Rad and Vincent Lepetit. Bb8: A scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth. In ICCV, 2017.","title":"Mahdi Rad and Vincent Lepetit. Bb8: A scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth","authors":"Rad, Lepetit","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2020] Chen Song, Jiaru Song, and Qixing Huang. Hybridpose: 6d object pose estimation under hy- brid representations. In CVPR, 2020.","title":"Hybridpose: 6d object pose estimation under hybrid representations","authors":"Chen Song, Jiaru Song, Qixing Huang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sun et al., 2010] Min Sun, Gary Bradski, Bing-Xin Xu, and Silvio Savarese. Depth-encoded hough voting for joint ob- ject detection and shape recovery. In ECCV, 2010.","title":"Depth-encoded hough voting for joint object detection and shape recovery","authors":"Sun","year":"2010","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sundermeyer et al., 2018] Martin Sundermeyer, Zoltan- Csaba Marton, Maximilian Durner, Manuel Brucker, and Rudolph Triebel. Implicit 3d orientation learning for 6d object detection from rgb images. In ECCV, 2018.","title":"Implicit 3d orientation learning for 6d object detection from rgb images","authors":"Martin Sundermeyer, Zoltan-Csaba Marton, Maximilian Durner, Manuel Brucker, Rudolph Triebel","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tekin et al., 2018] Bugra Tekin, Sudipta N Sinha, and Pas- cal Fua. Real-time seamless single shot 6d object pose prediction. In CVPR, 2018.","title":"Real-time seamless single shot 6d object pose prediction","authors":"Bugra Tekin, N Sudipta, Pascal Sinha, Fua","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2019a] Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, and Silvio Savarese. Densefusion: 6d object pose estimation by iter- ative dense fusion. In CVPR, 2019.","title":"Densefusion: 6d object pose estimation by iterative dense fusion","authors":"Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2019b] He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas J Guibas. Normalized object coordinate space for category- level 6d object pose and size estimation. In CVPR, 2019.","title":"Normalized object coordinate space for categorylevel 6d object pose and size estimation","authors":"Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xiang et al., 2018] Yu Xiang, Tanner Schmidt, Venkatra- man Narayanan, and Dieter Fox. Posecnn: A convolu- tional neural network for 6d object pose estimation in clut- tered scenes. RSS, 2018.","title":"Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes","authors":"Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, Dieter Fox","year":"2018","venue":"RSS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu et al., 2018] Danfei Xu, Dragomir Anguelov, and Ashesh Jain. Pointfusion: Deep sensor fusion for 3d bounding box estimation. In CVPR, 2018.","title":"Pointfusion: Deep sensor fusion for 3d bounding box estimation","authors":"Xu","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2021] Guangyuan Zhou, Huiqun Wang, Jiaxin Chen, and Di Huang. Pr-gcn: A deep graph convolutional network with point refinement for 6d pose estimation. In ICCV, 2021.","title":"Pr-gcn: A deep graph convolutional network with point refinement for 6d pose estimation","authors":"Guangyuan Zhou, Huiqun Wang, Jiaxin Chen, Di Huang","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}