{"type":"ScholarlyArticle","title":"HCR-Net: A Hybrid of Classification and Regression Network for Object Pose Estimation","authors":[{"name":"Zairan Wang","slug":"zairan-wang","given":"Zairan","family":"Wang","affiliation":"SAIT -China Lab , Samsung Research Institute China -Beijing (SRC-B )"},{"name":"Weiming Li","slug":"weiming-li","given":"Weiming","family":"Li","affiliation":"SAIT -China Lab , Samsung Research Institute China -Beijing (SRC-B )"},{"name":"Yueying Kao","slug":"yueying-kao","given":"Yueying","family":"Kao","affiliation":"SAIT -China Lab , Samsung Research Institute China -Beijing (SRC-B )"},{"name":"Dongqing Zou","slug":"dongqing-zou","given":"Dongqing","family":"Zou","affiliation":"SAIT -China Lab , Samsung Research Institute China -Beijing (SRC-B )"},{"name":"Qiang Wang","slug":"qiang-wang","given":"Qiang","family":"Wang","affiliation":"SAIT -China Lab , Samsung Research Institute China -Beijing (SRC-B )"},{"name":"Minsu Ahn","slug":"minsu-ahn","given":"Minsu","family":"Ahn","affiliation":"Samsung Advanced Institute of Technology (SAIT)"},{"name":"Sunghoon Hong","slug":"sunghoon-hong","given":"Sunghoon","family":"Hong","affiliation":"Samsung Advanced Institute of Technology (SAIT)"}],"abstract":"Object pose estimation from a single image is a fundamental and challenging problem in computer vision and robotics. Generally, current methods treat pose estimation as a classification or a regression problem. However, regression based methods usually suffer from the issue of imbalanced training data, while classification methods are difficult to discriminate nearby poses. In this paper, a hybrid CNN model, which we call it HCR-Net that integrates both a classification network and a regression network, is proposed to deal with these issues. Our model is inspired by that regression methods can get better accuracy on homogeneously distributed datasets while classification methods are more effective for coarse quantization of the poses even if the dataset is not well balanced. The classification methods and the regression methods essentially complement each other. Thus we integrate both them into a neural network in a hybrid fashion and train it end-to-end with two novel loss functions. As a result, our method surpass the state-of-the-art methods, even with imbalanced training data and  much less data augmentation. The experimental results on the challenging Pascal3D+ database demonstrate that our method outperforms the state-of-the-arts significantly, achieving improvements on ACC and AVP metrics up to 4% and 6%, respectively.","doi":"10.24963/ijcai.2018/141","url":"https://ijcai.org/proceedings/2018/141","pdf":"https://ijcai.org/proceedings/2018/0141.pdf","markdown":"https://ijcai.org/proceedings/2018/141.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/141","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":141,"pages":{"first":1014,"last":1020},"keywords":["Computer Vision: 2D and 3D Computer Vision","Computer Vision: Computer Vision","Computer Vision: Perception"],"videos":[],"references":[{"position":1,"raw":"[Aubry et al., 2014] Mathieu Aubry, Daniel Maturana, Alexei Efros, Bryan Russell, and Josef Sivic. Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models. In CVPR, 2014.","title":"Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models","authors":"Aubry Mathieu, Daniel Maturana, Alexei Efros, Bryan Russell, Josef Sivic","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2013] K. Chen, S. Gong, T. Xiang, and C. C. Loy. Cumulative attribute space for age and crowd density estimation. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 2467-2474, June 2013.","title":"Cumulative attribute space for age and crowd density estimation","authors":"K Chen, S Gong, T Xiang, C C Loy","year":"2013","venue":"2013 IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng et al., 2016] Yanhua Cheng, Xin Zhao, Rui Cai, Zhiwei Li, Kaiqi Huang, and Yong Rui. Semi-supervised multimodal deep learning for rgb-d object recognition. In International Joint Conference on Artificial Intelligence, pages 3345-3351, 2016.","title":"Semi-supervised multimodal deep learning for rgb-d object recognition","authors":"Yanhua Cheng, Xin Zhao, Rui Cai, Zhiwei Li, Kaiqi Huang, Yong Rui","year":"2016","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/473"},{"position":4,"raw":"[Deng et al., 2009] J. Deng, W. Dong, R. Socher, L. J. Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, June 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"J Deng, W Dong, R Socher, L J Li, Kai Li, Li Fei-Fei","year":"2009","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Elhoseiny et al., 2016] Mohamed Elhoseiny, Tarek Elgaaly, Amr Bakry, and Ahmed M Elgammal. A comparative analysis and study of multiview cnn models for joint object categorization and pose estimation. pages 888-897, 2016.","title":"A comparative analysis and study of multiview cnn models for joint object categorization and pose estimation","authors":"Mohamed Elhoseiny, Tarek Elgaaly, Amr Bakry, Ahmed M Elgammal","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"Everingham et al., ] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal- network.org/challenges/VOC/voc2012/workshop/index.html.","title":"The PASCAL Visual Object Classes Challenge 2012","authors":"Everingham","year":null,"venue":"VOC2012) Results","doi":null,"url":"http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html","internal_url":null},{"position":7,"raw":"[Fenzi et al., 2013] M. Fenzi, L. Leal-Taixe, B. Rosenhahn, and J. Ostermann. Class generative models based on feature regression for pose estimation of object categories. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 755-762, June 2013.","title":"Class generative models based on feature regression for pose estimation of object categories","authors":"M Fenzi, L Leal-Taixe, B Rosenhahn, J Ostermann","year":"2013","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Girshick et al., 2014] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 580-587, June 2014.","title":"Rich feature hierarchies for accurate object detection and semantic segmentation","authors":"R Girshick, J Donahue, T Darrell, J Malik","year":"2014","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jia et al., 2014] Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Jia","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097-1105.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"Curran Associates, Inc., 2012.","title":null,"authors":"Curran Associates, Inc","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lim et al., 2013] J. J. Lim, H. Pirsiavash, and A. Torralba. Parsing ikea objects: Fine pose estimation. In 2013 IEEE International Conference on Computer Vision, pages 2992-2999, Dec 2013.","title":"Parsing ikea objects: Fine pose estimation","authors":"J J Lim, H Pirsiavash, A Torralba","year":"2013","venue":"IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR, 2017.","title":"Feature pyramid networks for object detection","authors":"Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, Serge Belongie","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mahendran et al., 2017] Siddharth Mahendran, Haider Ali, and Rene Vidal. 3d pose regression using convolutional neural networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017.","title":"3d pose regression using convolutional neural networks","authors":"Haider Siddharth Mahendran, Rene Ali, Vidal","year":"2017","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Massa et al., 2014] Francisco Massa, Mathieu Aubry, and Renaud Marlet. Convolutional neural networks for joint object detection and pose estimation: A comparative study. Computer Science, 19(2a):412-417, 2014.","title":"Convolutional neural networks for joint object detection and pose estimation: A comparative study","authors":"Massa","year":"2014","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Massa et al., 2016] Francisco Massa, Renaud Marlet, and Mathieu Aubry. Crafting a multi-task CNN for viewpoint estimation. In Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, September 19- 22, 2016, 2016.","title":"Crafting a multi-task CNN for viewpoint estimation","authors":"Massa","year":"2016","venue":"Proceedings of the British Machine Vision Conference 2016, BMVC 2016","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pavlakos et al., 2017] Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G Derpanis, and Kostas Daniilidis. 6-dof object pose from semantic keypoints. In International Conference on Robotics and Automation (ICRA), 2017.","title":"6-dof object pose from semantic keypoints","authors":"Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G Derpanis, Kostas Daniilidis","year":"2017","venue":"International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Redondo-Cabrera et al., 2016] Carolina Redondo-Cabrera, Roberto J. López-Sastre, Yu Xiang, Tinne Tuytelaars, and Silvio Savarese. Pose estimation errors, the ultimate diagnosis. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision -ECCV 2016, pages 118-134, Cham, 2016. Springer International Publishing.","title":"Pose estimation errors, the ultimate diagnosis","authors":"Redondo-Cabrera","year":"2016","venue":"Computer Vision -ECCV 2016","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Simonyan and Zisserman, 2014] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"K Simonyan, A Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Su et al., 2015] H. Su, C. R. Qi, Y. Li, and L. J. Guibas. Render for cnn: Viewpoint estimation in images using cnns trained with rendered 3d model views. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 2686-2694, Dec 2015.","title":"Render for cnn: Viewpoint estimation in images using cnns trained with rendered 3d model views","authors":"H Su, C R Qi, Y Li, L J Guibas","year":"2015","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tulsiani and Malik, 2015] S. Tulsiani and J. Malik. Viewpoints and keypoints. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1510-1519, June 2015.","title":"Viewpoints and keypoints","authors":"S Tulsiani, J Malik","year":"2015","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu et al., 2016] Jiajun Wu, Tianfan Xue, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, and William T. Freeman. Single image 3d interpreter network. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision -ECCV 2016, pages 365-382, Cham, 2016. Springer International Publishing.","title":"Single image 3d interpreter network","authors":"Jiajun Wu, Tianfan Xue, Joseph J Lim, Yuandong Tian, Joshua B Tenenbaum, Antonio Torralba, William T Freeman","year":"2016","venue":"Computer Vision -ECCV 2016","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xiang et al., 2014] Yu Xiang, Roozbeh Mottaghi, and Silvio Savarese. Beyond pascal: A benchmark for 3d object detection in the wild. In IEEE Winter Conference on Applications of Computer Vision (WACV), 2014.","title":"Beyond pascal: A benchmark for 3d object detection in the wild","authors":"Yu Xiang, Roozbeh Mottaghi, Silvio Savarese","year":"2014","venue":"IEEE Winter Conference on Applications of Computer Vision (WACV)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xiang et al., 2017] Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. arXiv preprint arXiv:1711.00199, 2017.","title":"Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes","authors":"Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, Dieter Fox","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}