{"type":"ScholarlyArticle","title":"DRPose3D: Depth Ranking in 3D Human Pose Estimation","authors":[{"name":"Min Wang","slug":"min-wang","given":"Min","family":"Wang","affiliation":"Department of Computer Science and Engineering , Shanghai Jiao Tong University"},{"name":"Xipeng Chen","slug":"xipeng-chen","given":"Xipeng","family":"Chen","affiliation":"School of Data and Computer Science , Sun Yat-Sen University"},{"name":"Wentao Liu","slug":"wentao-liu","given":"Wentao","family":"Liu","affiliation":"Department of Computer Science and Technology , Tsinghua University"},{"name":"Chen Qian","slug":"chen-qian","given":"Chen","family":"Qian","affiliation":"SenseTime Group Limited"},{"name":"Liang Lin","slug":"liang-lin","given":"Liang","family":"Lin","affiliation":"School of Data and Computer Science , Sun Yat-Sen University; SenseTime Group Limited"},{"name":"Lizhuang Ma","slug":"lizhuang-ma","given":"Lizhuang","family":"Ma","affiliation":"School of Computer Science and Software Engineering , East China Normal University; Department of Computer Science and Engineering , Shanghai Jiao Tong University"}],"abstract":"In this paper, we propose a two-stage depth ranking based method (DRPose3D) to tackle the problem of 3D human pose estimation. Instead of accurate 3D positions, the depth ranking can be identified by human intuitively and learned using the deep neural network more easily by solving classification problems. Moreover, depth ranking contains rich 3D information. It prevents the 2D-to-3D pose regression in two-stage methods from being ill-posed. In our method, firstly, we design a Pairwise Ranking Convolutional Neural Network (PRCNN) to extract depth rankings of human joints from images.  Secondly, a coarse-to-fine 3D Pose Network(DPNet) is proposed to estimate 3D poses from both depth rankings and 2D human joint locations. Additionally, to improve the generality of our model, we introduce a statistical method to augment depth rankings. Our approach outperforms the state-of-the-art methods in the Human3.6M benchmark for all three testing protocols, indicating that depth ranking is an essential geometric feature which can be learned to improve the 3D pose estimation.","doi":"10.24963/ijcai.2018/136","url":"https://ijcai.org/proceedings/2018/136","pdf":"https://ijcai.org/proceedings/2018/0136.pdf","markdown":"https://ijcai.org/proceedings/2018/136.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/136","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":136,"pages":{"first":978,"last":984},"keywords":["Machine Learning: Deep Learning","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision","Computer Vision: 2D and 3D Computer Vision","Computer Vision: Biometrics, Face and Gesture Recognition"],"videos":[],"references":[{"position":1,"raw":"[Akhter and Black, 2015] Ijaz Akhter and Michael J Black. Pose-conditioned joint angle limits for 3d human pose re- construction. In CVPR, pages 1446-1455, 2015.","title":"Pose-conditioned joint angle limits for 3d human pose reconstruction","authors":"Ijaz Akhter, Michael J Black","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andriluka et al., 2014] Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2d human pose estimation: New benchmark and state of the art analysis. In CVPR, pages 3686-3693, 2014.","title":"2d human pose estimation: New benchmark and state of the art analysis","authors":"Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, Bernt Schiele","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bogo et al., 2016] Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J Black. Keep it smpl: Automatic estimation of 3d human pose and shape from a single image. In ECCV, pages 561-578. Springer, 2016.","title":"Keep it smpl: Automatic estimation of 3d human pose and shape from a single image","authors":"Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, Michael J Black","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Burges et al., 2005] Chris Burges, Tal Shaked, Erin Ren- shaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning to rank using gradient descent. In ICML, pages 89-96. ACM, 2005.","title":"Learning to rank using gradient descent","authors":"Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, Greg Hullender","year":"2005","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2007] Zhe Cao, Tao Qin, Tie-Yan Liu, Ming- Feng Tsai, and Hang Li. Learning to rank: from pairwise approach to listwise approach. In ICML, pages 129-136. ACM, 2007.","title":"Learning to rank: from pairwise approach to listwise approach","authors":"Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, Hang Li","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2016] Weifeng Chen, Zhao Fu, Dawei Yang, and Jia Deng. Single-image depth perception in the wild. In NIPS, pages 730-738, 2016.","title":"Single-image depth perception in the wild","authors":"Weifeng Chen, Zhao Fu, Dawei Yang, Jia Deng","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2017] Shixing Chen, Caojin Zhang, Ming Dong, Jialiang Le, and Mike Rao. Using ranking-cnn for age estimation. In CVPR, 2017.","title":"Using ranking-cnn for age estimation","authors":"Shixing Chen, Caojin Zhang, Ming Dong, Jialiang Le, Mike Rao","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cohen et al., 1998] William W Cohen, Robert E Schapire, and Yoram Singer. Learning to order things. In NIPS, pages 451-457, 1998.","title":"Learning to order things","authors":"Robert E William W Cohen, Yoram Schapire, Singer","year":"1998","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 2018] Haoshu Fang, Yuanlu Xu, Wenguan Wang, Xiaobai Liu, and Song-Chun Zhu. Learning knowledge-guided pose grammar machine for 3d human pose estimation. AAAI, 2018.","title":"Learning knowledge-guided pose grammar machine for 3d human pose estimation","authors":"Fang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ionescu et al., 2014] Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and predictive methods for 3d human sens- ing in natural environments. TPAMI, 36(7):1325-1339, 2014.","title":"Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments","authors":"Catalin Ionescu, Dragos Papava, Vlad Olaru, Cristian Sminchisescu","year":"2014","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jahangiri and Yuille, 2017] Ehsan Jahangiri and Alan L Yuille. Generating multiple hypotheses for human 3d pose consistent with 2d joint detections. arXiv preprint arXiv:1702.02258, 2017.","title":"Generating multiple hypotheses for human 3d pose consistent with 2d joint detections","authors":"Ehsan Jahangiri, Alan L Yuille","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2015] Jianshu Li, Chao Xiong, Luoqi Liu, Xi- angbo Shu, and Shuicheng Yan. Deep face beautification. In ACM MM, pages 793-794. ACM, 2015.","title":"Deep face beautification","authors":"Li","year":"2015","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2017] Mude Lin, Liang Lin, Xiaodan Liang, Keze Wang, and Hui Chen. Recurrent 3d pose sequence machines. In CVPR, 2017.","title":"Recurrent 3d pose sequence machines","authors":"Mude Lin, Liang Lin, Xiaodan Liang, Keze Wang, Hui Chen","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Martinez et al., 2017] Julieta Martinez, Rayat Hossain, Javier Romero, and James J. Little. A simple yet effec- tive baseline for 3d human pose estimation. In ICCV, Oct 2017.","title":"Little. A simple yet effective baseline for 3d human pose estimation","authors":"Julieta Martinez, Rayat Hossain, Javier Romero, James J","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mehta et al., 2016] Dushyant Mehta, Helge Rhodin, Dan Casas, Oleksandr Sotnychenko, Weipeng Xu, and Chris- tian Theobalt. Monocular 3d human pose estimation us- ing transfer learning and improved cnn supervision. arXiv preprint arXiv:1611.09813, 2016.","title":"Monocular 3d human pose estimation using transfer learning and improved cnn supervision","authors":"Dushyant Mehta, Helge Rhodin, Dan Casas, Oleksandr Sotnychenko, Weipeng Xu, Christian Theobalt","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Moreno-Noguer, 2017] Francesc Moreno-Noguer. 3d hu- man pose estimation from a single image via distance ma- trix regression. In CVPR, pages 1561-1570. IEEE, 2017.","title":"Francesc Moreno-Noguer. 3d human pose estimation from a single image via distance matrix regression","authors":"Moreno-Noguer","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mori and Malik, 2006] Greg Mori and Jitendra Malik. Re- covering 3d human body configurations using shape con- texts. TPAMI, 28(7):1052-1062, 2006.","title":"Recovering 3d human body configurations using shape contexts","authors":"Malik ; Mori, Greg Mori, Jitendra Malik","year":"2006","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Newell et al., 2016] Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hourglass networks for human pose esti- mation. In ECCV, pages 483-499. Springer, 2016.","title":"Stacked hourglass networks for human pose estimation","authors":"Alejandro Newell, Kaiyu Yang, Jia Deng","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pavlakos et al., 2017] Georgios Pavlakos, Xiaowei Zhou, Konstantinos G Derpanis, and Kostas Daniilidis. Coarse- to-fine volumetric prediction for single-image 3d human pose. In CVPR, pages 1263-1272. IEEE, 2017.","title":"Coarseto-fine volumetric prediction for single-image 3d human pose","authors":"Georgios Pavlakos, Xiaowei Zhou, Konstantinos G Derpanis, Kostas Daniilidis","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ramakrishna et al., 2012] Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. Reconstructing 3d human pose from 2d image landmarks. ECCV, pages 573-586, 2012.","title":"Reconstructing 3d human pose from 2d image landmarks","authors":"Varun Ramakrishna, Takeo Kanade, Yaser Sheikh","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rogez and Schmid, 2016] Grégory Rogez and Cordelia Schmid. Mocap-guided data augmentation for 3d pose es- timation in the wild. In NIPS, pages 3108-3116, 2016.","title":"Grégory Rogez and Cordelia Schmid. Mocap-guided data augmentation for 3d pose estimation in the wild","authors":"Schmid Rogez","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sun et al., 2017] Xiao Sun, Jiaxiang Shang, Shuang Liang, and Yichen Wei. Compositional human pose regression. In ICCV, Oct 2017.","title":"Compositional human pose regression","authors":"Xiao Sun, Jiaxiang Shang, Shuang Liang, Yichen Wei","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2016] Xingyi Zhou, Xiao Sun, Wei Zhang, Shuang Liang, and Yichen Wei. Deep kinematic pose re- gression. In ECCV Workshops, pages 186-201. Springer, 2016.","title":"Deep kinematic pose regression","authors":"Zhou","year":"2016","venue":"ECCV Workshops","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2017a] Xiaowei Zhou, Menglong Zhu, Geor- gios Pavlakos, Spyridon Leonardos, Kostantinos G Derpa- nis, and Kostas Daniilidis. Monocap: Monocular human motion capture using a cnn coupled with a geometric prior. arXiv preprint arXiv:1701.02354, 2017.","title":"Monocap: Monocular human motion capture using a cnn coupled with a geometric prior","authors":"Zhou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2017b] Xingyi Zhou, Qixing Huang, Xiao Sun, Xiangyang Xue, and Yichen Wei. Towards 3d human pose estimation in the wild: a weakly-supervised approach. In ICCV, 2017.","title":"Towards 3d human pose estimation in the wild: a weakly-supervised approach","authors":"Zhou","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}