{"type":"ScholarlyArticle","title":"Dite-HRNet: Dynamic Lightweight High-Resolution Network for Human Pose Estimation","authors":[{"name":"Qun Li","slug":"qun-li","given":"Qun","family":"Li","affiliation":"School of Computer Science , Nanjing University of Posts and Telecommunications , Nanjing , China"},{"name":"Ziyi Zhang","slug":"ziyi-zhang","given":"Ziyi","family":"Zhang","affiliation":"School of Computer Science , Nanjing University of Posts and Telecommunications , Nanjing , China"},{"name":"Fu Xiao","slug":"fu-xiao","given":"Fu","family":"Xiao","affiliation":"School of Computer Science , Nanjing University of Posts and Telecommunications , Nanjing , China"},{"name":"Feng Zhang","slug":"feng-zhang","given":"Feng","family":"Zhang","affiliation":"School of Computer Science , Nanjing University of Posts and Telecommunications , Nanjing , China"},{"name":"Bir Bhanu","slug":"bir-bhanu","given":"Bir","family":"Bhanu","affiliation":"Department of Electrical and Computer Engineering , University of California at Riverside , CA , USA"}],"abstract":"A high-resolution network exhibits remarkable capability in extracting multi-scale features for human pose estimation, but fails to capture long-range interactions between joints and has high computational complexity. To address these problems, we present a Dynamic lightweight High-Resolution Network (Dite-HRNet), which can efficiently extract multi-scale contextual information and model long-range spatial dependency for human pose estimation. Specifically, we propose two methods, dynamic split convolution and adaptive context modeling, and embed them into two novel lightweight blocks, which are named dynamic multi-scale context block and dynamic global context block. These two blocks, as the basic component units of our Dite-HRNet, are specially designed for the high-resolution networks to make full use of the parallel multi-resolution architecture. Experimental results show that the proposed network achieves superior performance on both COCO and MPII human pose estimation datasets, surpassing the state-of-the-art lightweight networks. Code is available at: https://github.com/ZiyiZhang27/Dite-HRNet.","doi":"10.24963/ijcai.2022/153","url":"https://ijcai.org/proceedings/2022/153","pdf":"https://ijcai.org/proceedings/2022/0153.pdf","markdown":"https://ijcai.org/proceedings/2022/153.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/153","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":153,"pages":{"first":1095,"last":1101},"keywords":["Computer Vision: Biometrics, Face, Gesture and Pose Recognition","Machine Learning: Convolutional Networks","Computer Vision: Action and Behaviour Recognition"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986681","length_sec":76,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984824","length_sec":320,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Andriluka et al., 2014] Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2D human pose estimation: New benchmark and state of the art analysis. In CVPR, pages 3686-3693, 2014.","title":"2D human pose estimation: New benchmark and state of the art analysis","authors":"Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, Bernt Schiele","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2019] Yue Cao, Jiarui Xu, Stephen Lin, Fangyun Wei, and Han Hu. GCNet: Non-local networks meet squeeze-excitation networks and beyond. In ICCV, pages 1971-1980, 2019.","title":"GCNet: Non-local networks meet squeeze-excitation networks and beyond","authors":"Yue Cao, Jiarui Xu, Stephen Lin, Fangyun Wei, Han Hu","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. Cas- caded pyramid network for multi-person pose estimation. In CVPR, pages 7103-7112, 2018.","title":"Cascaded pyramid network for multi-person pose estimation","authors":"Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, Jian Sun","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Yinpeng Chen, Xiyang Dai, Mengchen Liu, Dongdong Chen, Lu Yuan, and Zicheng Liu. Dy- namic convolution: Attention over convolution kernels. In CVPR, pages 11027-11036, 2020.","title":"Dynamic convolution: Attention over convolution kernels","authors":"Chen","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cheng et al., 2020] Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas S. Huang, and Lei Zhang. HigherHRNet: Scale-aware representation learning for bottom-up human pose estimation. In CVPR, pages 5385- 5394, 2020.","title":"HigherHRNet: Scale-aware representation learning for bottom-up human pose estimation","authors":"Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas S Huang, Lei Zhang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cui et al., 2021] Cheng Cui, Tingquan Gao, Shengyu Wei, Yuning Du, Ruoyu Guo, Shuilong Dong, Bin Lu, Ying Zhou, Xueying Lv, Qiwen Liu, Xiaoguang Hu, Dianhai Yu, and Yanjun Ma. PP-LCNet: A lightweight CPU con- volutional neural network. arXiv:2109.15099, 2021.","title":"PP-LCNet: A lightweight CPU convolutional neural network","authors":"Cheng Cui, Tingquan Gao, Shengyu Wei, Yuning Du, Ruoyu Guo, Shuilong Dong, Bin Lu, Ying Zhou, Xueying Lv, Qiwen Liu, Xiaoguang Hu, Dianhai Yu, Yanjun Ma","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Howard et al., 2017] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mo- bileNets: Efficient convolutional neural networks for mo- bile vision applications. arXiv:1704.04861, 2017.","title":"Mo-bileNets: Efficient convolutional neural networks for mobile vision applications","authors":"Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Howard et al., 2019] Andrew Howard, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, Yukun Zhu, Ruoming Pang, Hartwig Adam, and Quoc Le. Searching for MobileNetV3. In ICCV, pages 1314-1324, 2019.","title":"Searching for MobileNetV3","authors":"Andrew Howard, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, Yukun Zhu, Ruoming Pang, Hartwig Adam, Quoc Le","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hu et al., 2020] Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu. Squeeze-and-Excitation networks. IEEE Trans. Pattern Anal. Mach. Intell., 42(8):2011-2023, 2020.","title":"Squeeze-and-Excitation networks","authors":"Jie Hu, Li Shen, Samuel Albanie, Gang Sun, Enhua Wu","year":"2020","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2020] Junjie Huang, Zheng Zhu, Feng Guo, and Guan Huang. The devil is in the details: Delving into unbiased data processing for human pose estimation. In CVPR, pages 5699-5708, 2020.","title":"The devil is in the details: Delving into unbiased data processing for human pose estimation","authors":"Junjie Huang, Zheng Zhu, Feng Guo, Guan Huang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, pages 740-755, 2014.","title":"Microsoft COCO: Common objects in context","authors":"Lin","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ma et al., 2018] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. ShuffleNet V2: Practical guidelines for efficient CNN architecture design. In ECCV, pages 122-138, 2018.","title":"ShuffleNet V2: Practical guidelines for efficient CNN architecture design","authors":"Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, Jian Sun","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Newell et al., 2016] Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hourglass networks for human pose esti- mation. In ECCV, pages 483-499, 2016.","title":"Stacked hourglass networks for human pose estimation","authors":"Alejandro Newell, Kaiyu Yang, Jia Deng","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sandler et al., 2018] Mark Sandler, Andrew Howard, Men- glong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. MobileNetV2: Inverted residuals and linear bottlenecks. In CVPR, pages 4510-4520, 2018.","title":"MobileNetV2: Inverted residuals and linear bottlenecks","authors":"Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, Liang-Chieh Chen","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sun et al., 2019] Ke Sun, Bin Xiao, Dong Liu, and Jing- dong Wang. Deep high-resolution representation learning for human pose estimation. In CVPR, pages 5686-5696, 2019.","title":"Deep high-resolution representation learning for human pose estimation","authors":"Ke Sun, Bin Xiao, Dong Liu, Jingdong Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tan and Le, 2019] Mingxing Tan and Quoc V. Le. Mix- Conv: Mixed depthwise convolutional kernels. In BMVC, pages 1-13, 2019.","title":"Mix-Conv: Mixed depthwise convolutional kernels","authors":"Mingxing Tan, V Quoc, Le","year":"2019","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2018] Xiaolong Wang, Ross B. Girshick, Abhi- nav Gupta, and Kaiming He. Non-local neural networks. In CVPR, pages 7794-7803, 2018.","title":"Non-local neural networks","authors":"Xiaolong Wang, Ross B Girshick, Abhinav Gupta, Kaiming He","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2021] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, and Bin Xiao. Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell., 43(10):3349-3364, 2021.","title":"Deep high-resolution representation learning for visual recognition","authors":"Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, Bin Xiao","year":"2021","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xiao et al., 2018] Bin Xiao, Haiping Wu, and Yichen Wei. Simple baselines for human pose estimation and tracking. In ECCV, pages 472-487, 2018.","title":"Simple baselines for human pose estimation and tracking","authors":"Bin Xiao, Haiping Wu, Yichen Wei","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yang et al., 2019] Brandon Yang, Gabriel Bender, Quoc V. Le, and Jiquan Ngiam. CondConv: Conditionally param- eterized convolutions for efficient inference. In NeurIPS, pages 1305-1316, 2019.","title":"CondConv: Conditionally parameterized convolutions for efficient inference","authors":"Brandon Yang, Gabriel Bender, Quoc V Le, Jiquan Ngiam","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yu et al., 2021] Changqian Yu, Bin Xiao, Changxin Gao, Lu Yuan, Lei Zhang, Nong Sang, and Jingdong Wang. Lite-HRNet: A lightweight high-resolution network. In CVPR, pages 10435-10445, 2021.","title":"Lite-HRNet: A lightweight high-resolution network","authors":"Changqian Yu, Bin Xiao, Changxin Gao, Lu Yuan, Lei Zhang, Nong Sang, Jingdong Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2018] Xiangyu Zhang, Xinyu Zhou, Mengx- iao Lin, and Ran Sun. ShuffleNet: An extremely effi- cient convolutional neural network for mobile devices. In CVPR, pages 6848-6856, 2018.","title":"ShuffleNet: An extremely efficient convolutional neural network for mobile devices","authors":"Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, Ran Sun","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2020] Feng Zhang, Xiatian Zhu, Hanbin Dai, Mao Ye, and Ce Zhu. Distribution-aware coordinate rep- resentation for human pose estimation. In CVPR, pages 7091-7100, 2020.","title":"Distribution-aware coordinate representation for human pose estimation","authors":"Feng Zhang, Xiatian Zhu, Hanbin Dai, Mao Ye, Ce Zhu","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2020] Daquan Zhou, Qibin Hou, Yunpeng Chen, Jiashi Feng, and Shuicheng Yan. Rethinking bot- tleneck structure for efficient mobile network design. In ECCV, pages 680-697, 2020.","title":"Rethinking bottleneck structure for efficient mobile network design","authors":"Daquan Zhou, Qibin Hou, Yunpeng Chen, Jiashi Feng, Shuicheng Yan","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}