{"type":"ScholarlyArticle","title":"Pointsoup: High-Performance and Extremely Low-Decoding-Latency Learned Geometry Codec for Large-Scale Point Cloud Scenes","authors":[{"name":"Kang You","slug":"kang-you","given":"Kang","family":"You","affiliation":"Nanjing University of Aeronautics and Astronautics"},{"name":"Kai Liu","slug":"kai-liu","given":"Kai","family":"Liu","affiliation":"Nanjing University of Aeronautics and Astronautics"},{"name":"Li Yu","slug":"li-yu","given":"Li","family":"Yu","affiliation":""},{"name":"Pan Gao","slug":"pan-gao","given":"Pan","family":"Gao","affiliation":""},{"name":"Dandan Ding","slug":"dandan-ding","given":"Dandan","family":"Ding","affiliation":"Hangzhou Normal University"}],"abstract":"Despite considerable progress being achieved in point cloud geometry compression, there still remains a challenge in effectively compressing large-scale scenes with sparse surfaces. Another key challenge lies in reducing decoding latency, a crucial requirement in real-world application. In this paper, we propose Pointsoup, an efficient learning-based geometry codec that attains high-performance and extremely low-decoding-latency simultaneously. Inspired by  conventional Trisoup codec, a point model-based strategy is devised to characterize local surfaces. Specifically, skin features are embedded from  local windows via an attention-based encoder, and dilated windows are introduced as cross-scale priors to infer the distribution of quantized features in parallel. During decoding, features undergo fast refinement, followed by a folding-based point generator that reconstructs point coordinates with fairly fast speed. Experiments show that Pointsoup achieves state-of-the-art performance on multiple benchmarks with significantly lower decoding complexity, i.e., up to 90~160× faster than the G-PCCv23 Trisoup decoder on a comparatively low-end platform (e.g., one RTX 2080Ti). Furthermore, it offers variable-rate control with a single neural model (2.9MB), which is attractive for industrial practitioners.","doi":"10.24963/ijcai.2024/595","url":"https://ijcai.org/proceedings/2024/595","pdf":"https://ijcai.org/proceedings/2024/0595.pdf","markdown":"https://ijcai.org/proceedings/2024/595.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/595","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":595,"pages":{"first":5380,"last":5388},"keywords":["Computer Vision: CV: 3D computer vision","Machine Learning: ML: Geometric learning","Multidisciplinary Topics and Applications: MTA: Real-time systems","Robotics: ROB: Robotics and vision"],"videos":[],"references":[{"position":1,"raw":"[Abbasi et al., 2023] Rashid Abbasi, Ali Kashif Bashir, Hasan J. Alyamani, Farhan Amin, Jaehyeok Doh, and Jianwen Chen. Lidar point cloud compression, process- ing and learning for autonomous driving. IEEE Trans- actions on Intelligent Transportation Systems, 24(1):962- 979, 2023.","title":"Lidar point cloud compression, processing and learning for autonomous driving","authors":"Rashid Abbasi, Ali Kashif Bashir, J Hasan, Farhan Alyamani, Jaehyeok Amin, Jianwen Doh, Chen","year":"2023","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Armeni et al., 2016] Iro Armeni, Ozan Sener, Amir R. Za- mir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Sil- vio Savarese. 3d semantic parsing of large-scale indoor spaces. In Proceedings of the IEEE International Confer- ence on Computer Vision and Pattern Recognition, 2016.","title":"3d semantic parsing of large-scale indoor spaces","authors":"Iro Armeni, Ozan Sener, R Amir, Helen Zamir, Ioannis Jiang, Martin Brilakis, Silvio Fischer, Savarese","year":"2016","venue":"Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ballé et al., 2016] Johannes Ballé, Valero Laparra, and Eero P Simoncelli. End-to-end optimized image compres- sion. arXiv preprint arXiv:1611.01704, 2016.","title":"End-to-end optimized image compression","authors":"Johannes Ballé, Valero Laparra, Eero P Simoncelli","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Behley et al., 2019] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall. Se- manticKITTI: A Dataset for Semantic Scene Understand- ing of LiDAR Sequences. In Proc. of the IEEE/CVF Inter- national Conf. on Computer Vision (ICCV), 2019.","title":"Se-manticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences","authors":"J Behley, M Garbade, A Milioto, J Quenzel, S Behnke, C Stachniss, J Gall","year":"2019","venue":"Proc. of the IEEE/CVF International Conf. on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chang et al., 2015] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015.","title":"Shapenet: An information-rich 3d model repository","authors":"Thomas Angel X Chang, Leonidas Funkhouser, Pat Guibas, Qixing Hanrahan, Zimo Huang, Silvio Li, Manolis Savarese, Shuran Savva, Hao Song, Su","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2023] Anthony Chen, Shiwen Mao, Zhu Li, Minrui Xu, Hongliang Zhang, Dusit Niyato, and Zhu Han. An introduction to point cloud compression stan- dards. GetMobile: Mobile Computing and Communica- tions, 27(1):11-17, 2023.","title":"An introduction to point cloud compression standards","authors":"Anthony Chen, Shiwen Mao, Zhu Li, Minrui Xu, Hongliang Zhang, Dusit Niyato, Zhu Han","year":"2023","venue":"GetMobile: Mobile Computing and Communications","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dai et al., 2017] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017.","title":"Scannet: Richly-annotated 3d reconstructions of indoor scenes","authors":"Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner","year":"2017","venue":"Proc. Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fan et al., 2023] Tingyu Fan, Linyao Gao, Yiling Xu, Dong Wang, and Zhu Li. Multiscale latent-guided entropy model for lidar point cloud compression. IEEE Transactions on Circuits and Systems for Video Technology, 33(12):7857- 7869, 2023.","title":"Multiscale latent-guided entropy model for lidar point cloud compression","authors":"Fan","year":"2023","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fu et al., 2022] Chunyang Fu, Ge Li, Rui Song, Wei Gao, and Shan Liu. Octattention: Octree-based large-scale con- texts model for point cloud compression. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 36, pages 625-633, 2022.","title":"Octattention: Octree-based large-scale contexts model for point cloud compression","authors":"Chunyang Fu, Ge Li, Rui Song, Wei Gao, Shan Liu","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2021] Meng-Hao Guo, Jun-Xiong Cai, Zheng- Ning Liu, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pct: Point cloud transformer. Computational Visual Me- dia, 7:187-199, 2021.","title":"Pct: Point cloud transformer","authors":"Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Shi-Min Ralph R Martin, Hu","year":"2021","venue":"Computational Visual Media","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2022] Yun He, Xinlin Ren, Danhang Tang, Yinda Zhang, Xiangyang Xue, and Yanwei Fu. Density- preserving deep point cloud compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 2333-2342, 2022.","title":"Densitypreserving deep point cloud compression","authors":"He","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang and Wang, 2023] Runnan Huang and Miaohui Wang. Patch-wise lidar point cloud geometry compres- sion based on autoencoder. In International Conference on Image and Graphics, pages 299-310. Springer, 2023.","title":"Patch-wise lidar point cloud geometry compression based on autoencoder","authors":"Wang ; Huang, Runnan Huang, Miaohui Wang","year":"2023","venue":"International Conference on Image and Graphics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2022] Tianxin Huang, Jiangning Zhang, Jun Chen, Zhonggan Ding, Ying Tai, Zhenyu Zhang, Chengjie Wang, and Yong Liu. 3qnet: 3d point cloud geometry quantization compression network. ACM Transactions on Graphics (TOG), 41(6):1-13, 2022.","title":"3qnet: 3d point cloud geometry quantization compression network","authors":"Tianxin Huang, Jiangning Zhang, Jun Chen, Zhonggan Ding, Ying Tai, Zhenyu Zhang, Chengjie Wang, Yong Liu","year":"2022","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jamil et al., 2023] Sonain Jamil, Md Jalil Piran, MuhibUr Rahman, and Oh-Jin Kwon. Learning-driven lossy image compression: A comprehensive survey. Engineering Ap- plications of Artificial Intelligence, 123:106361, 2023.","title":"Learning-driven lossy image compression: A comprehensive survey","authors":"Jamil","year":"2023","venue":"Engineering Applications of Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2023a] Shanshan Li, Pan Gao, Xiaoyang Tan, and Mingqiang Wei. Proxyformer: Proxy alignment assisted point cloud completion with missing part sensitive trans- former. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9466- 9475, 2023.","title":"Proxyformer: Proxy alignment assisted point cloud completion with missing part sensitive transformer","authors":"Li","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2023b] Zihao Li, Pan Gao, Hui Yuan, and Ran Wei. Dynamic local feature aggregation for learning on point clouds. arXiv preprint arXiv:2301.02836, 2023.","title":"Dynamic local feature aggregation for learning on point clouds","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Quach et al., 2022] Maurice Quach, Jiahao Pang, Dong Tian, Giuseppe Valenzise, and Frédéric Dufaux. Survey on deep learning-based point cloud compression. Frontiers in Signal Processing, 2:846972, 2022.","title":"Survey on deep learning-based point cloud compression","authors":"Maurice Quach, Jiahao Pang, Dong Tian, Giuseppe Valenzise, Frédéric Dufaux","year":"2022","venue":"Frontiers in Signal Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schnabel and Klein, 2006] Ruwen Schnabel and Reinhard Klein. Octree-based point-cloud compression. PBG@ SIGGRAPH, 3, 2006.","title":"Octree-based point-cloud compression","authors":"Ruwen Schnabel, Reinhard Klein","year":"2006","venue":"PBG@ SIGGRAPH","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Song et al., 2023] Rui Song, Chunyang Fu, Shan Liu, and Ge Li. Efficient hierarchical entropy model for learned point cloud compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14368-14377, 2023.","title":"Efficient hierarchical entropy model for learned point cloud compression","authors":"Rui Song, Chunyang Fu, Shan Liu, Ge Li","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Vinodkumar et al., 2023] Prasoon Kumar Vinodkumar, Do- gus Karabulut, Egils Avots, Cagri Ozcinar, and Gholam- reza Anbarjafari. A survey on deep learning based seg- mentation, detection and classification for 3d point clouds. Entropy, 25(4):635, 2023.","title":"A survey on deep learning based segmentation, detection and classification for 3d point clouds","authors":"Prasoon Kumar Vinodkumar, Dogus Karabulut, Egils Avots, Cagri Ozcinar, Gholamreza Anbarjafari","year":"2023","venue":"Entropy","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2019] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog), 38(5):1-12, 2019.","title":"Dynamic graph cnn for learning on point clouds","authors":"Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, Justin M Solomon","year":"2019","venue":"ACM Transactions on Graphics (tog)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2021a] Jianqiang Wang, Dandan Ding, Zhu Li, and Zhan Ma. Multiscale point cloud geometry compres- sion. In 2021 Data Compression Conference (DCC), pages 73-82. IEEE, 2021.","title":"Multiscale point cloud geometry compression","authors":"Wang","year":"2021","venue":"2021 Data Compression Conference (DCC)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2021b] Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 44(9):4555-4576, 2021.","title":"A survey on curriculum learning","authors":"Wang","year":"2021","venue":"IEEE Trans-actions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2022] Jianqiang Wang, Dandan Ding, Zhu Li, Xiaoxing Feng, Chuntong Cao, and Zhan Ma. Sparse tensor-based multiscale representation for point cloud ge- ometry compression. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 2022.","title":"Sparse tensor-based multiscale representation for point cloud geometry compression","authors":"Jianqiang Wang, Dandan Ding, Zhu Li, Xiaoxing Feng, Chuntong Cao, Zhan Ma","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[WG 7 and Coding, 2023] MPEG 3D Graphics WG 7 and Haptics Coding. G-pcc 2nd edition codec description. ISO/IEC JTC 1/SC 29/WG 7, 2023.","title":"MPEG 3D Graphics WG 7 and Haptics Coding","authors":null,"year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[WG 7, 2020] MPEG 3D Graphics Coding WG 7. V-pcc codec description. ISO/IEC JTC 1/SC 29/WG 7, 2020.","title":"MPEG 3D Graphics Coding WG 7. V-pcc codec description","authors":null,"year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wiesmann et al., 2021] Louis Wiesmann, Andres Milioto, Xieyuanli Chen, Cyrill Stachniss, and Jens Behley. Deep compression for dense point cloud maps. IEEE Robotics and Automation Letters, 6(2):2060-2067, 2021.","title":"Deep compression for dense point cloud maps","authors":"Wiesmann","year":"2021","venue":"IEEE Robotics and Automation Letters","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wu et al., 2022] Xiaoyang Wu, Yixing Lao, Li Jiang, Xi- hui Liu, and Hengshuang Zhao. Point transformer v2: Grouped vector attention and partition-based pool- ing. Advances in Neural Information Processing Systems, 35:33330-33342, 2022.","title":"Point transformer v2: Grouped vector attention and partition-based pooling","authors":"Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, Hengshuang Zhao","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[You and Gao, 2021] Kang You and Pan Gao. Patch-based deep autoencoder for point cloud geometry compression. In ACM Multimedia Asia, pages 1-7. 2021.","title":"Patch-based deep autoencoder for point cloud geometry compression","authors":"Kang You, Pan Gao","year":"2021","venue":"ACM Multimedia Asia","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[You et al., 2022] Kang You, Pan Gao, and Qing Li. Ipdae: Improved patch-based deep autoencoder for lossy point cloud geometry compression. In Proceedings of the 1st International Workshop on Advances in Point Cloud Com- pression, Processing and Analysis, pages 1-10, 2022.","title":"Ipdae: Improved patch-based deep autoencoder for lossy point cloud geometry compression","authors":"Kang You, Pan Gao, Qing Li","year":"2022","venue":"Proceedings of the 1st International Workshop on Advances in Point Cloud Compression, Processing and Analysis","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2022] Junteng Zhang, Gexin Liu, Dandan Ding, and Zhan Ma. Transformer and upsampling-based point cloud compression. In Proceedings of the 1st Inter- national Workshop on Advances in Point Cloud Compres- sion, Processing and Analysis, pages 33-39, 2022.","title":"Transformer and upsampling-based point cloud compression","authors":"Zhang","year":"2022","venue":"Proceedings of the 1st International Workshop on Advances in Point Cloud Compression, Processing and Analysis","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2023a] Junteng Zhang, Tong Chen, Dandan Ding, and Zhan Ma. Yoga: Yet another geometry-based point cloud compressor. In Proceedings of the 31st ACM International Conference on Multimedia, pages 9070- 9081, 2023.","title":"Yoga: Yet another geometry-based point cloud compressor","authors":"Zhang","year":"2023","venue":"Proceedings of the 31st ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2023b] Renrui Zhang, Liuhui Wang, Yali Wang, Peng Gao, Hongsheng Li, and Jianbo Shi. Pa- rameter is not all you need: Starting from non-parametric networks for 3d point cloud analysis. arXiv preprint arXiv:2303.08134, 2023.","title":"Parameter is not all you need: Starting from non-parametric networks for 3d point cloud analysis","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhao et al., 2021] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 16259-16268, 2021.","title":"Point transformer","authors":"Hengshuang Zhao, Li Jiang, Jiaya Jia, Vladlen Philip Hs Torr, Koltun","year":"2021","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}