{"type":"ScholarlyArticle","title":"Encoding Auxiliary Information to Restore Compressed Point Cloud Geometry","authors":[{"name":"Gexin Liu","slug":"gexin-liu","given":"Gexin","family":"Liu","affiliation":"School of Information Science and Technology , Hangzhou Normal University , China"},{"name":"Jiahao Zhu","slug":"jiahao-zhu","given":"Jiahao","family":"Zhu","affiliation":"School of Information Science and Technology , Hangzhou Normal University , China"},{"name":"Dandan Ding","slug":"dandan-ding","given":"Dandan","family":"Ding","affiliation":"School of Information Science and Technology , Hangzhou Normal University , China"},{"name":"Zhan Ma","slug":"zhan-ma","given":"Zhan","family":"Ma","affiliation":"School of Electronic Science and Engineering , Nanjing University , China"}],"abstract":"The standardized Geometry-based Point Cloud Compression (G-PCC) suffers from limited coding performance and low-quality reconstruction. To address this, we propose AuxGR, a performance-complexity tradeoff solution for point cloud geometry restoration: leveraging auxiliary bitstream to enhance the quality of G-PCC compressed point cloud geometry. This auxiliary bitstream efficiently encapsulates spatio-temporal information. For static coding, we perform paired information embedding (PIE) on the G-PCC decoded frame by employing target convolutions from its original counterpart, producing an auxiliary bitstream containing abundant original information. For dynamic coding, in addition to PIE, we propose temporal information embedding (TIE) to capture motion information between the previously restored and the current G-PCC decoded frames. TIE applies target kNN attention between them, which ensures the temporal neighborhood construction for each point and implicitly represents motions. Due to the similarity across temporal frames, only the residuals between TIE and PIE outputs are compressed as auxiliary bitstream. Experimental results demonstrate that AuxGR notably outperforms existing methods in both static and dynamic coding scenarios. Moreover, our framework enables the flexible incorporation of auxiliary information under computation constraints, which is attractive to real applications.","doi":"10.24963/ijcai.2024/242","url":"https://ijcai.org/proceedings/2024/242","pdf":"https://ijcai.org/proceedings/2024/0242.pdf","markdown":"https://ijcai.org/proceedings/2024/242.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/242","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":242,"pages":{"first":2189,"last":2197},"keywords":["Data Mining: DM: Mining spatial and/or temporal data"],"videos":[],"references":[{"position":1,"raw":"[Akhtar et al., 2024] Anique Akhtar, Zhu Li, and Geert Van der Auwera. Inter-frame compression for dynamic point cloud geometry coding. IEEE Transactions on Im- age Processing, 33:584-594, 2024.","title":"Inter-frame compression for dynamic point cloud geometry coding","authors":"Anique Akhtar, Zhu Li, Geert Van Der Auwera","year":"2024","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alexandre et al., 2022] Zaghetto Alexandre, Danillo Graziosi, and Tabatabai Ali. Dataset Split for AI-PCC. ISO/IEC JTC 1/SC 29/WG 7 m58754, 2022.","title":"Dataset Split for AI-PCC","authors":"Alexandre","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bjøntegaard, 2001] G. Bjøntegaard. Calculation of aver- age PSNR differences between RD-curves. In ITU-T SG 16/Q6, 13th VCEG Meeting. document VCEG-M33, April 2001.","title":"Calculation of average PSNR differences between RD-curves","authors":"G Bjøntegaard","year":"2001","venue":"ITU-T SG 16/Q6, 13th VCEG Meeting","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bross et al., 2021] Benjamin Bross, Jianle Chen, Jens- Rainer Ohm, Gary J. Sullivan, and Ye-Kui Wang. De- velopments in international video coding standardization after AVC, with an overview of versatile video coding (VVC). Proceedings of the IEEE, 109(9):1463-1493, 2021.","title":"Developments in international video coding standardization after AVC, with an overview of versatile video coding (VVC)","authors":"Benjamin Bross, Jianle Chen, Jens-Rainer Ohm, Gary J Sullivan, Ye-Kui Wang","year":"2021","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2019] Chao Cao, Marius Preda, and Titus Za- haria. 3D point cloud compression: A survey. In The 24th International Conference on 3D Web Technology, pages 1-9, 2019.","title":"3D point cloud compression: A survey","authors":"Chao Cao, Marius Preda, Titus Zaharia","year":"2019","venue":"The 24th International Conference on 3D Web Technology","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chang et al., 2015] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. ShapeNet: An information-rich 3D model reposi- tory. arXiv preprint arXiv:1512.03012, 2015.","title":"ShapeNet: An information-rich 3D model repository","authors":"Thomas Angel X Chang, Leonidas Funkhouser, Pat Guibas, Qixing Hanrahan, Zimo Huang, Silvio Li, Manolis Savarese, Shuran Savva, Hao Song, Su","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Choy et al., 2019] Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4D spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3075-3084, 2019.","title":"4D spatio-temporal convnets: Minkowski convolutional neural networks","authors":"Christopher Choy, Junyoung Gwak, Silvio Savarese","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"['Eon et al., 2017] Eugene d'Eon, Bob Harrison, Taos Myers, and Philip A Chou. 8i voxelized full bodies-a voxelized point cloud dataset. ISO/IEC JTC1/SC29 Joint WG11/WG1 (MPEG/JPEG) input doc- ument WG11M40059/WG1M74006, 2017.","title":"8i voxelized full bodies-a voxelized point cloud dataset","authors":"Eon","year":"2017","venue":"ISO/IEC JTC1/SC29 Joint WG11/WG1 (MPEG/JPEG) input","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fan et al., 2022a] Tingyu Fan, Linyao Gao, Yiling Xu, Zhu Li, and Dong Wang. D-DPCC: deep dynamic point cloud compression via 3D motion prediction. In Proceedings of the Thirty-First International Joint Conference on Artifi- cial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, pages 898-904, 2022.","title":"D-DPCC: deep dynamic point cloud compression via 3D motion prediction","authors":"Fan","year":"2022","venue":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022","doi":null,"url":null,"internal_url":"/proceedings/2022/126"},{"position":10,"raw":"[Fan et al., 2022b] Xiaoqing Fan, Ge Li, Dingquan Li, Yu- rui Ren, Wei Gao, and Thomas H. Li. Deep geometry post-processing for decompressed point clouds. In 2022 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6, 2022.","title":"Deep geometry post-processing for decompressed point clouds","authors":"Fan","year":"2022","venue":"2022 IEEE International Conference on Multimedia and Expo (ICME)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fu et al., 2022] Chunyang Fu, Ge Li, Rui Song, Wei Gao, and Shan Liu. Octattention: Octree-based large-scale con- texts model for point cloud compression. In Proceed- ings of the AAAI conference on artificial intelligence, vol- ume 36, pages 625-633, 2022.","title":"Octattention: Octree-based large-scale contexts model for point cloud compression","authors":"Chunyang Fu, Ge Li, Rui Song, Wei Gao, Shan Liu","year":"2022","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gao et al., 2021] Linyao Gao, Tingyu Fan, Jianqiang Wan, Yiling Xu, Jun Sun, and Zhan Ma. Point cloud geometry compression via neural graph sampling. In 2021 IEEE In- ternational Conference on Image Processing (ICIP), pages 3373-3377. IEEE, 2021.","title":"Point cloud geometry compression via neural graph sampling","authors":"Gao","year":"2021","venue":"IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guarda et al., 2019] André F. R. Guarda, Nuno M. M. Ro- drigues, and Fernando Pereira. Point cloud coding: Adopt- ing a deep learning-based approach. In 2019 Picture Cod- ing Symposium (PCS), pages 1-5, 2019.","title":"Point cloud coding: Adopting a deep learning-based approach","authors":"F R André, Guarda, M M Nuno, Fernando Rodrigues, Pereira","year":"2019","venue":"Picture Coding Symposium (PCS)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang and Liu, 2019] Tianxin Huang and Yong Liu. 3d point cloud geometry compression on deep learning. In Proceedings of the 27th ACM international conference on multimedia, pages 890-898, 2019.","title":"3d point cloud geometry compression on deep learning","authors":"Liu ; ] Tianxin Huang, Yong Huang, Liu","year":"2019","venue":"Proceedings of the 27th ACM international conference on multimedia","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jia et al., 2021] Wei Jia, Li Li, Zhu Li, and Shan Liu. Deep learning geometry compression artifacts removal for video-based point cloud compression. International Jour- nal of Computer Vision, 129(11):2947-2964, 2021.","title":"Deep learning geometry compression artifacts removal for video-based point cloud compression","authors":"Wei Jia, Li Li, Zhu Li, Shan Liu","year":"2021","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2023] Gexin Liu, Ruixiang Xue, Jiaxin Li, Dan- dan Ding, and Zhan Ma. GRNet: Geometry restoration for g-pcc compressed point clouds using auxiliary density sig- naling. IEEE Transactions on Visualization and Computer Graphics, 2023.","title":"GRNet: Geometry restoration for g-pcc compressed point clouds using auxiliary density signaling","authors":"Liu","year":"2023","venue":"IEEE Transactions on Visualization and Computer Graphics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[MPEG, 2022] MPEG. Description of exploration experi- ment 5.3 on AI-based dynamic PC coding. ISO/IEC JTC 1/SC 29/WG 7 N00344, 2022.","title":"MPEG. Description of exploration experiment 5.3 on AI-based dynamic PC coding","authors":"Mpeg","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nguyen and Kaup, 2022] Dat Thanh Nguyen and André Kaup. Learning-based lossless point cloud geometry cod- ing using sparse tensors. In 2022 IEEE International Con- ference on Image Processing (ICIP), pages 2341-2345. IEEE, 2022.","title":"Learning-based lossless point cloud geometry coding using sparse tensors","authors":"Kaup ; Nguyen, Thanh Dat, André Nguyen, Kaup","year":"2022","venue":"IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nguyen and Kaup, 2023] Dat Thanh Nguyen and André Kaup. Lossless point cloud geometry and attribute com- pression using a learned conditional probability model. IEEE Transactions on Circuits and Systems for Video Technology, 2023.","title":"Lossless point cloud geometry and attribute compression using a learned conditional probability model","authors":"Kaup ; Nguyen, Thanh Dat, André Nguyen, Kaup","year":"2023","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nguyen et al., 2021a] Dat Thanh Nguyen, Maurice Quach, Giuseppe Valenzise, and Pierre Duhamel. Learning-based lossless compression of 3D point cloud geometry. In ICASSP 2021 -2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4220-4224, 2021.","title":"Learning-based lossless compression of 3D point cloud geometry","authors":"Nguyen","year":"2021","venue":"ICASSP 2021 -2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nguyen et al., 2021b] Dat Thanh Nguyen, Maurice Quach, Giuseppe Valenzise, and Pierre Duhamel. Multiscale deep context modeling for lossless point cloud geometry com- pression. In 2021 IEEE International Conference on Mul- timedia & Expo Workshops (ICMEW), pages 1-6, 2021.","title":"Multiscale deep context modeling for lossless point cloud geometry compression","authors":"Nguyen","year":"2021","venue":"IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pang et al., 2022] Jiahao Pang, Muhammad Asad Lodhi, and Dong Tian. Grasp-net: Geometric residual analysis and synthesis for point cloud compression. In Proceedings of the 1st International Workshop on Advances in Point Cloud Compression, Processing and Analysis, pages 11- 19, 2022.","title":"Grasp-net: Geometric residual analysis and synthesis for point cloud compression","authors":"Pang","year":"2022","venue":"Proceedings of the 1st International Workshop on Advances in Point Cloud Compression, Processing and Analysis","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Quach et al., 2019] Maurice Quach, Giuseppe Valenzise, and Frederic Dufaux. Learning convolutional transforms for lossy point cloud geometry compression. In 2019 IEEE International Conference on Image Processing (ICIP), pages 4320-4324, 2019.","title":"Learning convolutional transforms for lossy point cloud geometry compression","authors":"Maurice Quach, Giuseppe Valenzise, Frederic Dufaux","year":"2019","venue":"2019 IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Que et al., 2021] Zizheng Que, Guo Lu, and Dong Xu. Voxelcontext-net: An octree based framework for point cloud compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6042-6051, 2021.","title":"Voxelcontext-net: An octree based framework for point cloud compression","authors":"Que","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Song et al., 2023] Rui Song, Chunyang Fu, Shan Liu, and Ge Li. Efficient hierarchical entropy model for learned point cloud compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14368-14377, 2023.","title":"Efficient hierarchical entropy model for learned point cloud compression","authors":"Rui Song, Chunyang Fu, Shan Liu, Ge Li","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sullivan et al., 2012] Gary J Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. Overview of the high efficiency video coding (HEVC) standard. IEEE Trans- actions on Circuits and Systems for Video Technology, 22(12):1649-1668, 2012.","title":"Overview of the high efficiency video coding (HEVC) standard","authors":"Jens-Rainer Gary J Sullivan, Woo-Jin Ohm, Thomas Han, Wiegand","year":"2012","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Szegedy et al., 2016] Christian Szegedy, Vincent Van- houcke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818-2826, 2016.","title":"Rethinking the inception architecture for computer vision","authors":"Szegedy","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2021a] Jianqiang Wang, Dandan Ding, Zhu Li, and Zhan Ma. Multiscale point cloud geometry compres- sion. In 2021 Data Compression Conference (DCC), pages 73-82, 2021.","title":"Multiscale point cloud geometry compression","authors":"Wang","year":"2021","venue":"2021 Data Compression Conference (DCC)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2021b] Jianqiang Wang, Hao Zhu, Haojie Liu, and Zhan Ma. Lossy point cloud geometry compression via end-to-end learning. IEEE Transactions on Circuits and Systems for Video Technology, 31(12):4909-4923, 2021.","title":"Lossy point cloud geometry compression via end-to-end learning","authors":"Wang","year":"2021","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2022] Jianqiang Wang, Dandan Ding, Zhu Li, Xiaoxing Feng, Chuntong Cao, and Zhan Ma. Sparse tensor-based multiscale representation for point cloud ge- ometry compression. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, Nov. 2022.","title":"Sparse tensor-based multiscale representation for point cloud geometry compression","authors":"Jianqiang Wang, Dandan Ding, Zhu Li, Xiaoxing Feng, Chuntong Cao, Zhan Ma","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wen et al., 2020] Xuanzheng Wen, Xu Wang, Junhui Hou, Lin Ma, Yu Zhou, and Jianmin Jiang. Lossy geometry compression of 3D point cloud data via an adaptive octree- guided network. In 2020 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6, 2020.","title":"Lossy geometry compression of 3D point cloud data via an adaptive octreeguided network","authors":"Xuanzheng Wen, Xu Wang, Junhui Hou, Lin Ma, Yu Zhou, Jianmin Jiang","year":"2020","venue":"IEEE International Conference on Multimedia and Expo (ICME)","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xia et al., 2023] Shuting Xia, Tingyu Fan, Yiling Xu, Jenq- Neng Hwang, and Zhu Li. Learning dynamic point cloud compression via hierarchical inter-frame block matching. In Proceedings of the 31st ACM International Conference on Multimedia, pages 7993-8003, 2023.","title":"Learning dynamic point cloud compression via hierarchical inter-frame block matching","authors":"Shuting Xia, Tingyu Fan, Yiling Xu, Jenq-Neng Hwang, Zhu Li","year":"2023","venue":"Proceedings of the 31st ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xu et al., 2017] Yi Xu, Yao Lu, and Ziyu Wen. Owlii dynamic human mesh sequence dataset. ISO/IEC JTC1/SC29/WG11 m41658, 120th MPEG Meeting, 2017.","title":"Owlii dynamic human mesh sequence dataset","authors":"Yi Xu, Yao Lu, Ziyu Wen","year":"2017","venue":"120th MPEG Meeting","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yan et al., 2019] Wei Yan, Yiting shao, Shan Liu, Thomas H Li, Zhu Li, and Ge Li. Deep autoencoder-based lossy geometry compression for point clouds, 2019.","title":"Deep autoencoder-based lossy geometry compression for point clouds","authors":"Wei Yan, Yiting Shao, Shan Liu, Thomas H Li, Zhu Li, Ge Li","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yang et al., 2021] Ren Yang, Radu Timofte, et al. NTIRE 2021 challenge on quality enhancement of compressed video: Methods and results. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2021.","title":"NTIRE 2021 challenge on quality enhancement of compressed video: Methods and results","authors":"Ren Yang, Radu Timofte","year":"2021","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2022] Junteng Zhang, Gexin Liu, Dandan Ding, and Zhan Ma. Transformer and upsampling-based point cloud compression. In Proceedings of the 1st Inter- national Workshop on Advances in Point Cloud Compres- sion, Processing and Analysis, pages 33-39, 2022.","title":"Transformer and upsampling-based point cloud compression","authors":"Zhang","year":"2022","venue":"Proceedings of the 1st International Workshop on Advances in Point Cloud Compression, Processing and Analysis","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2023] Junzhe Zhang, Tong Chen, Dandan Ding, and Zhan Ma. G-PCC++: Enhanced geometry- based point cloud compression. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1352-1363, 2023.","title":"G-PCC++: Enhanced geometrybased point cloud compression","authors":"Junzhe Zhang, Tong Chen, Dandan Ding, Zhan Ma","year":"2023","venue":"Proceedings of the 31st ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}