{"type":"ScholarlyArticle","title":"BPNet: Bézier Primitive Segmentation on 3D Point Clouds","authors":[{"name":"Rao Fu","slug":"rao-fu","given":"Rao","family":"Fu","affiliation":"Inria , France; Geometry Factory , France"},{"name":"Cheng Wen","slug":"cheng-wen","given":"Cheng","family":"Wen","affiliation":"The University of Sydney , Australia"},{"name":"Qian Li","slug":"qian-li","given":"Qian","family":"Li","affiliation":"Inria , France"},{"name":"Xiao Xiao","slug":"xiao-xiao","given":"Xiao","family":"Xiao","affiliation":"Shanghai Jiao Tong University , P. R. China"},{"name":"Pierre Alliez","slug":"pierre-alliez","given":"Pierre","family":"Alliez","affiliation":"Inria , France"}],"abstract":"This paper proposes BPNet, a novel end-to-end deep learning framework to learn Bézier primitive segmentation on 3D point clouds. The existing works treat different primitive types separately, thus limiting them to finite shape categories. To address this issue, we seek a generalized primitive segmentation on point clouds. Taking inspiration from Bézier decomposition on NURBS models, we transfer it to guide point cloud segmentation casting off primitive types. A joint optimization framework is proposed to learn Bézier primitive segmentation and geometric fitting simultaneously on a cascaded architecture. Specifically, we introduce a soft voting regularizer to improve primitive segmentation and propose an auto-weight embedding module to cluster point features, making the network more robust and generic. We also introduce a reconstruction module where we successfully process multiple CAD models with different primitives simultaneously. We conducted extensive experiments on the synthetic ABC dataset and real-scan datasets to validate and compare our approach with different baseline methods. Experiments show superior performance over previous work in terms of segmentation, with a substantially faster inference speed.","doi":"10.24963/ijcai.2023/84","url":"https://ijcai.org/proceedings/2023/84","pdf":"https://ijcai.org/proceedings/2023/0084.pdf","markdown":"https://ijcai.org/proceedings/2023/84.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/84","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":84,"pages":{"first":754,"last":762},"keywords":["Computer Vision: CV: Segmentation","Computer Vision: CV: Machine learning for vision","Computer Vision: CV: 3D computer vision"],"videos":[],"references":[{"position":1,"raw":"[Bednarik et al., 2020] Jan Bednarik, Shaifali Parashar, Er- han Gundogdu, Mathieu Salzmann, and Pascal Fua. Shape reconstruction by learning differentiable surface represen- tations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4716-4725, 2020.","title":"Shape reconstruction by learning differentiable surface representations","authors":"Bednarik","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[CGAL, 2009] CGAL. The computational geometry algo- rithms library. https://www.cgal.org/, 2009. Accessed: 2023-05-15.","title":"CGAL. The computational geometry algorithms library","authors":"Cgal","year":"2009","venue":null,"doi":null,"url":"https://www.cgal.org/,2009","internal_url":null},{"position":3,"raw":"[Chen et al., 2009] Xiaobai Chen, Aleksey Golovinskiy, and Thomas Funkhouser. A benchmark for 3d mesh segmenta- tion. ACM Transactions on Graphics (TOG), 28(3):1-12, 2009.","title":"A benchmark for 3d mesh segmentation","authors":"Xiaobai Chen, Aleksey Golovinskiy, Thomas Funkhouser","year":"2009","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dai et al., 2016] Jifeng Dai, Kaiming He, and Jian Sun. Instance-aware semantic segmentation via multi-task net- work cascades. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 3150-3158, 2016.","title":"Instance-aware semantic segmentation via multi-task network cascades","authors":"Jifeng Dai, Kaiming He, Jian Sun","year":"2016","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2020] Zhantao Deng, Jan Bednařík, Mathieu Salzmann, and Pascal Fua. Better patch stitching for para- metric surface reconstruction. In 2020 International Con- ference on 3D Vision (3DV), pages 593-602. IEEE, 2020.","title":"Better patch stitching for parametric surface reconstruction","authors":"Deng","year":"2020","venue":"International Conference on 3D Vision (3DV)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Engelmann et al., 2020] Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, and Matthias Nießner. 3d-mpa: Multi-proposal aggregation for 3d semantic instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9031-9040, 2020.","title":"3d-mpa: Multi-proposal aggregation for 3d semantic instance segmentation","authors":"Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Falcidieno, 2004] Bianca Falcidieno. Aim@shape project presentation. In Proceedings of Shape Modeling Inter- national (SMI), pages 329-329. IEEE Computer Society, 2004.","title":"Aim@shape project presentation","authors":"Bianca Falcidieno","year":"2004","venue":"Proceedings of Shape Modeling International (SMI)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fan et al., 2017] Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object re- construction from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 605-613, 2017.","title":"A point set generation network for 3d object reconstruction from a single image","authors":"Fan","year":"2017","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fischler and Bolles, 1981] Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381- 395, 1981.","title":"Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography","authors":"A Martin, Robert C Fischler, Bolles","year":"1981","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Groueix et al., 2018] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. A papier-mâché approach to learning 3d surface generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (TPAMI), pages 216-224, 2018.","title":"A papier-mâché approach to learning 3d surface generation","authors":"Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, Mathieu Aubry","year":"2018","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (TPAMI)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2020] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learn- ing for 3d point clouds: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020.","title":"Deep learning for 3d point clouds: A survey","authors":"Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, Mohammed Bennamoun","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2021] Jingwei Huang, Yanfeng Zhang, and Mingwei Sun. Primitivenet: Primitive instance segmenta- tion with local primitive embedding under adversarial met- ric. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), pages 15343-15353, 2021.","title":"Primitivenet: Primitive instance segmentation with local primitive embedding under adversarial metric","authors":"Jingwei Huang, Yanfeng Zhang, Mingwei Sun","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jiang et al., 2020] Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, and Jiaya Jia. Pointgroup: Dual-set point grouping for 3d instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4867- 4876, 2020.","title":"Pointgroup: Dual-set point grouping for 3d instance segmentation","authors":"Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kaiser et al., 2019] Adrien Kaiser, Jose Alonso Ybanez Zepeda, and Tamy Boubekeur. A survey of simple geometric primitives detection methods for captured 3d data. Computer Graphics Forum (CGF), 38(1):167-196, 2019.","title":"A survey of simple geometric primitives detection methods for captured 3d data","authors":"Adrien Kaiser, Jose Alonso Ybanez, Tamy Zepeda, Boubekeur","year":"2019","venue":"Computer Graphics Forum (CGF)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Koch et al., 2019] Sebastian Koch, Albert Matveev, Zhong- shi Jiang, Francis Williams, Alexey Artemov, Evgeny Bur- naev, Marc Alexa, Denis Zorin, and Daniele Panozzo. Abc: A big cad model dataset for geometric deep learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.","title":"Abc: A big cad model dataset for geometric deep learning","authors":"Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, Daniele Panozzo","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krähenbühl and Koltun, 2013] Philipp Krähenbühl and Vladlen Koltun. Parameter learning and convergent infer- ence for dense random fields. In International Conference on Machine Learning (ICML), pages 513-521. PMLR, 2013.","title":"Philipp Krähenbühl and Vladlen Koltun. Parameter learning and convergent inference for dense random fields","authors":"Koltun Krähenbühl","year":"2013","venue":"International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kuhn, 1955] Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83-97, 1955.","title":"The hungarian method for the assignment problem","authors":"Harold W Kuhn","year":"1955","venue":"Naval research logistics quarterly","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lévy et al., 2002] Bruno Lévy, Sylvain Petitjean, Nicolas Ray, and Jérome Maillot. Least squares conformal maps for automatic texture atlas generation. ACM Transactions on Graphics (TOG), 21(3):362-371, 2002.","title":"Least squares conformal maps for automatic texture atlas generation","authors":"Bruno Lévy, Nicolas Sylvain Petitjean, Jérome Ray, Maillot","year":"2002","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2019] Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, and Leonidas J Guibas. Supervised fit- ting of geometric primitives to 3d point clouds. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 2652-2660, 2019.","title":"Supervised fitting of geometric primitives to 3d point clouds","authors":"Li","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense ob- ject detection. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV), pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Marshall et al., 2001] David Marshall, Gabor Lukacs, and Ralph Martin. Robust segmentation of primitives from range data in the presence of geometric degeneracy. IEEE Transactions on pattern analysis and machine intelligence (TPAMI), 23(3):304-314, 2001.","title":"Robust segmentation of primitives from range data in the presence of geometric degeneracy","authors":"David Marshall, Gabor Lukacs, Ralph Martin","year":"2001","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[OpenCascade, 2018] OpenCascade. Open cascade technol- ogy occt. https://www.opencascade.com/, 2018. Ac- cessed: 2023-05-15.","title":"OpenCascade. Open cascade technology occt","authors":"Opencascade","year":"2018","venue":null,"doi":null,"url":"https://www.opencascade.com/","internal_url":null},{"position":23,"raw":"[Rabbani et al., 2007] Tahir Rabbani, Sander Dijkman, Frank van den Heuvel, and George Vosselman. An integrated approach for modelling and global registration of point clouds. ISPRS journal of Photogrammetry and Remote Sensing, 61(6):355-370, 2007.","title":"An integrated approach for modelling and global registration of point clouds","authors":"Tahir Rabbani, Sander Dijkman, Frank Van Den, George Heuvel, Vosselman","year":"2007","venue":"ISPRS journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Schnabel et al., 2007] Ruwen Schnabel, Roland Wahl, and Reinhard Klein. Efficient ransac for point-cloud shape detection. Computer Graphics Forum (CGF), 26(2):214- 226, 2007.","title":"Efficient ransac for point-cloud shape detection","authors":"Ruwen Schnabel, Roland Wahl, Reinhard Klein","year":"2007","venue":"Computer Graphics Forum (CGF)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sharma et al., 2020] Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, and Radomír Měch. Parsenet: A parametric surface fitting net- work for 3d point clouds. In European Conference on Computer Vision (ECCV), pages 261-276. Springer, 2020.","title":"Parsenet: A parametric surface fitting network for 3d point clouds","authors":"Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, Radomír Měch","year":"2020","venue":"European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tulsiani et al., 2017] Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2635-2643, 2017.","title":"Learning shape abstractions by assembling volumetric primitives","authors":"Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A Efros, Jitendra Malik","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2018] Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Neumann. Sgpn: Similarity group pro- posal network for 3d point cloud instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2569-2578, 2018.","title":"Sgpn: Similarity group proposal network for 3d point cloud instance segmentation","authors":"Wang","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2019] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (TOG), 38(5):1-12, 2019.","title":"Dynamic graph cnn for learning on point clouds","authors":"Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, Justin M Solomon","year":"2019","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yan and Yang, 2021] Siming Yan and Zhenpei Yang. Hp- net: Deep primitive segmentation using hybrid representa- tions. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), 2021.","title":"Hpnet: Deep primitive segmentation using hybrid representations","authors":"Yan, Yang ;, Siming Yan, Zhenpei Yang","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yang et al., 2018] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 206-215, 2018.","title":"Foldingnet: Point cloud auto-encoder via deep grid deformation","authors":"Yaoqing Yang, Chen Feng, Yiru Shen, Dong Tian","year":"2018","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2019] Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, and Niki Trigoni. Learning object bounding boxes for 3d instance segmentation on point clouds. Advances in neural infor- mation processing systems (NIPS), 32, 2019.","title":"Learning object bounding boxes for 3d instance segmentation on point clouds","authors":"Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni","year":"2019","venue":"Advances in neural information processing systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yi et al., 2018] Li Yi, Haibin Huang, Difan Liu, Evangelos Kalogerakis, Hao Su, and Leonidas Guibas. Deep part induction from articulated object pairs. ACM Transactions on Graphics (TOG), 37(6):1-15, 2018.","title":"Deep part induction from articulated object pairs","authors":"Li Yi, Haibin Huang, Difan Liu, Evangelos Kalogerakis, Hao Su, Leonidas Guibas","year":"2018","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yi et al., 2019] Li Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas J Guibas. Gspn: Generative shape proposal network for 3d instance segmentation in point cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3947-3956, 2019.","title":"Gspn: Generative shape proposal network for 3d instance segmentation in point cloud","authors":"Li Yi, Wang Zhao, He Wang, Minhyuk Sung, Leonidas J Guibas","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang and Yang, 2021] Yu Zhang and Qiang Yang. A sur- vey on multi-task learning. IEEE Transactions on Knowl- edge and Data Engineering, 34(12):5586-5609, 2021.","title":"A survey on multi-task learning","authors":"Yu Zhang, Qiang Yang","year":"2021","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2014] Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial land- mark detection by deep multi-task learning. In European Conference on Computer Vision (ECCV), pages 94-108.","title":"Facial landmark detection by deep multi-task learning","authors":"Zhang","year":"2014","venue":"European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zou et al., 2017] Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 3d-prnn: Generating shape primitives with recurrent neural networks. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 900-909, 2017.","title":"3d-prnn: Generating shape primitives with recurrent neural networks","authors":"Zou","year":"2017","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}