{"type":"ScholarlyArticle","title":"Deep Cascade Generation on Point Sets","authors":[{"name":"Kaiqi Wang","slug":"kaiqi-wang","given":"Kaiqi","family":"Wang","affiliation":"South China University of Technology"},{"name":"Ke Chen","slug":"ke-chen","given":"Ke","family":"Chen","affiliation":"South China University of Technology"},{"name":"Kui Jia","slug":"kui-jia","given":"Kui","family":"Jia","affiliation":"South China University of Technology"}],"abstract":"This paper proposes a deep cascade network to generate 3D geometry of an object on a point cloud, consisting of a set of permutation-insensitive points. Such a surface representation is easy to learn from, but inhibits exploiting rich low-dimensional topological manifolds of the object shape due to lack of geometric connectivity. For benefiting from its simple structure yet utilizing rich neighborhood information across points, this paper proposes a two-stage cascade model on point sets. Specifically, our method adopts the state-of-the-art point set autoencoder to generate a sparsely coarse shape first, and then locally refines it by encoding neighborhood connectivity on a graph representation. An ensemble of sparse refined surface is designed to alleviate the suffering from local minima caused by modeling complex geometric manifolds. Moreover, our model develops a dynamically-weighted loss function for jointly penalizing the generation output of cascade levels at different training stages in a coarse-to-fine manner. Comparative evaluation on the publicly benchmarking ShapeNet dataset demonstrates superior performance of the proposed model to the state-of-the-art methods on both single-view shape reconstruction and shape autoencoding applications.","doi":"10.24963/ijcai.2019/517","url":"https://ijcai.org/proceedings/2019/517","pdf":"https://ijcai.org/proceedings/2019/0517.pdf","markdown":"https://ijcai.org/proceedings/2019/517.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/517","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":517,"pages":{"first":3726,"last":3732},"keywords":["Computer Vision: 2D and 3D Computer Vision","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Bronstein et al., 2017] Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, pages 18-42, 2017.","title":"Geometric deep learning: going beyond euclidean data","authors":"Joan Michael M Bronstein, Yann Bruna, Arthur Lecun, Pierre Szlam, Vandergheynst","year":"2017","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bruna et al., 2013] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and local- ly connected networks on graphs. arXiv preprint arX- iv:1312.6203, 2013.","title":"Spectral networks and local-ly connected networks on graphs","authors":"Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann Lecun","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chang et al., 2015] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model reposito- ry. arXiv preprint arXiv:1512.03012, 2015.","title":"Shapenet: An information-rich 3d model repository","authors":"Thomas Angel X Chang, Leonidas Funkhouser, Pat Guibas, Qixing Hanrahan, Zimo Huang, Silvio Li, Manolis Savarese, Shuran Savva, Hao Song, Su","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Choy et al., 2016] Christopher B Choy, Danfei Xu, JunY- oung Gwak, Kevin Chen, and Silvio Savarese. 3D-R2N2: A unified approach for single and multi-view 3d object re- construction. In ECCV, pages 628-644, 2016.","title":"3D-R2N2: A unified approach for single and multi-view 3d object reconstruction","authors":"Danfei Christopher B Choy, Junyoung Xu, Kevin Gwak, Silvio Chen, Savarese","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres- son, and Pierre Vandergheynst. Convolutional neural net- works on graphs with fast localized spectral filtering. In NIPS, pages 3844-3852, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fan et al., 2017] Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object re- construction from a single image. In CVPR, pages 605- 613, 2017.","title":"A point set generation network for 3d object reconstruction from a single image","authors":"Fan","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Girdhar et al., 2016] Rohit Girdhar, David F Fouhey, Mikel Rodriguez, and Abhinav Gupta. Learning a predictable and generative vector representation for objects. In ECCV, pages 484-499, 2016.","title":"Learning a predictable and generative vector representation for objects","authors":"Rohit Girdhar, David F Fouhey, Mikel Rodriguez, Abhinav Gupta","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Groueix et al., 2018] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. A papier-mâché approach to learning 3d surface generation. In CVPR, pages 216-224, 2018.","title":"A papier-mâché approach to learning 3d surface generation","authors":"Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, Mathieu Aubry","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2017] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connect- ed convolutional networks. In CVPR, pages 4700-4708, 2017.","title":"Densely connected convolutional networks","authors":"Gao Huang, Zhuang Liu, Laurens Van Der Maaten, Kilian Q Weinberger","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kar et al., 2017] Abhishek Kar, Christian Häne, and Jiten- dra Malik. Learning a multi-view stereo machine. In NIPS, pages 365-376, 2017.","title":"Learning a multi-view stereo machine","authors":"Abhishek Kar, Christian Häne, Jitendra Malik","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Karpathy et al., 2014] Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. Large-scale video classification with convolu- tional neural networks. In CVPR, pages 1725-1732, 2014.","title":"Large-scale video classification with convolutional neural networks","authors":"Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, Li Fei-Fei","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kazhdan et al., 2006] Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. Poisson surface reconstruction. In SGP, volume 7, 2006.","title":"Poisson surface reconstruction","authors":"Matthew Michael Kazhdan, Hugues Bolitho, Hoppe","year":"2006","venue":"SGP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2018] Jiaxin Li, Ben M Chen, and Gim Hee Lee. So-net: Self-organizing network for point cloud analysis. In CVPR, pages 9397-9406, 2018.","title":"So-net: Self-organizing network for point cloud analysis","authors":"Jiaxin Li, Ben M Chen, Gim Hee, Lee","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pan et al., 2018] Junyi Pan, Jun Li, Xiaoguang Han, and Kui Jia. Residual MeshNet: Learning to deform meshes for single-view 3d reconstruction. In 3DV, pages 719-727, 2018.","title":"Residual MeshNet: Learning to deform meshes for single-view 3d reconstruction","authors":"Junyi Pan, Jun Li, Xiaoguang Han, Kui Jia","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Qi et al., 2017a] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 652-660, 2017.","title":"Pointnet: Deep learning on point sets for 3d classification and segmentation","authors":"Qi","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qi et al., 2017b] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NIP- S, pages 5099-5108, 2017.","title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","authors":"Qi","year":"2017","venue":"NIP-S","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Simon et al., 2018] Martin Simon, Stefan Milz, Karl A- mende, and Horst-Michael Gross. Complex-YOLO: An euler-region-proposal for real-time 3d object detection on point clouds. In ECCV, pages 0-0, 2018.","title":"Complex-YOLO: An euler-region-proposal for real-time 3d object detection on point clouds","authors":"Martin Simon, Stefan Milz, Karl Amende, Horst-Michael Gross","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tang et al., 2009] Min Tang, Minkyoung Lee, and Young J Kim. Interactive hausdorff distance computation for gen- eral polygonal models. In TOG, page 74, 2009.","title":"Interactive hausdorff distance computation for general polygonal models","authors":"Tang","year":"2009","venue":"TOG","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tang et al., 2019] Jiapeng Tang, Xiaoguang Han, Junyi Pan, Kui Jia, and Xin Tong. A skeleton-bridged deep learn- ing approach for generating meshes of complex topolo- gies from single RGB images. arXiv preprint arX- iv:1903.04704, 2019.","title":"A skeleton-bridged deep learning approach for generating meshes of complex topologies from single RGB images","authors":"Tang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tatarchenko et al., 2016] Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. Multi-view 3d models from single images with a convolutional network. In ECCV, pages 322-337, 2016.","title":"Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. Multi-view 3d models from single images with a convolutional network","authors":"Tatarchenko","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tatarchenko et al., 2017] Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs. In ICCV, pages 2088-2096, 2017.","title":"Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs","authors":"Tatarchenko","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tulsiani et al., 2017] Shubham Tulsiani, Tinghui Zhou, Alexei A Efros, and Jitendra Malik. Multi-view super- vision for single-view reconstruction via differentiable ray consistency. In CVPR, pages 2626-2634, 2017.","title":"Multi-view supervision for single-view reconstruction via differentiable ray consistency","authors":"Shubham Tulsiani, Tinghui Zhou, Alexei A Efros, Jitendra Malik","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2018a] Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In ECCV, pages 52-67, 2018.","title":"Pixel2mesh: Generating 3d mesh models from single rgb images","authors":"Wang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2018b] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. arXiv preprint arXiv:1801.07829, 2018.","title":"Dynamic graph cnn for learning on point clouds","authors":"Wang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yan et al., 2016] Xinchen Yan, Jimei Yang, Ersin Yumer, Y- ijie Guo, and Honglak Lee. Perspective transformer nets: Learning single-view 3d object reconstruction without 3d supervision. In NIPS, pages 1696-1704, 2016.","title":"Perspective transformer nets: Learning single-view 3d object reconstruction without 3d supervision","authors":"Yan","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2017] Bo Yang, Hongkai Wen, Sen Wang, Ronald Clark, Andrew Markham, and Niki Trigoni. 3d object reconstruction from a single depth view with adver- sarial learning. In ICCV, pages 679-688, 2017.","title":"3d object reconstruction from a single depth view with adversarial learning","authors":"Bo Yang, Hongkai Wen, Sen Wang, Ronald Clark, Andrew Markham, Niki Trigoni","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang et al., 2018] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In CVPR, pages 206-215, 2018.","title":"Foldingnet: Point cloud auto-encoder via deep grid deformation","authors":"Yaoqing Yang, Chen Feng, Yiru Shen, Dong Tian","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yuan et al., 2018] Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. PCN: Point com- pletion network. In 3DV, pages 728-737, 2018.","title":"PCN: Point completion network","authors":"Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, Martial Hebert","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}