{"type":"ScholarlyArticle","title":"AttAN: Attention Adversarial Networks for 3D Point Cloud Semantic Segmentation","authors":[{"name":"Gege Zhang","slug":"gege-zhang","given":"Gege","family":"Zhang","affiliation":"Xidian University"},{"name":"Qinghua Ma","slug":"qinghua-ma","given":"Qinghua","family":"Ma","affiliation":"Xidian University"},{"name":"Licheng Jiao","slug":"licheng-jiao","given":"Licheng","family":"Jiao","affiliation":"Xidian University"},{"name":"Fang Liu","slug":"fang-liu","given":"Fang","family":"Liu","affiliation":""},{"name":"Qigong Sun","slug":"qigong-sun","given":"Qigong","family":"Sun","affiliation":"Xidian University"}],"abstract":"3D point cloud semantic segmentation has attracted wide attention with its extensive applications in autonomous driving, AR/VR, and robot sensing fields. However, in existing methods, each point in the segmentation results is predicted independently from each other. This property causes the non-contiguity of label sets in three-dimensional space and produces many noisy label points, which hinders the improvement of segmentation accuracy. To address this problem, we first extend adversarial learning to this task and propose a novel framework Attention Adversarial Networks (AttAN). With high-order correlations in label sets learned from the adversarial learning, segmentation network can predict labels closer to the real ones and correct noisy results. Moreover, we design an additive attention block for the segmentation network, which is used to automatically focus on regions critical to the segmentation task by learning the correlation between multi-scale features. Adversarial learning, which explores the underlying relationship between labels in high-dimensional space, opens up a new way in 3D point cloud semantic segmentation. Experimental results on ScanNet and S3DIS datasets show that this framework effectively improves the segmentation quality and outperforms other state-of-the-art methods.","doi":"10.24963/ijcai.2020/110","url":"https://ijcai.org/proceedings/2020/110","pdf":"https://ijcai.org/proceedings/2020/0110.pdf","markdown":"https://ijcai.org/proceedings/2020/110.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/110","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":110,"pages":{"first":789,"last":796},"keywords":["Machine Learning: Adversarial Machine Learning","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: 2D and 3D Computer Vision","Machine Learning: Deep Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24627/file_24627.m3u8","length_sec":298,"provider":"hls"}],"references":[{"position":1,"raw":"[Anderson et al., 2018] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for im- age captioning and visual question answering. In Proceed- ings of the IEEE Conference on Computer Vision and Pat- tern Recognition, pages 6077-6086, 2018.","title":"Bottom-up and top-down attention for image captioning and visual question answering","authors":"Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, Lei Zhang","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.","title":"Wasserstein gan","authors":"Martin Arjovsky, Soumith Chintala, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Armeni et al., 2016] Iro Armeni, Ozan Sener, Amir R Za- mir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Sil- vio Savarese. 3d semantic parsing of large-scale indoor spaces. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 1534-1543, 2016.","title":"3d semantic parsing of large-scale indoor spaces","authors":"Iro Armeni, Ozan Sener, Helen Amir R Zamir, Ioannis Jiang, Martin Brilakis, Silvio Fischer, Savarese","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2018] Liqun Chen, Shuyang Dai, Chenyang Tao, Haichao Zhang, Zhe Gan, Dinghan Shen, Yizhe Zhang, Guoyin Wang, Ruiyi Zhang, and Lawrence Carin. Adversarial text generation via feature-mover's distance. In Advances in Neural Information Processing Systems, pages 4666-4677, 2018.","title":"Adversarial text generation via feature-mover's distance","authors":"Liqun Chen, Shuyang Dai, Chenyang Tao, Haichao Zhang, Zhe Gan, Dinghan Shen, Yizhe Zhang, Guoyin Wang, Ruiyi Zhang, Lawrence Carin","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dai and Nießner, 2018] Angela Dai and Matthias Nießner. 3dmv: Joint 3d-multi-view prediction for 3d semantic scene segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 452-468, 2018.","title":"3dmv: Joint 3d-multi-view prediction for 3d semantic scene segmentation","authors":"Angela Dai, Matthias Nießner","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dai et al., 2017] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5828- 5839, 2017.","title":"Scannet: Richly-annotated 3d reconstructions of indoor scenes","authors":"Angela Dai, X Angel, Manolis Chang, Maciej Savva, Thomas Halber, Matthias Funkhouser, Nießner","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fu et al., 2018] Jun Fu, Jing Liu, Haijie Tian, Zhiwei Fang, and Hanqing Lu. Dual attention network for scene seg- mentation. arXiv preprint arXiv:1809.02983, 2018.","title":"Dual attention network for scene segmentation","authors":"Jun Fu, Jing Liu, Haijie Tian, Zhiwei Fang, Hanqing Lu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2019] Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, and Leonidas J Guibas. Texturenet: Consistent local parametrizations for learning from high-resolution signals on meshes. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4440-4449, 2019.","title":"Texturenet: Consistent local parametrizations for learning from high-resolution signals on meshes","authors":"Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, Leonidas J Guibas","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jang et al., 2016] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016.","title":"Categorical reparameterization with gumbel-softmax","authors":"Eric Jang, Shixiang Gu, Ben Poole","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2019] Li Jiang, Hengshuang Zhao, Shu Liu, Xi- aoyong Shen, Chi-Wing Fu, and Jiaya Jia. Hierarchical point-edge interaction network for point cloud semantic segmentation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 10432-10440, 2019.","title":"Hierarchical point-edge interaction network for point cloud semantic segmentation","authors":"Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia","year":"2019","venue":"IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Landrieu and Simonovsky, 2018] Loic Landrieu and Martin Simonovsky. Large-scale point cloud semantic segmenta- tion with superpoint graphs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4558-4567, 2018.","title":"Large-scale point cloud semantic segmentation with superpoint graphs","authors":"Simonovsky Landrieu, Loic Landrieu, Martin Simonovsky","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018a] Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabas Poczos, and Ruslan Salakhutdinov. Point cloud gan. arXiv preprint arXiv:1810.05795, 2018.","title":"Point cloud gan","authors":"Li","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2018b] Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointcnn: Convolu- tion on x-transformed points. In Advances in Neural In- formation Processing Systems, pages 820-830, 2018.","title":"Pointcnn: Convolution on x-transformed points","authors":"Li","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2019] Jing Li, Deheng Ye, and Shuo Shang. Ad- versarial transfer for named entity boundary detection with pointer networks. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 5053- 5059. AAAI Press, 2019.","title":"Adversarial transfer for named entity boundary detection with pointer networks","authors":"Jing Li, Deheng Ye, Shuo Shang","year":"2019","venue":"Proceedings of the 28th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2019/702"},{"position":15,"raw":"[Luc et al., 2016] Pauline Luc, Camille Couprie, Soumith Chintala, and Jakob Verbeek. Semantic segmen- tation using adversarial networks. arXiv preprint arXiv:1611.08408, 2016.","title":"Semantic segmentation using adversarial networks","authors":"Pauline Luc, Camille Couprie, Soumith Chintala, Jakob Verbeek","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Oktay et al., 2018] Ozan Oktay, Jo Schlemper, Loic Le Fol- goc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. Attention u-net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, 2018.","title":"Attention u-net: learning where to look for the pancreas","authors":"Ozan Oktay, Jo Schlemper, Le Loic, Matthew Folgoc, Mattias Lee, Kazunari Heinrich, Kensaku Misawa, Steven Mori, Nils Y Mcdonagh, Bernhard Hammerla, Kainz","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Qi et al., 2017a] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 652-660, 2017.","title":"Pointnet: Deep learning on point sets for 3d classification and segmentation","authors":"Qi","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qi et al., 2017b] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Ad- vances in Neural Information Processing Systems, pages 5099-5108, 2017.","title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","authors":"Qi","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shi et al., 2019] Shaoshuai Shi, Xiaogang Wang, and Hong- sheng Li. Pointrcnn: 3d object proposal generation and detection from point cloud. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-779, 2019.","title":"Pointrcnn: 3d object proposal generation and detection from point cloud","authors":"Shi","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Su et al., 2015] Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multi-view con- volutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on com- puter vision, pages 945-953, 2015.","title":"Multi-view convolutional neural networks for 3d shape recognition","authors":"Su","year":"2015","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Su et al., 2018] Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, and Jan Kautz. Splatnet: Sparse lattice networks for point cloud processing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2530-2539, 2018.","title":"Splatnet: Sparse lattice networks for point cloud processing","authors":"Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tatarchenko et al., 2018] Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, and Qian-Yi Zhou. Tangent convolutions for dense prediction in 3d. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3887-3896, 2018.","title":"Tangent convolutions for dense prediction in 3d","authors":"Tatarchenko","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tchapmi et al., 2017] Lyne Tchapmi, Christopher Choy, Iro Armeni, JunYoung Gwak, and Silvio Savarese. Segcloud: Semantic segmentation of 3d point clouds. In 2017 Inter- national Conference on 3D Vision (3DV), pages 537-547. IEEE, 2017.","title":"Segcloud: Semantic segmentation of 3d point clouds","authors":"Lyne Tchapmi, Christopher Choy, Iro Armeni, Junyoung Gwak, Silvio Savarese","year":"2017","venue":"2017 International Conference on 3D Vision (3DV)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Thomas et al., 2019] Hugues Thomas, Charles R Qi, Jean- Emmanuel Deschaud, Beatriz Marcotegui, Franc ¸ois Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. arXiv preprint arXiv:1904.08889, 2019.","title":"Kpconv: Flexible and deformable convolution for point clouds","authors":"Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, Leonidas J Franc ¸ois Goulette, Guibas","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang and Wan, 2018] Ke Wang and Xiaojun Wan. Senti- gan: Generating sentimental texts via mixture adversarial networks. In IJCAI, pages 4446-4452, 2018.","title":"Sentigan: Generating sentimental texts via mixture adversarial networks","authors":"Wang, Wan ;, Ke Wang, Xiaojun Wan","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/618"},{"position":27,"raw":"[Wang et al., 2017] Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, and Xiaoou Tang. Residual attention network for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3156- 3164, 2017.","title":"Residual attention network for image classification","authors":"Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, Xiaoou Tang","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2019] Lei Wang, Yuchun Huang, Yaolin Hou, Shenman Zhang, and Jie Shan. Graph attention convolu- tion for point cloud semantic segmentation. In Proceed- ings of the IEEE Conference on Computer Vision and Pat- tern Recognition, pages 10296-10305, 2019.","title":"Graph attention convolution for point cloud semantic segmentation","authors":"Lei Wang, Yuchun Huang, Yaolin Hou, Shenman Zhang, Jie Shan","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}