{"type":"ScholarlyArticle","title":"UniM-OV3D: Uni-Modality Open-Vocabulary 3D Scene Understanding with Fine-Grained Feature Representation","authors":[{"name":"Qingdong He","slug":"qingdong-he","given":"Qingdong","family":"He","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Jinlong Peng","slug":"jinlong-peng","given":"Jinlong","family":"Peng","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Zhengkai Jiang","slug":"zhengkai-jiang","given":"Zhengkai","family":"Jiang","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Kai Wu","slug":"kai-wu","given":"Kai","family":"Wu","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Xiaozhong Ji","slug":"xiaozhong-ji","given":"Xiaozhong","family":"Ji","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Jiangning Zhang","slug":"jiangning-zhang","given":"Jiangning","family":"Zhang","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Yabiao Wang","slug":"yabiao-wang","given":"Yabiao","family":"Wang","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Chengjie Wang","slug":"chengjie-wang","given":"Chengjie","family":"Wang","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Mingang Chen","slug":"mingang-chen","given":"Mingang","family":"Chen","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"},{"name":"Yunsheng Wu","slug":"yunsheng-wu","given":"Yunsheng","family":"Wu","affiliation":"Shanghai Development Center of Computer Software Technology {yingcaihe , jeromepeng ,"}],"abstract":"3D open-vocabulary scene understanding aims to recognize arbitrary novel categories beyond the base label space. However, existing works not only fail to fully utilize all the available modal information in the 3D domain but also lack sufficient granularity in representing the features of each modality. In this paper, we propose a unified multimodal 3D open-vocabulary scene understanding network, namely UniM-OV3D, aligning point clouds with image, language and depth. To better integrate global and local features of the point clouds, we design a hierarchical point cloud feature extraction module that learns fine-grained feature representations. Further, to facilitate the learning of coarse-to-fine point-semantic representations from captions, we propose the utilization of hierarchical 3D caption pairs, capitalizing on geometric constraints across various viewpoints of 3D scenes. Extensive experimental results have demonstrated the effectiveness and superiority of our method in open-vocabulary semantic and instance segmentation, which achieves state-of-the-art performance on both indoor and outdoor benchmarks such as ScanNet, ScanNet200, S3IDS and nuScenes. Code is available at https://github.com/hithqd/UniM-OV3D.","doi":"10.24963/ijcai.2024/90","url":"https://ijcai.org/proceedings/2024/90","pdf":"https://ijcai.org/proceedings/2024/0090.pdf","markdown":"https://ijcai.org/proceedings/2024/90.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/90","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":90,"pages":{"first":812,"last":820},"keywords":["Computer Vision: CV: 3D computer vision","Computer Vision: CV: Scene analysis and understanding","Computer Vision: CV: Applications"],"videos":[],"references":[{"position":1,"raw":"[Armeni et al., 2016] Iro Armeni, Ozan Sener, Amir R Za- mir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Sil- vio Savarese. 3d semantic parsing of large-scale indoor spaces. In CVPR, 2016.","title":"3d semantic parsing of large-scale indoor spaces","authors":"Iro Armeni, Ozan Sener, Helen Amir R Zamir, Ioannis Jiang, Martin Brilakis, Silvio Fischer, Savarese","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bangalath et al., 2022] Hanoona Bangalath, Muhammad Maaz, Muhammad Uzair Khattak, Salman H Khan, and Fahad Shahbaz Khan. Bridging the gap between object and image-level representations for open-vocabulary de- tection. NeurIPS, 2022.","title":"Bridging the gap between object and image-level representations for open-vocabulary detection","authors":"Bangalath","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Caesar et al., 2020] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In CVPR, 2020.","title":"nuscenes: A multimodal dataset for autonomous driving","authors":"Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, Oscar Beijbom","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Caron et al., 2021] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vi- sion transformers. In ICCV, 2021.","title":"Emerging properties in self-supervised vision transformers","authors":"Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, Armand Joulin","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2023] Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, and Wenping Wang. Clip2scene: Towards label- efficient 3d scene understanding by clip. In CVPR, 2023.","title":"Clip2scene: Towards labelefficient 3d scene understanding by clip","authors":"Chen","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cheraghian et al., 2020] Ali Cheraghian, Shafin Rahman, Dylan Campbell, and Lars Petersson. Transductive zero- shot learning for 3d point cloud classification. In CACV, 2020.","title":"Transductive zeroshot learning for 3d point cloud classification","authors":"Ali Cheraghian, Shafin Rahman, Dylan Campbell, Lars Petersson","year":"2020","venue":"CACV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Choy et al., 2019] Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In CVPR, 2019.","title":"4d spatio-temporal convnets: Minkowski convolutional neural networks","authors":"Christopher Choy, Junyoung Gwak, Silvio Savarese","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dai et al., 2017] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017.","title":"Scannet: Richly-annotated 3d reconstructions of indoor scenes","authors":"Angela Dai, X Angel, Manolis Chang, Maciej Savva, Thomas Halber, Matthias Funkhouser, Nießner","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ding et al., 2023a] Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. Lowis3d: Language-driven open-world instance-level 3d scene un- derstanding. arXiv preprint arXiv:2308.00353, 2023.","title":"Lowis3d: Language-driven open-world instance-level 3d scene understanding","authors":"Ding","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ding et al., 2023b] Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. Pla: Language-driven open-vocabulary 3d scene understand- ing. In CVPR, 2023.","title":"Pla: Language-driven open-vocabulary 3d scene understanding","authors":"Ding","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Graham et al., 2018] Benjamin Graham, Martin Engelcke, and Laurens Van Der Maaten. 3d semantic segmenta- tion with submanifold sparse convolutional networks. In CVPR, 2018.","title":"3d semantic segmentation with submanifold sparse convolutional networks","authors":"Benjamin Graham, Martin Engelcke, Laurens Van Der Maaten","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guo et al., 2023] Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li, Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understanding, generation, and instruction following. arXiv preprint arXiv:2309.00615, 2023.","title":"Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understanding, generation, and instruction following","authors":"Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li, Hongsheng Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, 2016.","title":"Identity mappings in deep residual networks","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang et al., 2022] Tianxin Huang, Jiangning Zhang, Jun Chen, Zhonggan Ding, Ying Tai, Zhenyu Zhang, Chengjie Wang, and Yong Liu. 3qnet: 3d point cloud geometry quantization compression network. ACM TOG, 2022.","title":"3qnet: 3d point cloud geometry quantization compression network","authors":"Tianxin Huang, Jiangning Zhang, Jun Chen, Zhonggan Ding, Ying Tai, Zhenyu Zhang, Chengjie Wang, Yong Liu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Huang et al., 2023a] Tianxin Huang, Zhonggan Ding, Jiangning Zhang, Ying Tai, Zhenyu Zhang, Mingang Chen, Chengjie Wang, and Yong Liu. Learning to mea- sure the point cloud reconstruction loss in a representation space. In CVPR, 2023.","title":"Learning to measure the point cloud reconstruction loss in a representation space","authors":"Huang","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huang et al., 2023b] Tianyu Huang, Bowen Dong, Yunhan Yang, Xiaoshui Huang, Rynson WH Lau, Wanli Ouyang, and Wangmeng Zuo. Clip2point: Transfer clip to point cloud classification with image-depth pre-training. In ICCV, 2023.","title":"Clip2point: Transfer clip to point cloud classification with image-depth pre-training","authors":"Huang","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Huang et al., 2023c] Zhening Huang, Xiaoyang Wu, Xi Chen, Hengshuang Zhao, Lei Zhu, and Joan Lasenby. Openins3d: Snap and lookup for 3d open-vocabulary instance segmentation. arXiv preprint arXiv:2309.00616, 2023.","title":"Openins3d: Snap and lookup for 3d open-vocabulary instance segmentation","authors":"Huang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2022] Boyi Li, Kilian Q Weinberger, Serge Be- longie, Vladlen Koltun, and Rene Ranftl. Language-driven semantic segmentation. In ICLR, 2022.","title":"Language-driven semantic segmentation","authors":"Boyi Li, Q Kilian, Serge Weinberger, Vladlen Belongie, Rene Koltun, Ranftl","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2023a] Kunhao Liu, Fangneng Zhan, Jiahui Zhang, Muyu Xu, Yingchen Yu, Abdulmotaleb El Saddik, Christian Theobalt, Eric Xing, and Shijian Lu. Weakly su- pervised 3d open-vocabulary segmentation. In NeurIPS, 2023.","title":"Weakly supervised 3d open-vocabulary segmentation","authors":"Liu","year":"2023","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2023b] Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open-world understand- ing. arXiv preprint arXiv:2305.10764, 2023.","title":"Openshape: Scaling up 3d shape representation towards open-world understanding","authors":"Liu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mao et al., 2022] Jiageng Mao, Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. 3d object detection for au- tonomous driving: A review and new outlooks. arXiv preprint arXiv:2206.09474, 2022.","title":"3d object detection for autonomous driving: A review and new outlooks","authors":"Mao","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Michele et al., 2021] Björn Michele, Alexandre Boulch, Gilles Puy, Maxime Bucher, and Renaud Marlet. Gen- erative zero-shot learning for semantic segmentation of 3d point clouds. In 3DV, 2021.","title":"Generative zero-shot learning for semantic segmentation of 3d point clouds","authors":"Björn Michele, Alexandre Boulch, Gilles Puy, Maxime Bucher, Renaud Marlet","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Misra et al., 2021] Ishan Misra, Rohit Girdhar, and Armand Joulin. An end-to-end transformer model for 3d object detection. In ICCV, 2021.","title":"An end-to-end transformer model for 3d object detection","authors":"Ishan Misra, Rohit Girdhar, Armand Joulin","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Mokady et al., 2021] Ron Mokady, Amir Hertz, and Amit H Bermano. Clipcap: Clip prefix for image cap- tioning. arXiv preprint arXiv:2111.09734, 2021.","title":"Clipcap: Clip prefix for image captioning","authors":"Ron Mokady, Amir Hertz, Amit H Bermano","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Pang et al., 2022] Yatian Pang, Wenxiao Wang, Francis EH Tay, Wei Liu, Yonghong Tian, and Li Yuan. Masked au- toencoders for point cloud self-supervised learning. In ECCV, 2022.","title":"Masked autoencoders for point cloud self-supervised learning","authors":"Pang","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Park et al., 2008] Youngmin Park, Vincent Lepetit, and Woontack Woo. Multiple 3d object tracking for aug- mented reality. In ISMAR, 2008.","title":"Multiple 3d object tracking for augmented reality","authors":"Park","year":"2008","venue":"ISMAR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Peng et al., 2023] Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. In CVPR, 2023.","title":"Openscene: 3d scene understanding with open vocabularies","authors":"Peng","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Qi et al., 2017a] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 652-660, 2017.","title":"Pointnet: Deep learning on point sets for 3d classification and segmentation","authors":"Qi","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Qi et al., 2017b] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. NeurIPS, 2017.","title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","authors":"Qi","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In ICML, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Rozenberszki et al., 2022] David Rozenberszki, Or Litany, and Angela Dai. Language-grounded indoor 3d semantic segmentation in the wild. In ECCV, 2022.","title":"Language-grounded indoor 3d semantic segmentation in the wild","authors":"David Rozenberszki, Or Litany, Angela Dai","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Shafiullah et al., 2022] Nur Muhammad Mahi Shafiullah, Chris Paxton, Lerrel Pinto, Soumith Chintala, and Arthur Szlam. Clip-fields: Weakly supervised semantic fields for robotic memory. arXiv preprint arXiv:2210.05663, 2022.","title":"Clip-fields: Weakly supervised semantic fields for robotic memory","authors":"Shafiullah","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Takmaz et al., 2023] Ayc ¸a Takmaz, Elisabetta Fedele, Robert W Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. Openmask3d: Open-vocabulary 3d instance segmentation. arXiv preprint arXiv:2306.13631, 2023.","title":"Ayc ¸a Takmaz","authors":"Takmaz","year":"2023","venue":"Openmask3d: Open-vocabulary 3d instance segmentation","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Vu et al., 2022] Thang Vu, Kookhoi Kim, Tung M Luu, Thanh Nguyen, and Chang D Yoo. Softgroup for 3d in- stance segmentation on point clouds. In CVPR, 2022.","title":"Softgroup for 3d instance segmentation on point clouds","authors":"Thang Vu, Kookhoi Kim, M Tung, Thanh Luu, Chang D Nguyen, Yoo","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2022] Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jin- gren Zhou, and Hongxia Yang. Ofa: Unifying architec- tures, tasks, and modalities through a simple sequence-to- sequence learning framework. In ICML, 2022.","title":"Ofa: Unifying architectures, tasks, and modalities through a simple sequence-tosequence learning framework","authors":"Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, Hongxia Yang","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wu et al., 2023a] Jianzong Wu, Xiangtai Li, Haobo Yuan, Shilin Xu, Henghui Ding, Yibo Yang, Xia Li, Jiangning Zhang, Yunhai Tong, Xudong Jiang, Bernard Ghanem, et al. Towards open vocabulary learning: A survey. arXiv preprint arXiv:2306.15880, 2023.","title":"Towards open vocabulary learning: A survey","authors":"Wu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wu et al., 2023b] Weijia Wu, Yuzhong Zhao, Mike Zheng Shou, Hong Zhou, and Chunhua Shen. Diffumask: Syn- thesizing images with pixel-level annotations for seman- tic segmentation using diffusion models. arXiv preprint arXiv:2303.11681, 2023.","title":"Diffumask: Synthesizing images with pixel-level annotations for semantic segmentation using diffusion models","authors":"Wu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Xu et al., 2023] Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds. arXiv preprint arXiv:2308.16911, 2023.","title":"Pointllm: Empowering large language models to understand point clouds","authors":"Xu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Xue et al., 2023a] Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learn- ing a unified representation of language, images, and point clouds for 3d understanding. In CVPR, 2023.","title":"Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding","authors":"Xue","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Xue et al., 2023b] Le Xue, Ning Yu, Shu Zhang, Junnan Li, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip-2: To- wards scalable multimodal pre-training for 3d understand- ing. arXiv preprint arXiv:2305.08275, 2023.","title":"Ulip-2: Towards scalable multimodal pre-training for 3d understanding","authors":"Xue","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yang et al., 2023] Jihan Yang, Runyu Ding, Zhe Wang, and Xiaojuan Qi. Regionplc: Regional point-language con- trastive learning for open-world 3d scene understanding. arXiv preprint arXiv:2304.00962, 2023.","title":"Regionplc: Regional point-language contrastive learning for open-world 3d scene understanding","authors":"Jihan Yang, Runyu Ding, Zhe Wang, Xiaojuan Qi","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yu et al., 2022] Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre- training 3d point cloud transformers with masked point modeling. In CVPR, 2022.","title":"Point-bert: Pretraining 3d point cloud transformers with masked point modeling","authors":"Yu","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zeng et al., 2023] Yihan Zeng, Chenhan Jiang, Jiageng Mao, Jianhua Han, Chaoqiang Ye, Qingqiu Huang, Dit- Yan Yeung, Zhen Yang, Xiaodan Liang, and Hang Xu. Clip2: Contrastive language-image-point pretraining from real-world point cloud data. In CVPR, 2023.","title":"Clip2: Contrastive language-image-point pretraining from real-world point cloud data","authors":"Yihan Zeng, Chenhan Jiang, Jiageng Mao, Jianhua Han, Chaoqiang Ye, Qingqiu Huang, Dit-Yan Yeung, Zhen Yang, Xiaodan Liang, Hang Xu","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhang et al., 2022] Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. Pointclip: Point cloud understanding by clip. In CVPR, 2022.","title":"Pointclip: Point cloud understanding by clip","authors":"Zhang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhang et al., 2023] Junbo Zhang, Runpei Dong, and Kaisheng Ma. Clip-fo3d: Learning free open-world 3d scene representations from 2d dense clip. arXiv preprint arXiv:2303.04748, 2023.","title":"Clip-fo3d: Learning free open-world 3d scene representations from 2d dense clip","authors":"Junbo Zhang, Runpei Dong, Kaisheng Ma","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhu et al., 2023] Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyu Guo, Ziyao Zeng, Zipeng Qin, Shanghang Zhang, and Peng Gao. Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning. In ICCV, 2023.","title":"Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning","authors":"Zhu","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}