{"type":"ScholarlyArticle","title":"Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-training","authors":[{"name":"Ziyu Guo","slug":"ziyu-guo","given":"Ziyu","family":"Guo","affiliation":"Department of Computer Science and Engineering , The Chinese University of Hong Kong"},{"name":"Renrui Zhang","slug":"renrui-zhang","given":"Renrui","family":"Zhang","affiliation":"CUHK MMLab"},{"name":"Longtian Qiu","slug":"longtian-qiu","given":"Longtian","family":"Qiu","affiliation":"ShanghaiTech University"},{"name":"Xianzhi Li","slug":"xianzhi-li","given":"Xianzhi","family":"Li","affiliation":"Huazhong University of Science and Technology"},{"name":"Pheng-Ann Heng","slug":"pheng-ann-heng","given":"Pheng-Ann","family":"Heng","affiliation":"Department of Computer Science and Engineering , The Chinese University of Hong Kong; Institute of Medical Intelligence and XR , The Chinese University of Hong Kong"}],"abstract":"Masked Autoencoders (MAE) have shown promising performance in self-supervised learning for both 2D and 3D computer vision. However, existing MAE-style methods can only learn from the data of a single modality, i.e., either images or point clouds, which neglect the implicit semantic and geometric correlation between 2D and 3D. In this paper, we explore how the 2D modality can benefit 3D masked autoencoding, and propose Joint-MAE, a 2D-3D joint MAE framework for self-supervised 3D point cloud pre-training. Joint-MAE randomly masks an input 3D point cloud and its projected 2D images, and then reconstructs the masked information of the two modalities. For better cross-modal interaction, we construct our JointMAE by two hierarchical 2D-3D embedding modules, a joint encoder, and a joint decoder with modal-shared and model-specific decoders. On top of this, we further introduce two cross-modal strategies to boost the 3D representation learning, which are local-aligned attention mechanisms for 2D-3D semantic cues, and a cross-reconstruction loss for 2D-3D geometric constraints. By our pre-training paradigm, Joint-MAE achieves superior performance on multiple downstream tasks, e.g., 92.4% accuracy for linear SVM on ModelNet40 and 86.07% accuracy on the hardest split of ScanObjectNN.","doi":"10.24963/ijcai.2023/88","url":"https://ijcai.org/proceedings/2023/88","pdf":"https://ijcai.org/proceedings/2023/0088.pdf","markdown":"https://ijcai.org/proceedings/2023/88.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/88","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":88,"pages":{"first":791,"last":799},"keywords":["Computer Vision: CV: Representation learning","Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning","Computer Vision: CV: 3D computer vision"],"videos":[],"references":[{"position":1,"raw":"[Afham et al., 2022] Mohamed Afham, Isuru Dissanayake, Dinithi Dissanayake, Amaya Dharmasiri, Kanchana Thi- lakarathna, and Ranga Rodrigo. Crosspoint: Self- supervised cross-modal contrastive learning for 3d point cloud understanding. arXiv preprint arXiv:2203.00680, 2022.","title":"Crosspoint: Selfsupervised cross-modal contrastive learning for 3d point cloud understanding","authors":"Mohamed Afham, Isuru Dissanayake, Dinithi Dissanayake, Amaya Dharmasiri, Kanchana Thilakarathna, Ranga Rodrigo","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang et al., 2015] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015.","title":"Shapenet: An information-rich 3d model repository","authors":"Thomas Angel X Chang, Leonidas Funkhouser, Pat Guibas, Qixing Hanrahan, Zimo Huang, Silvio Li, Manolis Savarese, Shuran Savva, Hao Song, Su","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Desai and Johnson, 2021] Karan Desai and Justin Johnson. Virtex: Learning visual representations from textual an- notations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11162- 11173, 2021.","title":"Virtex: Learning visual representations from textual annotations","authors":"Karan Desai, Justin Johnson","year":"2021","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fan et al., 2017] Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object re- construction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 605-613, 2017.","title":"A point set generation network for 3d object reconstruction from a single image","authors":"Haoqiang Fan, Hao Su, Leonidas J Guibas","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gao et al., 2021] Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. arXiv preprint arXiv:2110.04544, 2021.","title":"Clip-adapter: Better vision-language models with feature adapters","authors":"Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, Yu Qiao","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gao et al., 2022] Peng Gao, Teli Ma, Hongsheng Li, Jifeng Dai, and Yu Qiao. Convmae: Masked convolution meets masked autoencoders. arXiv preprint arXiv:2205.03892, 2022.","title":"Convmae: Masked convolution meets masked autoencoders","authors":"Peng Gao, Teli Ma, Hongsheng Li, Jifeng Dai, Yu Qiao","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao et al., 2023] Peng Gao, Renrui Zhang, Rongyao Fang, Ziyi Lin, Hongyang Li, Hongsheng Li, and Qiao Yu. Mimic before reconstruct: Enhancing masked autoencoders with feature mimicking. arXiv preprint arXiv:2303.05475, 2023.","title":"Mimic before reconstruct: Enhancing masked autoencoders with feature mimicking","authors":"Peng Gao, Renrui Zhang, Rongyao Fang, Ziyi Lin, Hongyang Li, Hongsheng Li, Qiao Yu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goyal et al., 2021] Ankit Goyal, Hei Law, Bowei Liu, Ale- jandro Newell, and Jia Deng. Revisiting point cloud shape classification with a simple and effective baseline. arXiv preprint arXiv:2106.05304, 2021.","title":"Revisiting point cloud shape classification with a simple and effective baseline","authors":"Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, Jia Deng","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guo et al., 2021] Meng-Hao Guo, Jun-Xiong Cai, Zheng- Ning Liu, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pct: Point cloud transformer. Computational Visual Me- dia, 7(2):187-199, 2021.","title":"Pct: Point cloud transformer","authors":"Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Shi-Min Ralph R Martin, Hu","year":"2021","venue":"Computational Visual Media","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2022] Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzheng Ma, Xupeng Miao, Xuming He, and Bin Cui. Calip: Zero-shot enhancement of clip with parameter-free attention. arXiv preprint arXiv:2209.14169, 2022.","title":"Calip: Zero-shot enhancement of clip with parameter-free attention","authors":"Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzheng Ma, Xupeng Miao, Xuming He, Bin Cui","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Han et al., 2019] Zhizhong Han, Mingyang Shang, Yu- Shen Liu, and Matthias Zwicker. View inter-prediction gan: Unsupervised representation learning for 3d shapes by learning global shape memories to support local view predictions. In Proceedings of the AAAI Conference on Ar- tificial Intelligence, volume 33, pages 8376-8384, 2019.","title":"View inter-prediction gan: Unsupervised representation learning for 3d shapes by learning global shape memories to support local view predictions","authors":"Zhizhong Han, Mingyang Shang, Yu-Shen Liu, Matthias Zwicker","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2021] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. arXiv preprint arXiv:2111.06377, 2021.","title":"Masked autoencoders are scalable vision learners","authors":"He","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2018] Jiaxin Li, Ben M Chen, and Gim Hee Lee. So-net: Self-organizing network for point cloud analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9397-9406, 2018.","title":"So-net: Self-organizing network for point cloud analysis","authors":"Jiaxin Li, Ben M Chen, Gim Hee, Lee","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2022] Haotian Liu, Mu Cai, and Yong Jae Lee. Masked discrimination for self-supervised learning on point clouds. arXiv preprint arXiv:2203.11183, 2022.","title":"Masked discrimination for self-supervised learning on point clouds","authors":"Liu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nichol et al., 2022] Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts. arXiv preprint arXiv:2212.08751, 2022.","title":"Point-e: A system for generating 3d point clouds from complex prompts","authors":"Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, Mark Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pang et al., 2022] Yatian Pang, Wenxiao Wang, Francis EH Tay, Wei Liu, Yonghong Tian, and Li Yuan. Masked au- toencoders for point cloud self-supervised learning. arXiv preprint arXiv:2203.06604, 2022.","title":"Masked autoencoders for point cloud self-supervised learning","authors":"Pang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Poursaeed et al., 2020] Omid Poursaeed, Tianxing Jiang, Han Qiao, Nayun Xu, and Vladimir G Kim. Self- supervised learning of point clouds via orientation esti- mation. In 2020 International Conference on 3D Vision (3DV), pages 1018-1028. IEEE, 2020.","title":"Selfsupervised learning of point clouds via orientation estimation","authors":"Omid Poursaeed, Tianxing Jiang, Han Qiao, Nayun Xu, Vladimir G Kim","year":"2020","venue":"2020 International Conference on 3D Vision (3DV)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qi et al., 2017a] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.","title":"Pointnet: Deep learning on point sets for 3d classification and segmentation","authors":"Qi","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Qi et al., 2017b] Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413, 2017.","title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","authors":"Qi","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In International Conference on Machine Learning, pages 8748-8763. PMLR, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rao et al., 2020] Yongming Rao, Jiwen Lu, and Jie Zhou. Global-local bidirectional reasoning for unsupervised rep- resentation learning of 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 5376-5385, 2020.","title":"Global-local bidirectional reasoning for unsupervised representation learning of 3d point clouds","authors":"Yongming Rao, Jiwen Lu, Jie Zhou","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sauder and Sievers, 2019] Jonathan Sauder and Bjarne Sievers. Self-supervised deep learning on point clouds by reconstructing space. Advances in Neural Information Processing Systems, 32, 2019.","title":"Self-supervised deep learning on point clouds by reconstructing space","authors":"Sievers Sauder, Jonathan Sauder, Bjarne Sievers","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Uy et al., 2019] Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung. Revis- iting point cloud classification: A new benchmark dataset and classification model on real-world data. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 1588-1597, 2019.","title":"Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data","authors":"Angelina Mikaela, Quang-Hieu Uy, Binh-Son Pham, Thanh Hua, Sai-Kit Nguyen, Yeung","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2019] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog), 38(5):1-12, 2019.","title":"Dynamic graph cnn for learning on point clouds","authors":"Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, Justin M Solomon","year":"2019","venue":"Acm Transactions On Graphics (tog)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2021] Hanchen Wang, Qi Liu, Xiangyu Yue, Joan Lasenby, and Matt J Kusner. Unsupervised point cloud pre-training via occlusion completion. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 9782-9792, 2021.","title":"Unsupervised point cloud pre-training via occlusion completion","authors":"Wang","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wu et al., 2015a] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912- 1920, 2015.","title":"3d shapenets: A deep representation for volumetric shapes","authors":"Wu","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wu et al., 2015b] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912- 1920, 2015.","title":"3d shapenets: A deep representation for volumetric shapes","authors":"Wu","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xie et al., 2016] Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. A theory of generative convnet. In International Conference on Machine Learning, pages 2635-2644. PMLR, 2016.","title":"A theory of generative convnet","authors":"Jianwen Xie, Yang Lu, Song-Chun Zhu, Yingnian Wu","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xie et al., 2018] Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, and Ying Nian Wu. Learning descriptor networks for 3d shape synthesis and analysis. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pages 8629-8638, 2018.","title":"Learning descriptor networks for 3d shape synthesis and analysis","authors":"Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, Ying Nian Wu","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xie et al., 2020] Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, and Ying Nian Wu. Generative voxelnet: learning energy-based models for 3d shape synthesis and analysis. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 44(5):2468-2484, 2020.","title":"Generative voxelnet: learning energy-based models for 3d shape synthesis and analysis","authors":"Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, Ying Nian Wu","year":"2020","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xie et al., 2021] Jianwen Xie, Yifei Xu, Zilong Zheng, Song-Chun Zhu, and Ying Nian Wu. Generative pointnet: Deep energy-based learning on unordered point sets for 3d generation, reconstruction and classification. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14976-14985, 2021.","title":"Generative pointnet: Deep energy-based learning on unordered point sets for 3d generation, reconstruction and classification","authors":"Jianwen Xie, Yifei Xu, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xu et al., 2021] Chenfeng Xu, Shijia Yang, Bohan Zhai, Bichen Wu, Xiangyu Yue, Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka. Image2point: 3d point-cloud understanding with pretrained 2d convnets. arXiv preprint arXiv:2106.04180, 2021.","title":"Image2point: 3d point-cloud understanding with pretrained 2d convnets","authors":"Chenfeng Xu, Shijia Yang, Bohan Zhai, Bichen Wu, Xiangyu Yue, Wei Zhan, Peter Vajda, Kurt Keutzer, Masayoshi Tomizuka","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2018] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 206-215, 2018.","title":"Foldingnet: Point cloud auto-encoder via deep grid deformation","authors":"Yang","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yi et al., 2016] Li Yi, Vladimir G Kim, Duygu Ceylan, I- Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collec- tions. ACM Transactions on Graphics (ToG), 35(6):1-12, 2016.","title":"A scalable active framework for region annotation in 3d shape collections","authors":"Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, Leonidas Guibas","year":"2016","venue":"ACM Transactions on Graphics (ToG)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yu et al., 2021] Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre- training 3d point cloud transformers with masked point modeling. arXiv preprint arXiv:2111.14819, 2021.","title":"Point-bert: Pretraining 3d point cloud transformers with masked point modeling","authors":"Yu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2021a] Renrui Zhang, Rongyao Fang, Wei Zhang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free clip-adapter for better vision-language modeling. arXiv preprint arXiv:2111.03930, 2021.","title":"Tip-adapter: Training-free clip-adapter for better vision-language modeling","authors":"Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhang et al., 2021b] Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. Pointclip: Point cloud understanding by clip. arXiv preprint arXiv:2112.02413, 2021.","title":"Pointclip: Point cloud understanding by clip","authors":"Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhang et al., 2021c] Zaiwei Zhang, Rohit Girdhar, Armand Joulin, and Ishan Misra. Self-supervised pretraining of 3d features on any point-cloud. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10252-10263, 2021.","title":"Self-supervised pretraining of 3d features on any point-cloud","authors":"Zhang","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang et al., 2022a] Renrui Zhang, Ziyu Guo, Peng Gao, Rongyao Fang, Bin Zhao, Dong Wang, Yu Qiao, and Hongsheng Li. Point-m2ae: Multi-scale masked autoen- coders for hierarchical point cloud pre-training. arXiv preprint arXiv:2205.14401, 2022.","title":"Point-m2ae: Multi-scale masked autoencoders for hierarchical point cloud pre-training","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhang et al., 2022b] Renrui Zhang, Liuhui Wang, Yu Qiao, Peng Gao, and Hongsheng Li. Learning 3d representations from 2d pre-trained models via image-to-point masked au- toencoders. arXiv preprint arXiv:2212.06785, 2022.","title":"Learning 3d representations from 2d pre-trained models via image-to-point masked autoencoders","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2022c] Renrui Zhang, Ziyao Zeng, Ziyu Guo, and Yafeng Li. Can language understand depth? In Pro- ceedings of the 30th ACM International Conference on Multimedia, pages 6868-6874, 2022.","title":"Can language understand depth?","authors":"Zhang","year":"2022","venue":"Proceedings of the 30th ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhang et al., 2023] Renrui Zhang, Liuhui Wang, Ziyu Guo, and Jianbo Shi. Nearest neighbors meet deep neural net- works for point cloud analysis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 1246-1255, 2023.","title":"Nearest neighbors meet deep neural networks for point cloud analysis","authors":"Zhang","year":"2023","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhao et al., 2021] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16259-16268, 2021.","title":"Point transformer","authors":"Hengshuang Zhao, Li Jiang, Jiaya Jia, Vladlen Philip Hs Torr, Koltun","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhu et al., 2022] Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyao Zeng, Shanghang Zhang, and Peng Gao. Pointclip v2: Adapting clip for powerful 3d open-world learning. arXiv preprint arXiv:2211.11682, 2022.","title":"Pointclip v2: Adapting clip for powerful 3d open-world learning","authors":"Zhu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}