{"type":"ScholarlyArticle","title":"Rethinking Loss Design for Large-scale 3D Shape Retrieval","authors":[{"name":"Zhaoqun Li","slug":"zhaoqun-li","given":"Zhaoqun","family":"Li","affiliation":"School of Computer Science and Engineering , Beihang University , Beijing , 100191"},{"name":"Cheng Xu","slug":"cheng-xu","given":"Cheng","family":"Xu","affiliation":"School of Computer Science and Engineering , Beihang University , Beijing , 100191"},{"name":"Biao Leng","slug":"biao-leng","given":"Biao","family":"Leng","affiliation":"School of Computer Science and Engineering , Beihang University , Beijing , 100191; Beijing Advanced Innovation Center for Big Data and Brain Computing , Beihang University , Beijing , 100191; State Key Laboratory of Virtual Reality Technology and Systems , Beihang University , Beijing , 100191"}],"abstract":"Learning discriminative shape representations is a crucial issue for large-scale 3D shape retrieval. In this paper, we propose the Collaborative Inner Product Loss (CIP Loss) to obtain ideal shape embedding that discriminative among different categories and clustered within the same class. Utilizing simple inner product operation, CIP loss explicitly enforces the features of the same class to be clustered in a linear subspace, while inter-class subspaces are constrained to be at least orthogonal. Compared to previous metric loss functions, CIP loss could provide more clear geometric interpretation for the embedding than Euclidean margin, and is easy to implement without normalization operation referring to cosine margin. Moreover, our proposed loss term can combine with other commonly used loss functions and can be easily plugged into existing off-the-shelf architectures. Extensive experiments conducted on the two public 3D object retrieval datasets, ModelNet and ShapeNetCore 55, \r\ndemonstrate the effectiveness of our proposal, and our method has achieved state-of-the-art results on both datasets.","doi":"10.24963/ijcai.2019/118","url":"https://ijcai.org/proceedings/2019/118","pdf":"https://ijcai.org/proceedings/2019/0118.pdf","markdown":"https://ijcai.org/proceedings/2019/118.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/118","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":118,"pages":{"first":840,"last":846},"keywords":["Computer Vision: Computer Vision","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: 2D and 3D Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Bai et al., 2016] Song Bai, Xiang Bai, Zhichao Zhou, Zhaoxiang Zhang, and Longin Jan Latecki. Gift: A real- time and scalable 3d shape search engine. In CVPR, pages 5023-5032, 2016.","title":"Gift: A realtime and scalable 3d shape search engine","authors":"Song Bai, Xiang Bai, Zhichao Zhou, Zhaoxiang Zhang, Longin, Jan Latecki","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang et al., 2015] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015.","title":"Shapenet: An information-rich 3d model repository","authors":"Thomas Angel X Chang, Leonidas Funkhouser, Pat Guibas, Qixing Hanrahan, Zimo Huang, Silvio Li, Manolis Savarese, Shuran Savva, Hao Song, Su","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chatfield et al., 2014] Ken Chatfield, Karen Simonyan, An- drea Vedaldi, and Andrew Zisserman. Return of the devil in the details: Delving deep into convolutional nets. 2014.","title":"Return of the devil in the details: Delving deep into convolutional nets","authors":"Ken Chatfield, Karen Simonyan, Andrea Vedaldi, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dai et al., 2018] Guoxian Dai, Jin Xie, and Yi Fang. Siamese cnn-bilstm architecture for 3d shape representa- tion learning. In IJCAI, pages 670-676, 2018.","title":"Siamese cnn-bilstm architecture for 3d shape representation learning","authors":"Dai","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/93"},{"position":5,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feng et al., 2018] Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, and Yue Gao. Gvcnn: Group-view convolu- tional neural networks for 3d shape recognition. In CVPR, pages 264-272, 2018.","title":"Gvcnn: Group-view convolutional neural networks for 3d shape recognition","authors":"Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, Yue Gao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2018] Xinwei He, Yang Zhou, Zhichao Zhou, Song Bai, and Xiang Bai. Triplet-center loss for multi- view 3d object retrieval. In CVPR, 2018.","title":"Triplet-center loss for multiview 3d object retrieval","authors":"Xinwei He, Yang Zhou, Zhichao Zhou, Song Bai, Xiang Bai","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2018] Haibin Huang, Evangelos Kalogerakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir G Kim, and Ersin Yumer. Learning local shape descriptors from part correspondences with multiview convolutional net- works. ACM Transactions on Graphics (TOG), 2018.","title":"Learning local shape descriptors from part correspondences with multiview convolutional networks","authors":"Haibin Huang, Evangelos Kalogerakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir G Kim, Ersin Yumer","year":"2018","venue":"ACM Transactions on Graphics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. CoRR, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Sergey Ioffe, Christian Szegedy","year":"2015","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Johns et al., 2016] Edward Johns, Stefan Leutenegger, and Andrew J Davison. Pairwise decomposition of image se- quences for active multi-view recognition. In CVPR, pages 3813-3822, 2016.","title":"Pairwise decomposition of image sequences for active multi-view recognition","authors":"Edward Johns, Stefan Leutenegger, Andrew J Davison","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Klokov and Lempitsky, 2017] Roman Klokov and Victor Lempitsky. Escape from cells: Deep kd-networks for the recognition of 3d point cloud models. In ICCV, pages 863- 872, 2017.","title":"Roman Klokov and Victor Lempitsky. Escape from cells: Deep kd-networks for the recognition of 3d point cloud models","authors":"Lempitsky Klokov","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2019] Zhaoqun Li, Cheng Xu, and Biao Leng. An- gular triplet-center loss for multi-view 3d shape retrieval. In AAAI, 2019.","title":"Angular triplet-center loss for multi-view 3d shape retrieval","authors":"Zhaoqun Li, Cheng Xu, Biao Leng","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Yu Liu, Hongyang Li, and Xiaogang Wang. Rethinking feature discrimination and poly- merization for large-scale recognition. arXiv preprint arXiv:1710.00870, 2017.","title":"Rethinking feature discrimination and polymerization for large-scale recognition","authors":"Yu Liu, Hongyang Li, Xiaogang Wang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Qi et al., 2017] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NeurIPS, pages 5099-5108, 2017.","title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","authors":"Charles Ruizhongtai, Qi, Li Yi, Hao Su, Leonidas J Guibas","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Savva et al., 2016] Manolis Savva, Fisher Yu, Hao Su, M Aono, B Chen, D Cohen-Or, W Deng, Hang Su, Song Bai, Xiang Bai, et al. Shrec'16 track large-scale 3d shape retrieval from shapenet core55. In Proceedings of the eu- rographics workshop on 3D object retrieval, 2016.","title":"Shrec'16 track large-scale 3d shape retrieval from shapenet core55","authors":"Manolis Savva, Fisher Yu, Hao Su, M Aono, B Chen, D Cohen-Or, W Deng, Hang Su, Song Bai, Xiang Bai","year":"2016","venue":"Proceedings of the eurographics workshop on 3D object retrieval","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. CVPR, pages 815-823, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sfikas et al., 2017] Konstantinos Sfikas, Theoharis Theo- haris, and Ioannis Pratikakis. Exploiting the PANORAMA Representation for Convolutional Neural Network Clas- sification and Retrieval. In Ioannis Pratikakis, Florent Dupont, and Maks Ovsjanikov, editors, Eurographics Workshop on 3D Object Retrieval, pages 1-7. The Euro- graphics Association, 2017.","title":"Exploiting the PANORAMA Representation for Convolutional Neural Network Classification and Retrieval","authors":"Konstantinos Sfikas, Theoharis Theoharis, Ioannis Pratikakis","year":"2017","venue":"Eurographics Workshop on 3D Object Retrieval","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shen et al., 2018] Yiru Shen, Chen Feng, Yaoqing Yang, and Dong Tian. Mining point cloud local structures by kernel correlation and graph pooling. In CVPR, 2018.","title":"Mining point cloud local structures by kernel correlation and graph pooling","authors":"Yiru Shen, Chen Feng, Yaoqing Yang, Dong Tian","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shi et al., 2015] Baoguang Shi, Song Bai, Zhichao Zhou, and Xiang Bai. Deeppano: Deep panoramic representa- tion for 3-d shape recognition. IEEE Signal Processing Letters, 22(12):2339-2343, 2015.","title":"Deeppano: Deep panoramic representation for 3-d shape recognition","authors":"Shi","year":"2015","venue":"IEEE Signal Processing Letters","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Song et al., 2016] Hyun Oh Song, Xiang Yu, Stefanie Jegelka, and Silvio Savarese. Deep metric learning via lifted structured feature embedding. In CVPR, pages 4004-4012, 2016.","title":"Deep metric learning via lifted structured feature embedding","authors":"Hyun Oh Song, Xiang Yu, Stefanie Jegelka, Silvio Savarese","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2017] Bai Song, Zhichao Zhou, Jingdong Wang, Bai Xiang, and Tian Qi. Ensemble diffusion for retrieval. In ICCV, pages 774-783, 2017.","title":"Ensemble diffusion for retrieval","authors":"Bai Song, Zhichao Zhou, Jingdong Wang, Bai Xiang, Tian Qi","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Su et al., 2015] Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multi-view con- volutional neural networks for 3d shape recognition. In ICCV, pages 945-953, 2015.","title":"Multi-view convolutional neural networks for 3d shape recognition","authors":"Su","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[van der Maaten and Hinton, 2008] L.J.P. van der Maaten and G.E. Hinton. Visualizing high-dimensional data using t-sne. In Journal of Machine Learning Research 9(Nov), 2008.","title":"Visualizing high-dimensional data using t-sne","authors":"Van Der Maaten, ; ] L J P Hinton, G E Van Der Maaten, Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wen et al., 2016] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning ap- proach for deep face recognition. In ECCV, pages 499- 515, 2016.","title":"A discriminative feature learning approach for deep face recognition","authors":"Yandong Wen, Kaipeng Zhang, Zhifeng Li, Yu Qiao","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2015] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pages 1912-1920, 2015.","title":"3d shapenets: A deep representation for volumetric shapes","authors":"Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, Jianxiong Xiao","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2017] Jin Xie, Guoxian Dai, Fan Zhu, Edward K Wong, and Yi Fang. Deepshape: Deep-learned shape de- scriptor for 3d shape retrieval. IEEE transactions on pat- tern analysis and machine intelligence, pages 1335-1345, 2017.","title":"Deepshape: Deep-learned shape descriptor for 3d shape retrieval","authors":"Jin Xie, Guoxian Dai, Fan Zhu, Edward K Wong, Yi Fang","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2018] Cheng Xu, Biao Leng, Cheng Zhang, and Xiaochen Zhou. Emphasizing 3d properties in recurrent multi-view aggregation for 3d shape retrieval. In AAAI, pages 7428-7435, 2018.","title":"Emphasizing 3d properties in recurrent multi-view aggregation for 3d shape retrieval","authors":"Cheng Xu, Biao Leng, Cheng Zhang, Xiaochen Zhou","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}