{"type":"ScholarlyArticle","title":"Consistent Domain Structure Learning and Domain Alignment for 2D Image-Based 3D Objects Retrieval","authors":[{"name":"Yuting Su","slug":"yuting-su","given":"Yuting","family":"Su","affiliation":"School of Electrical and Information Engineering , Tianjin University , China"},{"name":"Yuqian Li","slug":"yuqian-li","given":"Yuqian","family":"Li","affiliation":"School of Electrical and Information Engineering , Tianjin University , China"},{"name":"Dan Song","slug":"dan-song","given":"Dan","family":"Song","affiliation":"School of Electrical and Information Engineering , Tianjin University , China"},{"name":"Weizhi Nie","slug":"weizhi-nie","given":"Weizhi","family":"Nie","affiliation":"School of Electrical and Information Engineering , Tianjin University , China"},{"name":"Wenhui Li","slug":"wenhui-li","given":"Wenhui","family":"Li","affiliation":"School of Electrical and Information Engineering , Tianjin University , China"},{"name":"An-An Liu","slug":"an-an-liu","given":"An-An","family":"Liu","affiliation":"School of Electrical and Information Engineering , Tianjin University , China"}],"abstract":"2D image-based 3D objects retrieval is a new topic for 3D objects retrieval which can be used to manage 3D data with 2D images. The goal is to search some related 3D objects when given a 2D image. The task is challenging due to the large domain gap between 2D images and 3D objects. Therefore, it is essential to consider domain adaptation problems to reduce domain discrepancy. However, most of the existing domain adaptation methods only utilize the semantic information from the source domain to predict labels in the target domain and neglect the intrinsic structure of the target domain. In this paper, we propose a domain alignment framework with consistent domain structure learning to reduce the large gap between 2D images and 3D objects. The domain structure learning module makes use of both the semantic information from the source domain and the intrinsic structure of the target domain, which provides more reliable predicted labels to the domain alignment module to better align the conditional distribution. We conducted experiments on two public datasets, MI3DOR and MI3DOR-2, and the experimental results demonstrate the proposed method outperforms the state-of-the-art methods.","doi":"10.24963/ijcai.2020/123","url":"https://ijcai.org/proceedings/2020/123","pdf":"https://ijcai.org/proceedings/2020/0123.pdf","markdown":"https://ijcai.org/proceedings/2020/123.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/123","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":123,"pages":{"first":883,"last":889},"keywords":["Computer Vision: 2D and 3D Computer Vision","Machine Learning: Transfer, Adaptation, Multi-task Learning","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24629/file_24629.m3u8","length_sec":291,"provider":"hls"}],"references":[{"position":1,"raw":"[Chang and et al., 2015] Angel X. Chang and et al. Shapenet: An information-rich 3d model repository. CoRR, abs/1512.03012, 2015.","title":"Shapenet: An information-rich 3d model repository","authors":"Chang","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cheng et al., 2018] Zhiyong Cheng, Ying Ding, and et al. Aˆ3ncf: An adaptive aspect attention model for rating pre- diction. In IJCAI, pages 3748-3754, 2018.","title":"Aˆ3ncf: An adaptive aspect attention model for rating prediction","authors":"Zhiyong Cheng, Ying Ding","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/521"},{"position":3,"raw":"[Dai et al., 2017] Guoxian Dai, Jin Xie, Fan Zhu, and Yi Fang. Deep correlated metric learning for sketch-based 3d shape retrieval. In AAAI, pages 4002-4008, 2017.","title":"Deep correlated metric learning for sketch-based 3d shape retrieval","authors":"Dai","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Feng et al., 2018] Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, and Yue Gao. GVCNN: group-view con- volutional neural networks for 3d shape recognition. In CVPR, pages 264-272, 2018.","title":"GVCNN: group-view convolutional neural networks for 3d shape recognition","authors":"Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, Yue Gao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ghadai et al., 2019] Sambit Ghadai, Xian Yeow Lee, Aditya Balu, Soumik Sarkar, and Adarsh Krishnamurthy. Multi-level 3d CNN for learning multi-scale spatial fea- tures. In CVPR Workshops 2019, page 0, 2019.","title":"Multi-level 3d CNN for learning multi-scale spatial features","authors":"Sambit Ghadai, Yeow Xian, Aditya Lee, Soumik Balu, Adarsh Sarkar, Krishnamurthy","year":"2019","venue":"CVPR Workshops 2019","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gong et al., 2012] Boqing Gong, Yuan Shi, and Fei Sha et al. Geodesic flow kernel for unsupervised domain adap- tation. In CVPR, pages 2066-2073, 2012.","title":"Geodesic flow kernel for unsupervised domain adaptation","authors":"Gong","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gopalan et al., 2011] Raghuraman Gopalan, Ruonan Li, and et al. Domain adaptation for object recognition: An unsupervised approach. In ICCV, pages 999-1006, 2011.","title":"Domain adaptation for object recognition: An unsupervised approach","authors":"Raghuraman Gopalan, Ruonan Li","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gretton et al., 2012] Arthur Gretton, Karsten M. Borg- wardt, and et al. A kernel two-sample test. J. Mach. Learn. Res., 13:723-773, 2012.","title":"A kernel two-sample test","authors":"Arthur Gretton, Karsten M Borgwardt","year":"2012","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He and Niyogi, 2003] Xiaofei He and Partha Niyogi. Local- ity preserving projections. In NIPS, pages 153-160, 2003.","title":"Locality preserving projections","authors":"Xiaofei He, Partha Niyogi","year":"2003","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2018] Xinwei He, Yang Zhou, and et al. Triplet- center loss for multi-view 3d object retrieval. In CVPR, pages 1945-1954, 2018.","title":"Tripletcenter loss for multi-view 3d object retrieval","authors":"He","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hong et al., 2016] Richang Hong, Zhenzhen Hu, and et al. Multi-view object retrieval via multi-scale topic models. IEEE Trans. Image Processing, 25(12):5814-5827, 2016.","title":"Multi-view object retrieval via multi-scale topic models","authors":"Hong","year":"2016","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and et al. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1106-1114, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lee et al., 2018] Tang Lee, Yen-Liang Lin, and et al. Cross- domain image-based 3d shape retrieval by view sequence learning. In 2018 International Conference on 3D Vision, pages 258-266, 2018.","title":"Crossdomain image-based 3d shape retrieval by view sequence learning","authors":"Tang Lee, Yen-Liang Lin","year":"2018","venue":"International Conference on 3D Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2019a] Wenhui Li, Anan Liu, and Weizhi Nieet al. Monocular image based 3d model retrieval. In 3DOR, pages 103-110, 2019.","title":"Monocular image based 3d model retrieval","authors":"Li","year":"2019","venue":"3DOR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2019b] Zhaoqun Li, Cheng Xu, and Biao Leng. Angular triplet-center loss for multi-view 3d shape re- trieval. In AAAI, pages 8682-8689, 2019.","title":"Angular triplet-center loss for multi-view 3d shape retrieval","authors":"Li","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2016] Anan Liu, Weizhi Nie, Yue Gao, and Yut- ing Su. Multi-modal clique-graph matching for view- based 3d model retrieval. IEEE Trans. Image Processing, 25(5):2103-2116, 2016.","title":"Multi-modal clique-graph matching for viewbased 3d model retrieval","authors":"Anan Liu, Weizhi Nie, Yue Gao, Yuting Su","year":"2016","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2018a] An-An Liu, Weizhi Nie, Yue Gao, and Yuting Su. View-based 3-d model retrieval: A benchmark. IEEE Trans. Cybernetics, 48(3):916-928, 2018.","title":"View-based 3-d model retrieval: A benchmark","authors":"Liu","year":"2018","venue":"IEEE Trans. Cybernetics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2018b] Anan Liu, Shu Xiang, Wenhui Li, Weizhi Nie, and Yuting Su. Cross-domain 3d model retrieval via visual domain adaption. In IJCAI, pages 828-834, 2018.","title":"Cross-domain 3d model retrieval via visual domain adaption","authors":"Liu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/115"},{"position":19,"raw":"[Long et al., 2017] Mingsheng Long, Han Zhu, and et al. Deep transfer learning with joint adaptation networks. In ICML, pages 2208-2217, 2017.","title":"Deep transfer learning with joint adaptation networks","authors":"Mingsheng Long, Han Zhu","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lu et al., 2019a] Xu Lu, Lei Zhu, and et al. Flexible online multi-modal hashing for large-scale multimedia retrieval. In ACMMM 2019, pages 1129-1137, 2019.","title":"Flexible online multi-modal hashing for large-scale multimedia retrieval","authors":"Lu","year":"2019","venue":"ACMMM 2019","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lu et al., 2019b] Xu Lu, Lei Zhu, and et al. Online multi- modal hashing with dynamic query-adaption. In ACM SI- GIR, pages 715-724, 2019.","title":"Online multimodal hashing with dynamic query-adaption","authors":"Lu","year":"2019","venue":"ACM SI-GIR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Phong, 1975] Bui Tuong Phong. Illumination for computer generated pictures. Commun. ACM, 18(6):311-317, 1975.","title":"Illumination for computer generated pictures","authors":"Phong Bui Tuong","year":"1975","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Qi et al., 2017] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NIPS, pages 5099-5108, 2017.","title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","authors":"Charles Ruizhongtai, Qi, Li Yi, Hao Su, Leonidas J Guibas","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shen et al., 2018] Yiru Shen, Chen Feng, Yaoqing Yang, and Dong Tian. Mining point cloud local structures by ker- nel correlation and graph pooling. In CVPR, pages 4548- 4557, 2018.","title":"Mining point cloud local structures by kernel correlation and graph pooling","authors":"Yiru Shen, Chen Feng, Yaoqing Yang, Dong Tian","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Su et al., 2015] Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik G. Learned-Miller. Multi-view con- volutional neural networks for 3d shape recognition. In ICCV, pages 945-953, 2015.","title":"Learned-Miller. Multi-view convolutional neural networks for 3d shape recognition","authors":"Su","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Su et al., 2019] Yu-Ting Su, Yu-Qian Li, and et al. Joint het- erogeneous feature learning and distribution alignment for 2d image-based 3d object retrieval. IEEE Trans. Circuits Syst. Video Techn., 2019.","title":"Joint heterogeneous feature learning and distribution alignment for 2d image-based 3d object retrieval","authors":"Yu-Ting Su, Yu-Qian Li","year":"2019","venue":"IEEE Trans. Circuits Syst. Video Techn","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sun et al., 2016] Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In AAAI, pages 2058-2065, 2016.","title":"Return of frustratingly easy domain adaptation","authors":"Sun","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang and Breckon, 2019] Qian Wang and Toby P. Breckon. Unsupervised domain adaptation via structured prediction based selective pseudo-labeling. CoRR, abs/1911.07982, 2019.","title":"Unsupervised domain adaptation via structured prediction based selective pseudo-labeling","authors":"Qian Wang, Toby P Breckon","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang and Chen, 2017] Qian Wang and Ke Chen. Zero- shot visual recognition via bidirectional latent embedding. IJCV, 124(3):356-383, 2017.","title":"Zeroshot visual recognition via bidirectional latent embedding","authors":"Qian Wang, Ke Chen","year":"2017","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2015] Fang Wang, Le Kang, and Yi Li. Sketch- based 3d shape retrieval using convolutional neural net- works. In CVPR, pages 1875-1883, 2015.","title":"Sketchbased 3d shape retrieval using convolutional neural networks","authors":"Fang Wang, Le Kang, Yi Li","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2018] Jindong Wang, Wenjie Feng, and Yiqiang Chen et al. Visual domain adaptation with manifold embedded distribution alignment. In MM, pages 402-410, 2018.","title":"Visual domain adaptation with manifold embedded distribution alignment","authors":"Jindong Wang, Wenjie Feng, Yiqiang Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wu and et al., 2015] Zhirong Wu and Shuran Song et al. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pages 1912-1920, 2015.","title":"3d shapenets: A deep representation for volumetric shapes","authors":"Zhirong Wu, Shuran Song","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xie et al., 2017] Jin Xie, Guoxian Dai, Fan Zhu, Edward K. Wong, and Yi Fang. Deepshape: Deep-learned shape de- scriptor for 3d shape retrieval. IEEE Trans. Pattern Anal. Mach. Intell., 39(7):1335-1345, 2017.","title":"Deepshape: Deep-learned shape descriptor for 3d shape retrieval","authors":"Jin Xie, Guoxian Dai, Fan Zhu, Edward K Wong, Yi Fang","year":"2017","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xie et al., 2018] Shaoan Xie, Zibin Zheng, and et al. Learn- ing semantic representations for unsupervised domain adaptation. In ICML, pages 5419-5428, 2018.","title":"Learning semantic representations for unsupervised domain adaptation","authors":"Xie","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang and et al., 2017] Jing Zhang and et al. Joint geomet- rical and statistical alignment for visual domain adapta- tion. In CVPR, pages 5150-5158, 2017.","title":"Joint geometrical and statistical alignment for visual domain adaptation","authors":"Zhang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2019] Lei Zhang, Shanshan Wang, and et al. Manifold criterion guided transfer learning via intermedi- ate domain generation. CoRR, abs/1903.10211, 2019.","title":"Manifold criterion guided transfer learning via intermediate domain generation","authors":"Lei Zhang, Shanshan Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}