{"type":"ScholarlyArticle","title":"Image Composition with Depth Registration","authors":[{"name":"Zan Li","slug":"zan-li","given":"Zan","family":"Li","affiliation":"State Key Laboratory of Computer Science , Institute of Software , Chinese Academy of Sciences; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Wencheng Wang","slug":"wencheng-wang","given":"Wencheng","family":"Wang","affiliation":"State Key Laboratory of Computer Science , Institute of Software , Chinese Academy of Sciences; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Fei Hou","slug":"fei-hou","given":"Fei","family":"Hou","affiliation":"State Key Laboratory of Computer Science , Institute of Software , Chinese Academy of Sciences; University of Chinese Academy of Sciences , Beijing , China"}],"abstract":"Handling occlusions is still a challenging problem for image composition. It always requires the source contents to be completely in front of the target contents or needs manual interventions to adjust occlusions, which is very tedious. Though several methods have suggested exploiting priors or learning techniques for promoting occlusion determination, their potentials are much limited. This paper addresses the challenge by presenting a depth registration method for merging the source contents seamlessly into the 3D space that the target image represents. Thus, the occlusions between the source contents and target contents can be conveniently handled through pixel-wise depth comparisons, allowing the user to more efficiently focus on the designs for image composition. Experimental results show that we can conveniently handle occlusions in image composition and improve efficiency by about 4 times compared to Photoshop.","doi":"10.24963/ijcai.2023/126","url":"https://ijcai.org/proceedings/2023/126","pdf":"https://ijcai.org/proceedings/2023/0126.pdf","markdown":"https://ijcai.org/proceedings/2023/126.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/126","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":126,"pages":{"first":1134,"last":1141},"keywords":["Computer Vision: CV: Scene analysis and understanding","Multidisciplinary Topics and Applications: MDA: Arts and creativity"],"videos":[],"references":[{"position":1,"raw":"[Azadi et al., 2020] Samaneh Azadi, Deepak Pathak, Sayna Ebrahimi, and Trevor Darrell. Compositional gan: Learn- ing image-conditional binary composition. International Journal of Computer Vision, 128(10):2570-2585, 2020.","title":"Compositional gan: Learning image-conditional binary composition","authors":"Azadi","year":"2020","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bao et al., 2022] Zhongyun Bao, Chengjiang Long, Gang Fu, Daquan Liu, Yuanzhen Li, Jiaming Wu, and Chunxia Xiao. Deep image-based illumination harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18542-18551, 2022.","title":"Deep image-based illumination harmonization","authors":"Bao","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. In Proceedings of the Euro- pean conference on computer vision (ECCV), pages 801- 818, 2018.","title":"Encoder-decoder with atrous separable convolution for semantic image segmentation","authors":"Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, Hartwig Adam","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cong et al., 2021] Wenyan Cong, Li Niu, Jianfu Zhang, Jing Liang, and Liqing Zhang. Bargainnet: Background- guided domain translation for image harmonization. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEE, 2021.","title":"Bargainnet: Backgroundguided domain translation for image harmonization","authors":"Wenyan Cong, Li Niu, Jianfu Zhang, Jing Liang, Liqing Zhang","year":"2021","venue":"2021 IEEE International Conference on Multimedia and Expo (ICME)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cong et al., 2022] Wenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, and Liqing Zhang. High- resolution image harmonization via collaborative dual transformations. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 18470-18479, 2022.","title":"Highresolution image harmonization via collaborative dual transformations","authors":"Wenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, Liqing Zhang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Farbman et al., 2011] Zeev Farbman, Raanan Fattal, and Dani Lischinski. Convolution pyramids. ACM Trans. Graph., 30(6):175, 2011.","title":"Convolution pyramids","authors":"Zeev Farbman, Raanan Fattal, Dani Lischinski","year":"2011","venue":"ACM Trans. Graph","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fu et al., 2008] Xinyuan Fu, He Guo, Yuxin Wang, Tianyang Liu, and Han Li. Arbitrary image cloning. In Signal Processing for Image Enhancement and Multime- dia Processing, pages 289-300. Springer, 2008.","title":"Arbitrary image cloning","authors":"Xinyuan Fu, He Guo, Yuxin Wang, Tianyang Liu, Han Li","year":"2008","venue":"Signal Processing for Image Enhancement and Multimedia Processing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo and Sim, 2009] Dong Guo and Terence Sim. Color me right-seamless image compositing. In International Conference on Computer Analysis of Images and Patterns, pages 444-451. Springer, 2009.","title":"Color me right-seamless image compositing","authors":"Sim ; Guo, Dong Guo, Terence Sim","year":"2009","venue":"International Conference on Computer Analysis of Images and Patterns","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hang et al., 2022] Yucheng Hang, Bin Xia, Wenming Yang, and Qingmin Liao. Scs-co: Self-consistent style con- trastive learning for image harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 19710-19719, 2022.","title":"Scs-co: Self-consistent style contrastive learning for image harmonization","authors":"Yucheng Hang, Bin Xia, Wenming Yang, Qingmin Liao","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2019] Han Huang, Yihui Liang, Xiaowei Yang, and Zhifeng Hao. Pixel-level discrete multiobjec- tive sampling for image matting. IEEE Transactions on Image Processing, 28(8):3739-3751, 2019.","title":"Pixel-level discrete multiobjective sampling for image matting","authors":"Han Huang, Yihui Liang, Xiaowei Yang, Zhifeng Hao","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kornilov and Safonov, 2018] Anton S Kornilov and Ilia V Safonov. An overview of watershed algorithm imple- mentations in open source libraries. Journal of Imaging, 4(10):123, 2018.","title":"An overview of watershed algorithm implementations in open source libraries","authors":"Safonov Kornilov, Anton S Kornilov, Ilia V Safonov","year":"2018","venue":"Journal of Imaging","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Levin et al., 2007] Anat Levin, Dani Lischinski, and Yair Weiss. A closed-form solution to natural image matting. IEEE transactions on pattern analysis and machine intel- ligence, 30(2):228-242, 2007.","title":"A closed-form solution to natural image matting","authors":"Levin","year":"2007","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Niu et al., 2021] Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, and Liqing Zhang. Making images real again: A comprehensive survey on deep image com- position. arXiv preprint arXiv:2106.14490, 2021.","title":"Making images real again: A comprehensive survey on deep image composition","authors":"Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, Liqing Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pérez et al., 2003] Patrick Pérez, Michel Gangnet, and An- drew Blake. Poisson image editing. In ACM SIGGRAPH 2003 Papers, pages 313-318. 2003.","title":"Poisson image editing","authors":"Patrick Pérez, Michel Gangnet","year":"2003","venue":"ACM SIGGRAPH 2003 Papers","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rother et al., 2004] Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. \"GrabCut\" interactive foreground ex- traction using iterated graph cuts. ACM transactions on graphics (TOG), 23(3):309-314, 2004.","title":"GrabCut\" interactive foreground extraction using iterated graph cuts","authors":"Carsten Rother, Vladimir Kolmogorov, Andrew Blake","year":"2004","venue":"ACM transactions on graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tan et al., 2019] Xuehan Tan, Panpan Xu, Shihui Guo, and Wencheng Wang. Image composition of partially occluded objects. Computer Graphics Forum, 38(7):641-650, 2019.","title":"Image composition of partially occluded objects","authors":"Xuehan Tan, Panpan Xu, Shihui Guo, Wencheng Wang","year":"2019","venue":"Computer Graphics Forum","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tang et al., 2019] Jingwei Tang, Yagiz Aksoy, Cengiz Oztireli, Markus Gross, and Tunc Ozan Aydin. Learning- based sampling for natural image matting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 3055-3063, 2019.","title":"Learningbased sampling for natural image matting","authors":"Tang","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2016] Wencheng Wang, Panpan Xu, Xiaohui Bie, and Miao Hua. Enhanced use of mattes for easy im- age composition. IEEE Transactions on Image Processing, 25(10):4608-4616, 2016.","title":"Enhanced use of mattes for easy image composition","authors":"Wencheng Wang, Panpan Xu, Xiaohui Bie, Miao Hua","year":"2016","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yu et al., 2021] Bing Yu, Youdong Ding, Zhifeng Xie, and Dongjin Huang. Stacked generative adversarial networks for image compositing. EURASIP Journal on Image and Video Processing, 2021(1):1-20, 2021.","title":"Stacked generative adversarial networks for image compositing","authors":"Yu","year":"2021","venue":"EURASIP Journal on Image and Video Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yuan et al., 2022] Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. Newcrfs: Neural window fully-connected crfs for monocular depth estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022.","title":"Newcrfs: Neural window fully-connected crfs for monocular depth estimation","authors":"Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, Ping Tan","year":"2022","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhan et al., 2020] Fangneng Zhan, Shijian Lu, Changgong Zhang, Feiying Ma, and Xuansong Xie. Adversarial image composition with auxiliary illumination. In Proceedings of the Asian Conference on Computer Vision, 2020.","title":"Adversarial image composition with auxiliary illumination","authors":"Zhan","year":"2020","venue":"Proceedings of the Asian Conference on Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}