{"type":"ScholarlyArticle","title":"Iterative Geometry-Aware Cross Guidance Network for Stereo Image Inpainting","authors":[{"name":"Ang Li","slug":"ang-li","given":"Ang","family":"Li","affiliation":"Aviation University of Air Force"},{"name":"Shanshan Zhao","slug":"shanshan-zhao","given":"Shanshan","family":"Zhao","affiliation":"JD Explore Academy , JD.COM"},{"name":"Zhang Qingjie","slug":"zhang-qingjie","given":"Zhang","family":"Qingjie","affiliation":"Aviation University of Air Force"},{"name":"Qiuhong Ke","slug":"qiuhong-ke","given":"Qiuhong","family":"Ke","affiliation":"The University of Melbourne"}],"abstract":"Currently,  single  image  inpainting  has  achieved promising results based on deep convolutional neural  networks.   However,  inpainting  on  stereo  images  with  missing  regions  has  not  been  explored thoroughly, which is also a significant but different problem. One crucial requirement for stereo image inpainting is stereo consistency.  To achieve it, we propose an Iterative Geometry-Aware Cross Guidance Network (IGGNet). The IGGNet contains two key ingredients, i.e., a Geometry-Aware Attention(GAA) module and an Iterative  Cross Guidance(ICG) strategy. The GAA module relies on the epipolar geometry  cues and learns the geometry-aware guidance from one view to another,  which is beneficial to make the corresponding regions in two views consistent. However, learning guidance from co-existing missing regions is challenging. To address  this  issue,  the  ICG  strategy  is  proposed, which can alternately narrow down the missing regions of the two views in an iterative manner. Experimental  results demonstrate that our proposed network outperforms the latest stereo  image inpainting model and state-of-the-art single image inpainting models.","doi":"10.24963/ijcai.2022/147","url":"https://ijcai.org/proceedings/2022/147","pdf":"https://ijcai.org/proceedings/2022/0147.pdf","markdown":"https://ijcai.org/proceedings/2022/147.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/147","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":147,"pages":{"first":1053,"last":1059},"keywords":["Computer Vision: Applications"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986822","length_sec":85,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984965","length_sec":337,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Butler et al., 2012] D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. In Proceedings of the European Confer- ence on Computer Vision, 2012.","title":"A naturalistic open source movie for optical flow evaluation","authors":"D J Butler, J Wulff, G B Stanley, M J Black","year":"2012","venue":"Proceedings of the European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2019] Shen Chen, Wei Ma, and Yue Qin. Cnn- based stereoscopic image inpainting. In International Conference on Image and Graphics, 2019.","title":"Cnnbased stereoscopic image inpainting","authors":"Shen Chen, Wei Ma, Yue Qin","year":"2019","venue":"International Conference on Image and Graphics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Proceedings of the Annual Con- ference on Neural Information Processing Systems, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Proceedings of the Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Guo et al., 2019] Zongyu Guo, Z. Chen, Tao Yu, Jianxian Chen, and S. Liu. Progressive image inpainting with full- resolution residual network. Proceedings of the ACM In- ternational Conference on Multimedia, 2019.","title":"Progressive image inpainting with fullresolution residual network","authors":"Z Zongyu Guo, Tao Chen, Jianxian Yu, S Chen, Liu","year":"2019","venue":"Proceedings of the ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hervieu et al., 2010] Alexandre Hervieu, Nicolas Pa- padakis, Aurélie Bugeau, Pau Gargallo, and Vicent Caselles. Stereoscopic image inpainting: distinct depth maps and images inpainting. In International Conference on Pattern Recognition, 2010.","title":"Stereoscopic image inpainting: distinct depth maps and images inpainting","authors":"Alexandre Hervieu, Nicolas Papadakis, Aurélie Bugeau, Pau Gargallo, Vicent Caselles","year":"2010","venue":"International Conference on Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Heusel et al., 2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of the Annual Conference on Neural Information Processing Systems, 2017.","title":"Gans trained by a two time-scale update rule converge to a local nash equilibrium","authors":"Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Sepp Hochreiter","year":"2017","venue":"Proceedings of the Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hirschmuller, 2005] Heiko Hirschmuller. Accurate and ef- ficient stereo processing by semi-global matching and mu- tual information. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2005.","title":"Accurate and efficient stereo processing by semi-global matching and mutual information","authors":"Heiko Hirschmuller","year":"2005","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Iizuka et al., 2017] Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Globally and locally consistent image completion. ACM Transactions on Graphics, 2017.","title":"Globally and locally consistent image completion","authors":"Satoshi Iizuka, Edgar Simo-Serra, Hiroshi Ishikawa","year":"2017","venue":"ACM Transactions on Graphics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2019a] Ang Li, Jianzhong Qi, Rui Zhang, Xingjun Ma, and Kotagiri Ramamohanarao. Generative image in- painting with submanifold alignment. In Proceedings of the Twenty-Eighth International Joint Conference on Arti- ficial Intelligence, 2019.","title":"Generative image inpainting with submanifold alignment","authors":"Li","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2019/114"},{"position":10,"raw":"[Li et al., 2019b] Ang Li, Jianzhong Qi, Rui Zhang, and Ko- tagiri Ramamohanarao. Boosted gan with semantically in- terpretable information for image inpainting. In Proceed- ings of the International Joint Conference on Neural Net- works, 2019.","title":"Boosted gan with semantically interpretable information for image inpainting","authors":"Li","year":"2019","venue":"Proceedings of the International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2020] Jingyuan Li, Ning Wang, Lefei Zhang, Bo Du, and Dacheng Tao. Recurrent feature reasoning for image inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.","title":"Recurrent feature reasoning for image inpainting","authors":"Jingyuan Li, Ning Wang, Lefei Zhang, Bo Du, Dacheng Tao","year":"2020","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liang et al., 2018] Zhengfa Liang, Yiliu Feng, Yulan Guo, Hengzhu Liu, Wei Chen, Linbo Qiao, Li Zhou, and Jian- feng Zhang. Learning for disparity estimation through fea- ture constancy. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.","title":"Learning for disparity estimation through feature constancy","authors":"Liang","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2018] Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Im- age inpainting for irregular holes using partial convolu- tions. In Proceedings of the European Conference on Computer Vision, 2018.","title":"Image inpainting for irregular holes using partial convolutions","authors":"Liu","year":"2018","venue":"Proceedings of the European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2019] Hongyu Liu, Bin Jiang, Yi Xiao, and Chao Yang. Coherent semantic attention for image inpainting. In Proceedings of the IEEE international Conference on Computer Vision, 2019.","title":"Coherent semantic attention for image inpainting","authors":"Hongyu Liu, Bin Jiang, Yi Xiao, Chao Yang","year":"2019","venue":"Proceedings of the IEEE international Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2020] Wei Ma, Mana Zheng, Wenguang Ma, Shibiao Xu, and Xiaopeng Zhang. Learning across views for stereo image completion. IET Computer Vision, 14:482-492, 2020.","title":"Learning across views for stereo image completion","authors":"Wei Ma, Mana Zheng, Wenguang Ma, Shibiao Xu, Xiaopeng Zhang","year":"2020","venue":"IET Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mayer et al., 2016] Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Doso- vitskiy, and Thomas Brox. A large dataset to train con- volutional networks for disparity, optical flow, and scene flow estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.","title":"A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation","authors":"Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, Thomas Brox","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Miyato et al., 2018] Takeru Miyato, T. Kataoka, Masanori Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks. ArXiv, abs/1802.05957, 2018.","title":"Spectral normalization for generative adversarial networks","authors":"T Takeru Miyato, Masanori Kataoka, Y Koyama, Yoshida","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Morse et al., 2012] Bryan Morse, Joel Howard, Scott Co- hen, and Brian Price. Patchmatch-based content comple- tion of stereo image pairs. In International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, 2012.","title":"Patchmatch-based content completion of stereo image pairs","authors":"Bryan Morse, Joel Howard, Scott Cohen, Brian Price","year":"2012","venue":"International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pathak et al., 2016] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.","title":"Context encoders: Feature learning by inpainting","authors":"Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, Alexei A Efros","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2008] Liang Wang, Hailin Jin, Ruigang Yang, and Minglun Gong. Stereoscopic inpainting: Joint color and depth completion from stereo images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008.","title":"Stereoscopic inpainting: Joint color and depth completion from stereo images","authors":"Liang Wang, Hailin Jin, Ruigang Yang, Minglun Gong","year":"2008","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yi et al., 2020] Zili Yi, Qiang Tang, Shekoofeh Azizi, Dae- sik Jang, and Zhan Xu. Contextual residual aggregation for ultra high-resolution image inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.","title":"Contextual residual aggregation for ultra high-resolution image inpainting","authors":"Zili Yi, Qiang Tang, Shekoofeh Azizi, Daesik Jang, Zhan Xu","year":"2020","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yu et al., 2018] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with contextual attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, 2018.","title":"Generative image inpainting with contextual attention","authors":"Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, Thomas S Huang","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu et al., 2019] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas Huang. Free-form image in- painting with gated convolution. In Proceedings of the IEEE International Conference on Computer Vision, 2019.","title":"Free-form image inpainting with gated convolution","authors":"Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, Thomas Huang","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zeng et al., 2021] Yu Zeng, Zhe Lin, Huchuan Lu, and Vishal M. Patel. Cr-fill: Generative image inpainting with auxiliary contextual reconstruction. In Proceedings of the IEEE International Conference on Computer Vision, 2021.","title":"Cr-fill: Generative image inpainting with auxiliary contextual reconstruction","authors":"Yu Zeng, Zhe Lin, Huchuan Lu, Vishal M Patel","year":"2021","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}