{"type":"ScholarlyArticle","title":"Intrinsic Image Decomposition by Pursuing Reflectance Image","authors":[{"name":"Tzu-Heng Lin","slug":"tzu-heng-lin","given":"Tzu-Heng","family":"Lin","affiliation":"School of Computer Science , Peking University"},{"name":"Pengxiao Wang","slug":"pengxiao-wang","given":"Pengxiao","family":"Wang","affiliation":"School of Computer Science , Peking University"},{"name":"Yizhou Wang","slug":"yizhou-wang","given":"Yizhou","family":"Wang","affiliation":"School of Computer Science , Peking University; Center on Frontiers of Computing Studies , Peking University"}],"abstract":"Intrinsic image decomposition is a fundamental problem for many computer vision applications. While recent deep learning based methods have achieved very promising results on the synthetic densely labeled datasets, the results on the real-world dataset are still far from human level performance. This is mostly because collecting dense supervision on a real-world dataset is impossible. Only a sparse set of pairwise judgement from human is often used. It's very difficult for models to learn in such settings. \r\nIn this paper, we investigate the possibilities of only using reflectance images for supervision during training. In this way, the demand for labeled data is greatly reduced. In order to achieve this goal, we take a deep investigation into the reflectance images. We find that reflectance images are actually comprised of two components: the flat surfaces with low frequency information, and the boundaries with high frequency details. Then, we propose to disentangle the learning process of the two components of the reflectance images. We argue that through this procedure, the reflectance images can be better modeled, and in the meantime, the shading images, though not supervised, can also achieve decent result. Extensive experiments show that our proposed network outperforms current state-of-the-art results by a large margin on the most challenging real-world IIW dataset. We also surprisingly find that on the densely labeled datasets (MIT and MPI-Sintel), our network can also achieve state-of-the-art results on both reflectance and shading images, when we only apply supervision on the reflectance images during training.","doi":"10.24963/ijcai.2022/163","url":"https://ijcai.org/proceedings/2022/163","pdf":"https://ijcai.org/proceedings/2022/0163.pdf","markdown":"https://ijcai.org/proceedings/2022/163.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/163","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":163,"pages":{"first":1166,"last":1172},"keywords":["Computer Vision: Structural and Model-Based Approaches, Knowledge Representation and Reasoning","Computer Vision: Scene analysis and understanding"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986589","length_sec":70,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984732","length_sec":330,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Barron and Malik, 2014] Jonathan T Barron and Jitendra Malik. Shape, illumination, and reflectance from shading. TPAMI, 2014.","title":"Shape, illumination, and reflectance from shading","authors":"Jonathan T Barron, Jitendra Malik","year":"2014","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baslamisli et al., 2018] Anil S Baslamisli, Thomas T Groenestege, Partha Das, Hoang-An Le, Sezer Karaoglu, and Theo Gevers. Joint learning of intrinsic images and semantic segmentation. In ECCV, 2018.","title":"Joint learning of intrinsic images and semantic segmentation","authors":"S Anil, Baslamisli, Partha Thomas T Groenestege, Hoang-An Das, Sezer Le, Theo Karaoglu, Gevers","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bell et al., 2014] Sean Bell, Kavita Bala, and Noah Snavely. Intrinsic images in the wild. TOG, 2014.","title":"Intrinsic images in the wild","authors":"Sean Bell, Kavita Bala, Noah Snavely","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bi et al., 2015] Sai Bi, Xiaoguang Han, and Yizhou Yu. An l1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition. TOG, 2015.","title":"An l1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition","authors":"Sai Bi, Xiaoguang Han, Yizhou Yu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Butler et al., 2012] Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In ECCV, 2012.","title":"A naturalistic open source movie for optical flow evaluation","authors":"J Daniel, Jonas Butler, Garrett B Wulff, Michael J Stanley, Black","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen and Koltun, 2013] Qifeng Chen and Vladlen Koltun. A simple model for intrinsic image decomposition with depth cues. In ICCV, 2013.","title":"A simple model for intrinsic image decomposition with depth cues","authors":"Chen, Koltun ;, Qifeng Chen, Vladlen Koltun","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2019] Yunpeng Chen, Haoqi Fan, Bing Xu, Zhicheng Yan, Yannis Kalantidis, Marcus Rohrbach, Shuicheng Yan, and Jiashi Feng. Drop an octave: Reduc- ing spatial redundancy in convolutional neural networks with octave convolution. In ICCV, 2019.","title":"Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution","authors":"Yunpeng Chen, Haoqi Fan, Bing Xu, Zhicheng Yan, Yannis Kalantidis, Marcus Rohrbach, Shuicheng Yan, Jiashi Feng","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cheng et al., 2018] Lechao Cheng, Chengyi Zhang, and Zicheng Liao. Intrinsic image transformation via scale space decomposition. In CVPR, 2018.","title":"Intrinsic image transformation via scale space decomposition","authors":"Cheng","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fan et al., 2018] Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, and David Wipf. Revisiting deep intrinsic image decompositions. In CVPR, 2018.","title":"Revisiting deep intrinsic image decompositions","authors":"Fan","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Grosse et al., 2009] Roger Grosse, Micah K Johnson, Ed- ward H Adelson, and William T Freeman. Ground truth dataset and baseline evaluations for intrinsic image algo- rithms. In ICCV, 2009.","title":"Ground truth dataset and baseline evaluations for intrinsic image algorithms","authors":"Roger Grosse, Micah K Johnson, Edward H Adelson, William T Freeman","year":"2009","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kovacs et al., 2017] Balazs Kovacs, Sean Bell, Noah Snavely, and Kavita Bala. Shading annotations in the wild. In CVPR, 2017.","title":"Shading annotations in the wild","authors":"Balazs Kovacs, Sean Bell, Noah Snavely, Kavita Bala","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lettry et al., 2018] Louis Lettry, Kenneth Vanhoey, and Luc Van Gool. Darn: a deep adversarial residual network for intrinsic image decomposition. In WACV, 2018.","title":"Darn: a deep adversarial residual network for intrinsic image decomposition","authors":"Lettry","year":"2018","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Snavely, 2018a] Zhengqi Li and Noah Snavely. Cgintrinsics: Better intrinsic image decomposition through physically-based rendering. In ECCV, 2018.","title":"Cgintrinsics: Better intrinsic image decomposition through physically-based rendering","authors":"Snavely ; Li, Zhengqi Li, Noah Snavely","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li and Snavely, 2018b] Zhengqi Li and Noah Snavely. Learning intrinsic image decomposition from watching the world. In CVPR, 2018.","title":"Learning intrinsic image decomposition from watching the world","authors":"Snavely ; Li, Zhengqi Li, Noah Snavely","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu and Lu, 2020] Yunfei Liu and Feng Lu. Separate in la- tent space: Unsupervised single image layer separation. In AAAI, 2020.","title":"Separate in latent space: Unsupervised single image layer separation","authors":"Yunfei Liu, Feng Lu","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2019] Risheng Liu, Cheng Yang, Long Ma, Miao Zhang, Xin Fan, and Zhongxuan Luo. Enhanced residual dense intrinsic network for intrinsic image decomposition. In ICME, 2019.","title":"Enhanced residual dense intrinsic network for intrinsic image decomposition","authors":"Risheng Liu, Cheng Yang, Long Ma, Miao Zhang, Xin Fan, Zhongxuan Luo","year":"2019","venue":"ICME","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2020] Yunfei Liu, Yu Li, Shaodi You, and Feng Lu. Unsupervised learning for intrinsic image decomposi- tion from a single image. In CVPR, 2020.","title":"Unsupervised learning for intrinsic image decomposition from a single image","authors":"Yunfei Liu, Yu Li, Shaodi You, Feng Lu","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Narihira et al., 2015a] Takuya Narihira, Michael Maire, and Stella X Yu. Direct intrinsics: Learning albedo-shading decomposition by convolutional regression. In ICCV, 2015.","title":"Direct intrinsics: Learning albedo-shading decomposition by convolutional regression","authors":"Narihira","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Narihira et al., 2015b] Takuya Narihira, Michael Maire, and Stella X Yu. Learning lightness from human judge- ment on relative reflectance. In CVPR, 2015.","title":"Learning lightness from human judgement on relative reflectance","authors":"Narihira","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nestmeyer and Gehler, 2017] Thomas Nestmeyer and Pe- ter V Gehler. Reflectance adaptive filtering improves in- trinsic image estimation. In CVPR, 2017.","title":"Thomas Nestmeyer and Peter V Gehler. Reflectance adaptive filtering improves intrinsic image estimation","authors":"Gehler Nestmeyer","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shen and Yeo, 2011] Li Shen and Chuohao Yeo. Intrinsic images decomposition using a local and global sparse rep- resentation of reflectance. In CVPR, 2011.","title":"Intrinsic images decomposition using a local and global sparse representation of reflectance","authors":"Yeo ; ] Li Shen, Chuohao Shen, Yeo","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shi et al., 2017] Jian Shi, Yue Dong, Hao Su, and Stella X Yu. Learning non-lambertian object intrinsics across shapenet categories. In CVPR, 2017.","title":"Learning non-lambertian object intrinsics across shapenet categories","authors":"Jian Shi, Yue Dong, Hao Su, Stella X Yu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sun et al., 2019] Ke Sun, Bin Xiao, Dong Liu, and Jing- dong Wang. Deep high-resolution representation learning for human pose estimation. In CVPR, 2019.","title":"Deep high-resolution representation learning for human pose estimation","authors":"Ke Sun, Bin Xiao, Dong Liu, Jingdong Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang and Lu, 2019] Zongji Wang and Feng Lu. Single im- age intrinsic decomposition with discriminative feature en- coding. In ICCV Workshop, 2019.","title":"Single image intrinsic decomposition with discriminative feature encoding","authors":"Wang, Lu ; ] Zongji Wang, Feng Lu","year":"2019","venue":"ICCV Workshop","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2021] Qing Zhang, Jin Zhou, Lei Zhu, Wei Sun, Chunxia Xiao, and Wei-Shi Zheng. Unsupervised in- trinsic image decomposition using internal self-similarity cues. PAMI, 2021.","title":"Unsupervised intrinsic image decomposition using internal self-similarity cues","authors":"Qing Zhang, Jin Zhou, Lei Zhu, Wei Sun, Chunxia Xiao, Wei-Shi Zheng","year":"2021","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2012] Qi Zhao, Ping Tan, Qiang Dai, Li Shen, Enhua Wu, and Stephen Lin. A closed-form solution to retinex with nonlocal texture constraints. TPAMI, 2012.","title":"A closed-form solution to retinex with nonlocal texture constraints","authors":"Qi Zhao, Ping Tan, Qiang Dai, Li Shen, Enhua Wu, Stephen Lin","year":"2012","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2015] Tinghui Zhou, Philipp Krahenbuhl, and Alexei A Efros. Learning data-driven reflectance priors for intrinsic image decomposition. In ICCV, 2015.","title":"Learning data-driven reflectance priors for intrinsic image decomposition","authors":"Zhou","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2019] Hao Zhou, Xiang Yu, and David W. Ja- cobs. Glosh: Global-local spherical harmonics for intrin- sic image decomposition. In ICCV, 2019.","title":"Glosh: Global-local spherical harmonics for intrinsic image decomposition","authors":"Hao Zhou, Xiang Yu, David W Jacobs","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zoran et al., 2015] Daniel Zoran, Phillip Isola, Dilip Krish- nan, and William T Freeman. Learning ordinal relation- ships for mid-level vision. In ICCV, 2015.","title":"Learning ordinal relationships for mid-level vision","authors":"Phillip Daniel Zoran, Dilip Isola, William T Krishnan, Freeman","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}