{"type":"ScholarlyArticle","title":"Self-supervised Learning and Adaptation for Single Image Dehazing","authors":[{"name":"Yudong Liang","slug":"yudong-liang","given":"Yudong","family":"Liang","affiliation":"School of Computer and Information Technology , Institute of Big Data Science and Industry , Shanxi University , Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education , Shanxi , China"},{"name":"Bin Wang","slug":"bin-wang","given":"Bin","family":"Wang","affiliation":"School of Computer and Information Technology , Institute of Big Data Science and Industry , Shanxi University , Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education , Shanxi , China"},{"name":"Wangmeng Zuo","slug":"wangmeng-zuo","given":"Wangmeng","family":"Zuo","affiliation":"School of Computer Science at Harbin Institute of Technology , Haerbin , China"},{"name":"Jiaying Liu","slug":"jiaying-liu","given":"Jiaying","family":"Liu","affiliation":"Wangxuan Institute of Computer Technology , Peking University , Beijing , China"},{"name":"Wenqi Ren","slug":"wenqi-ren","given":"Wenqi","family":"Ren","affiliation":"School of Cyber Science and Technology , Shenzhen Campus , Sun Yat-sen University , Shenzhen , China"}],"abstract":"Existing deep image dehazing methods usually depend on supervised learning with a large number of hazy-clean image pairs which are expensive or difficult to collect. Moreover, dehazing performance of the learned model may deteriorate significantly when the training hazy-clean image pairs are insufficient and are different from real hazy images in applications. In this paper, we show that exploiting large scale training set and adapting to real hazy images are two critical issues in learning effective deep dehazing models. Under the depth guidance estimated by a well-trained depth estimation network, we leverage the conventional atmospheric scattering model to generate massive hazy-clean image pairs for the self-supervised pre-training of dehazing network. Furthermore, self-supervised adaptation is presented to adapt pre-trained network to real hazy images. Learning without forgetting strategy is also deployed in self-supervised adaptation by combining self-supervision and model adaptation via contrastive learning. Experiments show that our proposed method performs favorably against the state-of-the-art methods, and is quite efficient, i.e., handling a 4K image in 23 ms. The codes are available at https://github.com/DongLiangSXU/SLAdehazing.","doi":"10.24963/ijcai.2022/159","url":"https://ijcai.org/proceedings/2022/159","pdf":"https://ijcai.org/proceedings/2022/0159.pdf","markdown":"https://ijcai.org/proceedings/2022/159.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/159","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":159,"pages":{"first":1137,"last":1143},"keywords":["Computer Vision: Transfer, low-shot, semi- and un- supervised learning","Computer Vision: Neural generative models, auto encoders, GANs","Computer Vision: Applications","Computer Vision: Machine Learning for Vision"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987034","length_sec":77,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985177","length_sec":712,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Alhashim and Wonka, 2018] Ibraheem Alhashim and Peter Wonka. High quality monocular depth estimation via transfer learning. arXiv e-prints, abs/1812.11941, 2018.","title":"Ibraheem Alhashim and Peter Wonka. High quality monocular depth estimation via transfer learning","authors":"Wonka Alhashim","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Berman et al., 2016] Dana Berman, Shai Avidan, et al. Non-local image dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1674-1682, 2016.","title":"Non-local image dehazing","authors":"Dana Berman, Shai Avidan","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2016] Bolun Cai, Xiangmin Xu, Kui Jia, Chun- mei Qing, and Dacheng Tao. Dehazenet: An end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 25(11):5187-5198, 2016.","title":"Dehazenet: An end-to-end system for single image haze removal","authors":"Cai","year":"2016","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Zeyuan Chen, Yangchao Wang, Yang Yang, and Dong Liu. Psd: Principled synthetic-to-real de- hazing guided by physical priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7180-7189, 2021.","title":"Psd: Principled synthetic-to-real dehazing guided by physical priors","authors":"Chen","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2020] Hang Dong, Jinshan Pan, Lei Xiang, Zhe Hu, Xinyi Zhang, Fei Wang, and Ming-Hsuan Yang. Multi-scale boosted dehazing network with dense feature fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2157- 2167, 2020.","title":"Multi-scale boosted dehazing network with dense feature fusion","authors":"Dong","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Golts et al., 2019] Alona Golts, Daniel Freedman, and Michael Elad. Unsupervised single image dehazing us- ing dark channel prior loss. IEEE transactions on Image Processing, 29:2692-2701, 2019.","title":"Unsupervised single image dehazing using dark channel prior loss","authors":"Alona Golts, Daniel Freedman, Michael Elad","year":"2019","venue":"IEEE transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2010] Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. IEEE transactions on pattern analysis and machine intelligence, 33(12):2341-2353, 2010.","title":"Single image haze removal using dark channel prior","authors":"Kaiming He, Jian Sun, Xiaoou Tang","year":"2010","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Israël and Kasten, 1959] Hans Israël and Fritz Kasten. Koschmieders theorie der horizontalen sichtweite. In Die Sichtweite im Nebel und die Möglichkeiten ihrer künstlichen Beeinflussung, pages 7-10. Springer, 1959.","title":"Die Sichtweite im Nebel und die Möglichkeiten ihrer künstlichen Beeinflussung","authors":"Kasten Israël, Hans Israël, Fritz Kasten","year":"1959","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kar et al., 2021] Aupendu Kar, Sobhan Kanti Dhara, De- bashis Sen, and Prabir Kumar Biswas. Zero-shot sin- gle image restoration through controlled perturbation of koschmieder's model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16205-16215, 2021.","title":"Zero-shot single image restoration through controlled perturbation of koschmieder's model","authors":"Kar","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2017] Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, and Dan Feng. Aod-net: All-in-one dehaz- ing network. In Proceedings of the IEEE International Conference on Computer Vision, pages 4770-4778, 2017.","title":"Aod-net: All-in-one dehazing network","authors":"Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, Dan Feng","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2018] Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, and Zhangyang Wang. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 28(1):492-505, 2018.","title":"Benchmarking single-image dehazing and beyond","authors":"Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, Zhangyang Wang","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2020] Boyun Li, Yuanbiao Gou, Jerry Zitao Liu, Hongyuan Zhu, Joey Tianyi Zhou, and Xi Peng. Zero-shot image dehazing. IEEE Transactions on Image Processing, 29:8457-8466, 2020.","title":"Zero-shot image dehazing","authors":"Li","year":"2020","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2021a] Boyun Li, Yuanbiao Gou, Shuhang Gu, Jerry Zitao Liu, Joey Tianyi Zhou, and Xi Peng. You only look yourself: Unsupervised and untrained single image dehazing neural network. International Journal of Com- puter Vision, 129(5):1754-1767, 2021.","title":"You only look yourself: Unsupervised and untrained single image dehazing neural network","authors":"Li","year":"2021","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2021b] Hongyu Li, Jia Li, Dong Zhao, and Long Xu. Dehazeflow: Multi-scale conditional flow network for single image dehazing. In Proceedings of the 29th ACM International Conference on Multimedia, pages 2577- 2585, 2021.","title":"Dehazeflow: Multi-scale conditional flow network for single image dehazing","authors":"Li","year":"2021","venue":"Proceedings of the 29th ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2019] Xiaohong Liu, Yongrui Ma, Zhihao Shi, and Jun Chen. Griddehazenet: Attention-based multi- scale network for image dehazing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7314-7323, 2019.","title":"Griddehazenet: Attention-based multiscale network for image dehazing","authors":"Xiaohong Liu, Yongrui Ma, Zhihao Shi, Jun Chen","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Qin et al., 2019] Xu Qin, Zhilin Wang, Yuanchao Bai, Xi- aodong Xie, and Huizhu Jia. Ffa-net: Feature fusion at- tention network for single image dehazing. arXiv preprint arXiv:1911.07559, 2019.","title":"Ffa-net: Feature fusion attention network for single image dehazing","authors":"Zhilin Xu Qin, Yuanchao Wang, Xiaodong Bai, Huizhu Xie, Jia","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ren et al., 2016] Wenqi Ren, Si Liu, Hua Zhang, Jinshan Pan, Xiaochun Cao, and Ming-Hsuan Yang. Single image dehazing via multi-scale convolutional neural networks. In European conference on computer vision, pages 154-169. Springer, 2016.","title":"Single image dehazing via multi-scale convolutional neural networks","authors":"Si Wenqi Ren, Hua Liu, Jinshan Zhang, Xiaochun Pan, Ming-Hsuan Cao, Yang","year":"2016","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pages 234-241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical image computing and computerassisted intervention","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tan, 2008] Robby T Tan. Visibility in bad weather from a single image. In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8. IEEE, 2008.","title":"Visibility in bad weather from a single image","authors":"Robby T Tan","year":"2008","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ulyanov et al., 2018] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 9446-9454, 2018.","title":"Deep image prior","authors":"Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu et al., 2021] Haiyan Wu, Yanyun Qu, Shaohui Lin, Jian Zhou, Ruizhi Qiao, Zhizhong Zhang, Yuan Xie, and Lizhuang Ma. Contrastive learning for compact single im- age dehazing. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 10551-10560, 2021.","title":"Contrastive learning for compact single image dehazing","authors":"Haiyan Wu, Yanyun Qu, Shaohui Lin, Jian Zhou, Ruizhi Qiao, Zhizhong Zhang, Yuan Xie, Lizhuang Ma","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yuanjie et al., 2020] Shao Yuanjie, Li Lerenhan, Ren Wenqi, Gao Changxin, and Sang Nong. Domain adapta- tion for image dehazing. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, 2020.","title":"Domain adaptation for image dehazing","authors":"Li Shao Yuanjie, Ren Lerenhan, Gao Wenqi, Sang Changxin, Nong","year":"2020","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zheng et al., 2021] Zhuoran Zheng, Wenqi Ren, Xiaochun Cao, Xiaobin Hu, Tao Wang, Fenglong Song, and Xi- uyi Jia. Ultra-high-definition image dehazing via multi- guided bilateral learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16185-16194, 2021.","title":"Ultra-high-definition image dehazing via multiguided bilateral learning","authors":"Zheng","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}