{"type":"ScholarlyArticle","title":"Implicit Prompt Learning for Image Denoising","authors":[{"name":"Yao Lu","slug":"yao-lu","given":"Yao","family":"Lu","affiliation":"Department of Computer Science , Harbin Institute of Technology at Shenzhen , China"},{"name":"Bo Jiang","slug":"bo-jiang","given":"Bo","family":"Jiang","affiliation":"College of Mechanical and Electronic Engineering , Northwest A&F University , China"},{"name":"Guangming Lu","slug":"guangming-lu","given":"Guangming","family":"Lu","affiliation":"Department of Computer Science , Harbin Institute of Technology at Shenzhen , China"},{"name":"Bob Zhang","slug":"bob-zhang","given":"Bob","family":"Zhang","affiliation":"Department of Computer and Information Science , University of Macau , China"}],"abstract":"Recently, various deep denoising methods have been proposed to solve the insufficient feature problem in image denoising. These methods can be mainly classified into two categories: (1) Injecting learnable tensors into denoising backbone to supplement feature, which is effective to some extent but may cause serious over-fitting. (2) Using diverse natural images from large image datasets to synthesize noisy images and pre-train denoising models, which can bring model generalization but require large model size and expensive training costs. To address these issues, this paper proposes Implicit Prompt Learning for Image Denoising (IPLID) method to flexibly generate adaptive prompts without meticulously designing them. Specifically, we first introduce an efficient Linear Prompt (LP) block with ultra-few parameters to produce dynamic prompts for both different stages and samples in denoising procedure. We further propose an efficient Compact Feature Fusion (CFF) block to process previous multi-level prompted denoising feature to reconstruct the denoising images. Finally, to further efficiently and effectively produce satisfactory prompt and denoising performance, a Gradient Accumulation (GA) learning scheme is proposed. Experiments on multiple benchmarks showed that the proposed IPLID achieves competitive results with only 1 percent of pre-trained backbone parameters, outperforming classical denoising methods in both efficiency and quality of restored images.","doi":"10.24963/ijcai.2024/517","url":"https://ijcai.org/proceedings/2024/517","pdf":"https://ijcai.org/proceedings/2024/0517.pdf","markdown":"https://ijcai.org/proceedings/2024/517.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/517","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":517,"pages":{"first":4678,"last":4686},"keywords":["Machine Learning: ML: Knowledge-aided learning"],"videos":[],"references":[{"position":1,"raw":"[Abdelhamed et al., 2018] A. Abdelhamed, Stephen Lin, and M. S. Brown. A high-quality denoising dataset for smartphone cameras. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1692- 1700, 2018.","title":"A high-quality denoising dataset for smartphone cameras","authors":"A Abdelhamed, Stephen Lin, M S Brown","year":"2018","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2021] Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. Pre-trained image processing transformer. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12294-12305, 2021.","title":"Pre-trained image processing transformer","authors":"Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, Wen Gao","year":"2021","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fan et al., 2017] Xiangmin Fan, Wencan Luo, Muhsin Menekse, Diane Litman, and Jingtao Wang. Scaling re- flection prompts in large classrooms via mobile interfaces and natural language processing. In Proceedings of the 22nd International Conference on Intelligent User Inter- faces, pages 363-374, 2017.","title":"Scaling reflection prompts in large classrooms via mobile interfaces and natural language processing","authors":"Fan","year":"2017","venue":"Proceedings of the 22nd International Conference on Intelligent User Interfaces","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hendrycks and Gimpel, 2016] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016.","title":"Gaussian error linear units","authors":"Gimpel Hendrycks, Dan Hendrycks, Kevin Gimpel","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jiang et al., 2022a] Bo Jiang, Yao Lu, Jiahuan Wang, Guangming Lu, and David Zhang. Deep image denois- ing with adaptive priors. IEEE Transactions on Circuits and Systems for Video Technology, 32:5124-5136, 2022.","title":"Deep image denoising with adaptive priors","authors":"Jiang","year":"2022","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jiang et al., 2022b] Bo Jiang, Yao Lu, Jiahuan Wang, Guangming Lu, and David Zhang. Deep image denois- ing with adaptive priors. IEEE Transactions on Circuits and Systems for Video Technology, 2022.","title":"Deep image denoising with adaptive priors","authors":"Jiang","year":"2022","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kolesnikov et al., 2021] Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. 2021.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lester et al., 2021] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021.","title":"The power of scale for parameter-efficient prompt tuning","authors":"Brian Lester, Rami Al-Rfou, Noah Constant","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2021] Wenbo Li, Xin Lu, Jiangbo Lu, X. Zhang, and Jiaya Jia. On efficient transformer and image pre- training for low-level vision. ArXiv, abs/2112.10175, 2021.","title":"On efficient transformer and image pretraining for low-level vision","authors":"Wenbo Li, Xin Lu, Jiangbo Lu, X Zhang, Jiaya Jia","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liang et al., 2021] Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1833-1844, 2021.","title":"Swinir: Image restoration using swin transformer","authors":"Liang","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2018] Pengju Liu, Hongzhi Zhang, K. Zhang, Liang Lin, and Wangmeng Zuo. Multi-level wavelet- cnn for image restoration. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 886-88609, 2018.","title":"Multi-level waveletcnn for image restoration","authors":"Liu","year":"2018","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2023] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023.","title":"Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing","authors":"Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig","year":"2023","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Loshchilov and Hutter, 2017] Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. ArXiv, abs/1711.05101, 2017.","title":"Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam","authors":"Hutter Loshchilov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nam et al., 2016] Seonghyeon Nam, Youngbae Hwang, Ya- suyuki Matsushita, and Seon Joo Kim. A holistic ap- proach to cross-channel image noise modeling and its ap- plication to image denoising. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1683-1691, 2016.","title":"A holistic approach to cross-channel image noise modeling and its application to image denoising","authors":"Nam","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Su et al., 2021] Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi- Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, et al. On transferability of prompt tuning for natural language processing. arXiv preprint arXiv:2111.06719, 2021.","title":"On transferability of prompt tuning for natural language processing","authors":"Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2021] Zhendong Wang, Xiaodong Cun, Jian- min Bao, and Jianzhuang Liu. Uformer: A general u- shaped transformer for image restoration. arXiv preprint arXiv:2106.03106, 2021.","title":"Uformer: A general ushaped transformer for image restoration","authors":"Zhendong Wang, Xiaodong Cun, Jianmin Bao, Jianzhuang Liu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2023] Jiaqi Wang, Zhengliang Liu, Lin Zhao, Zihao Wu, Chong Ma, Sigang Yu, Haixing Dai, Qiushi Yang, Yiheng Liu, Songyao Zhang, et al. Review of large vision models and visual prompt engineering. arXiv preprint arXiv:2307.00855, 2023.","title":"Review of large vision models and visual prompt engineering","authors":"Jiaqi Wang, Zhengliang Liu, Lin Zhao, Zihao Wu, Chong Ma, Sigang Yu, Haixing Dai, Qiushi Yang, Yiheng Liu, Songyao Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2018] Jun Xu, Hui Li, Zhetong Liang, David C. Zhang, and Lei Zhang. Real-world noisy image denoising: A new benchmark. ArXiv, abs/1804.02603, 2018.","title":"Real-world noisy image denoising: A new benchmark","authors":"Jun Xu, Hui Li, Zhetong Liang, David C Zhang, Lei Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zamir et al., 2021] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. arXiv preprint arXiv:2111.09881, 2021.","title":"Restormer: Efficient transformer for high-resolution image restoration","authors":"Aditya Syed Waqas Zamir, Salman Arora, Munawar Khan, Fahad Hayat, Ming-Hsuan Shahbaz Khan, Yang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2017] K. Zhang, W. Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Resid- ual learning of deep cnn for image denoising. IEEE Trans- actions on Image Processing, 26:3142-3155, 2017.","title":"Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising","authors":"K Zhang, W Zuo, Yunjin Chen, Deyu Meng, Lei Zhang","year":"2017","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2018] K. Zhang, Wangmeng Zuo, and Lei Zhang. Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Transactions on Image Processing, 27:4608-4622, 2018.","title":"Ffdnet: Toward a fast and flexible solution for cnn-based image denoising","authors":"K Zhang, Wangmeng Zuo, Lei Zhang","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2022] K. Zhang, Yawei Li, Jingyun Liang, Jiezhang Cao, Yulun Zhang, Hao Tang, Radu Timofte, and Luc Van Gool. Practical blind denoising via swin-conv- unet and data synthesis. ArXiv, abs/2203.13278, 2022.","title":"Practical blind denoising via swin-convunet and data synthesis","authors":"K Zhang, Yawei Li, Jingyun Liang, Jiezhang Cao, Yulun Zhang, Hao Tang, Radu Timofte, Luc Van Gool","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2022] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337-2348, 2022.","title":"Learning to prompt for vision-language models","authors":"Zhou","year":"2022","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}