{"type":"ScholarlyArticle","title":"Hybrid Frequency Modulation Network for Image Restoration","authors":[{"name":"Yuning Cui","slug":"yuning-cui","given":"Yuning","family":"Cui","affiliation":""},{"name":"Mingyu Liu","slug":"mingyu-liu","given":"Mingyu","family":"Liu","affiliation":""},{"name":"Wenqi Ren","slug":"wenqi-ren","given":"Wenqi","family":"Ren","affiliation":""},{"name":"Alois Knoll","slug":"alois-knoll","given":"Alois","family":"Knoll","affiliation":""}],"abstract":"Image restoration involves recovering a high-quality image from its corrupted counterpart. This paper presents an effective and efficient framework for image restoration, termed CSNet, based on ``channel + spatial\" hybrid frequency modulation. Different feature channels include different degradation patterns and degrees, however, most current networks ignore the importance of channel interactions. To alleviate this issue, we propose a frequency-based channel feature modulation module to facilitate channel interactions through the channel-dimension Fourier transform. Furthermore, based on our observations, we develop a multi-scale frequency-based spatial feature modulation module to refine the direct-current component of features using extremely lightweight learnable parameters. This module contains a densely connected coarse-to-fine learning paradigm for enhancing multi-scale representation learning. In addition, we introduce a frequency-inspired loss function to achieve omni-frequency learning. Extensive experiments on nine datasets demonstrate that the proposed network achieves state-of-the-art performance for three image restoration tasks, including image dehazing, image defocus deblurring, and image desnowing. The code and models are available at https://github.com/c-yn/CSNet.","doi":"10.24963/ijcai.2024/80","url":"https://ijcai.org/proceedings/2024/80","pdf":"https://ijcai.org/proceedings/2024/0080.pdf","markdown":"https://ijcai.org/proceedings/2024/80.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/80","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":80,"pages":{"first":722,"last":730},"keywords":["Computer Vision: CV: Applications","Computer Vision: CV: Representation learning","Computer Vision: CV: Computational photography"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}