{"type":"ScholarlyArticle","title":"On Efficient Transformer-Based Image Pre-training for Low-Level Vision","authors":[{"name":"Wenbo Li","slug":"wenbo-li","given":"Wenbo","family":"Li","affiliation":"The Chinese University of Hong Kong"},{"name":"Xin Lu","slug":"xin-lu","given":"Xin","family":"Lu","affiliation":"Deeproute ."},{"name":"Shengju Qian","slug":"shengju-qian","given":"Shengju","family":"Qian","affiliation":"The Chinese University of Hong Kong"},{"name":"Jiangbo Lu","slug":"jiangbo-lu","given":"Jiangbo","family":"Lu","affiliation":"SmartMore Corporation"}],"abstract":"Pre-training has marked numerous state of the arts in high-level computer vision, while few attempts have ever been made to investigate how pre-training acts in image processing systems. In this paper, we tailor transformer-based pre-training regimes that boost various low-level tasks. To comprehensively diagnose the influence of pre-training, we design a whole set of principled evaluation tools that uncover its effects on internal representations. The observations demonstrate that pre-training plays strikingly different roles in low-level tasks. For example, pre-training introduces more local information to intermediate layers in super-resolution (SR), yielding significant performance gains, while pre-training hardly affects internal feature representations in denoising, resulting in limited gains. Further, we explore different methods of pre-training, revealing that multi-related-task pre-training is more effective and data-efficient than other alternatives. Finally, we extend our study to varying data scales and model sizes, as well as comparisons between transformers and CNNs. Based on the study, we successfully develop state-of-the-art models for multiple low-level tasks.","doi":"10.24963/ijcai.2023/121","url":"https://ijcai.org/proceedings/2023/121","pdf":"https://ijcai.org/proceedings/2023/0121.pdf","markdown":"https://ijcai.org/proceedings/2023/121.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/121","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":121,"pages":{"first":1089,"last":1097},"keywords":["Computer Vision: CV: Computational photography","Computer Vision: CV: Applications","Computer Vision: CV: Representation learning"],"videos":[],"references":[{"position":1,"raw":"[Agustsson and Timofte, 2017] Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super- resolution: Dataset and study. In CVPRW, pages 126-135, 2017.","title":"Ntire 2017 challenge on single image superresolution: Dataset and study","authors":"Eirikur Agustsson, Radu Timofte","year":"2017","venue":"CVPRW","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brown et al., 2020] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020.","title":"Language models are few-shot learners","authors":"Benjamin Tom B Brown, Nick Mann, Melanie Ryder, Jared Subbiah, Prafulla Kaplan, Arvind Dhariwal, Pranav Neelakantan, Girish Shyam, Amanda Sastry, Askell","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convo- lutional nets, atrous convolution, and fully connected crfs. PAMI, 40(4):834-848, 2017.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2017","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. Pre-trained image processing transformer. In CVPR, pages 12299-12310, 2021.","title":"Pre-trained image processing transformer","authors":"Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, Wen Gao","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022] X Chen, X Wang, J Zhou, and C Dong. Activating more pixels in image super-resolution trans- former. arxiv 2022. arXiv preprint arXiv:2205.04437, 2022.","title":"Activating more pixels in image super-resolution transformer","authors":"X Chen, J Wang, C Zhou, Dong","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cortes et al., 2012] Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Algorithms for learning kernels based on centered alignment. The Journal of Machine Learning Research, 13(1):795-828, 2012.","title":"Algorithms for learning kernels based on centered alignment","authors":"Corinna Cortes, Mehryar Mohri, Afshin Rostamizadeh","year":"2012","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dabov et al., 2007] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. TIP, 16(8):2080-2095, 2007.","title":"Image denoising by sparse 3-d transform-domain collaborative filtering","authors":"Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, Karen Egiazarian","year":"2007","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dai et al., 2019] Tao Dai, Jianrui Cai, Yongbing Zhang, Shu-Tao Xia, and Lei Zhang. Second-order attention net- work for single image super-resolution. In CVPR, pages 11065-11074, 2019.","title":"Second-order attention network for single image super-resolution","authors":"Tao Dai, Jianrui Cai, Yongbing Zhang, Shu-Tao Xia, Lei Zhang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":null,"venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fu et al., 2017a] Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley. Clearing the skies: A deep network architecture for single-image rain removal. TIP, 26(6):2944-2956, 2017.","title":"Clearing the skies: A deep network architecture for single-image rain removal","authors":"Fu","year":"2017","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Fu et al., 2017b] Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. Remov- ing rain from single images via a deep detail network. In CVPR, pages 3855-3863, 2017.","title":"Removing rain from single images via a deep detail network","authors":"Fu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Girshick et al., 2014] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, pages 580-587, 2014.","title":"Rich feature hierarchies for accurate object detection and semantic segmentation","authors":"Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Girshick, 2015] Ross Girshick. Fast r-cnn. In ICCV, pages 1440-1448, 2015.","title":"Fast r-cnn","authors":"Ross Girshick","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gretton et al., 2007] Arthur Gretton, Kenji Fukumizu, Choon Hui Teo, Le Song, Bernhard Schölkopf, Alexan- der J Smola, et al. A kernel statistical test of independence. In Nips, volume 20, pages 585-592.","title":"A kernel statistical test of independence","authors":"Arthur Gretton, Kenji Fukumizu, Choon Hui Teo, Le Song, Bernhard Schölkopf, Alexander J Smola","year":null,"venue":"Nips","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gu et al., 2017] Shuhang Gu, Deyu Meng, Wangmeng Zuo, and Lei Zhang. Joint convolutional analysis and synthesis sparse representation for single image layer separation. In ICCV, pages 1708-1716, 2017.","title":"Joint convolutional analysis and synthesis sparse representation for single image layer separation","authors":"Gu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[He et al., 2021] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. arXiv preprint arXiv:2111.06377, 2021.","title":"Masked autoencoders are scalable vision learners","authors":"Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Huang et al., 2015] Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja. Single image super-resolution from transformed self-exemplars. In CVPR, pages 5197-5206, 2015.","title":"Single image super-resolution from transformed self-exemplars","authors":"Jia-Bin Huang, Abhishek Singh, Narendra Ahuja","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kolesnikov et al., 2020] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. In ECCV, pages 491-507. Springer, 2020.","title":"Big transfer (bit): General visual representation learning","authors":"Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, Neil Houlsby","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kornblith et al., 2019] Simon Kornblith, Jonathon Shlens, and Quoc V Le. Do better imagenet models transfer bet- ter? In CVPR, pages 2661-2671, 2019.","title":"Do better imagenet models transfer better?","authors":"Simon Kornblith, Jonathon Shlens, Quoc V Le","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[LeCun et al., 1989] Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation ap- plied to handwritten zip code recognition. Neural compu- tation, 1(4):541-551, 1989.","title":"Backpropagation applied to handwritten zip code recognition","authors":"Yann Lecun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, Lawrence D Jackel","year":"1989","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Li et al., 2016] Yu Li, Robby T Tan, Xiaojie Guo, Jiangbo Lu, and Michael S Brown. Rain streak removal using layer priors. In CVPR, pages 2736-2744, 2016.","title":"Rain streak removal using layer priors","authors":"Yu Li, Robby T Tan, Xiaojie Guo, Jiangbo Lu, Michael S Brown","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li et al., 2018] Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In ECCV, pages 254-269, 2018.","title":"Recurrent squeeze-and-excitation context aggregation net for single image deraining","authors":"Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, Hongbin Zha","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liang et al., 2021] Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In ICCVW, pages 1833-1844, 2021.","title":"Swinir: Image restoration using swin transformer","authors":"Liang","year":"2021","venue":"ICCVW","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Liu et al., 2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. ICCV, 2021.","title":"Swin transformer: Hierarchical vision transformer using shifted windows","authors":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Liu et al., 2022] Lin Liu, Lingxi Xie, Xiaopeng Zhang, Shanxin Yuan, Xiangyu Chen, Wengang Zhou, Houqiang Li, and Qi Tian. Tape: Task-agnostic prior embedding for image restoration. In ECCV, pages 447-464. Springer, 2022.","title":"Tape: Task-agnostic prior embedding for image restoration","authors":"Lin Liu, Lingxi Xie, Xiaopeng Zhang, Shanxin Yuan, Xiangyu Chen, Wengang Zhou, Houqiang Li, Qi Tian","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Luo et al., 2015] Yu Luo, Yong Xu, and Hui Ji. Removing rain from a single image via discriminative sparse coding. In ICCV, pages 3397-3405, 2015.","title":"Removing rain from a single image via discriminative sparse coding","authors":"Yu Luo, Yong Xu, Hui Ji","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Mahajan et al., 2018] Dhruv Mahajan, Ross Girshick, Vig- nesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. Ex- ploring the limits of weakly supervised pretraining. In ECCV, pages 181-196, 2018.","title":"Exploring the limits of weakly supervised pretraining","authors":"Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, Laurens Van Der Maaten","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Mei et al., 2021] Yiqun Mei, Yuchen Fan, and Yuqian Zhou. Image super-resolution with non-local sparse attention. In CVPR, pages 3517-3526, 2021.","title":"Image super-resolution with non-local sparse attention","authors":"Yiqun Mei, Yuchen Fan, Yuqian Zhou","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Nguyen et al., 2020] Thao Nguyen, Maithra Raghu, and Si- mon Kornblith. Do wide and deep networks learn the same things? uncovering how neural network repre- sentations vary with width and depth. arXiv preprint arXiv:2010.15327, 2020.","title":"Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth","authors":"Thao Nguyen, Maithra Raghu, Simon Kornblith","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Radford et al., 2018] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. 2018.","title":"Improving language understanding by generative pre-training","authors":"Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Raghu et al., 2021] Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovit- skiy. Do vision transformers see like convolutional neural networks? arXiv preprint arXiv:2108.08810, 2021.","title":"Do vision transformers see like convolutional neural networks","authors":"Raghu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Ren et al., 2019] Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng. Progressive image de- raining networks: A better and simpler baseline. In CVPR, pages 3937-3946, 2019.","title":"Progressive image deraining networks: A better and simpler baseline","authors":"Ren","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Sun et al., 2017] Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable ef- fectiveness of data in deep learning era. In ICCV, pages 843-852, 2017.","title":"Revisiting unreasonable effectiveness of data in deep learning era","authors":"Chen Sun, Abhinav Shrivastava, Saurabh Singh, Abhinav Gupta","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Tian et al., 2020] Chunwei Tian, Yong Xu, and Wangmeng Zuo. Image denoising using deep cnn with batch renor- malization. Neural Networks, 121:461-473, 2020.","title":"Image denoising using deep cnn with batch renormalization","authors":"Tian","year":"2020","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wang et al., 2018] Xintao Wang, Ke Yu, Shixiang Wu, Jin- jin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution gener- ative adversarial networks. In ECCVW, pages 0-0, 2018.","title":"Esrgan: Enhanced super-resolution generative adversarial networks","authors":"Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, Chen Change Loy","year":"2018","venue":"ECCVW","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wang et al., 2019] Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson WH Lau. Spa- tial attentive single-image deraining with a high quality real rain dataset. In CVPR, pages 12270-12279, 2019.","title":"Spatial attentive single-image deraining with a high quality real rain dataset","authors":"Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, Rynson Wh Lau","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Wang et al., 2020] Hong Wang, Qi Xie, Qian Zhao, and Deyu Meng. A model-driven deep neural network for sin- gle image rain removal. In CVPR, pages 3103-3112, 2020.","title":"A model-driven deep neural network for single image rain removal","authors":"Wang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Wang et al., 2021a] Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A ver- satile backbone for dense prediction without convolutions. arXiv preprint arXiv:2102.12122, 2021.","title":"Pyramid vision transformer: A versatile backbone for dense prediction without convolutions","authors":"Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wang et al., 2021b] Zhendong Wang, Xiaodong Cun, Jian- min Bao, and Jianzhuang Liu. Uformer: A general u- shaped transformer for image restoration. arXiv preprint arXiv:2106.03106, 2021.","title":"Uformer: A general ushaped transformer for image restoration","authors":"Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wei et al., 2019] Wei Wei, Deyu Meng, Qian Zhao, Zong- ben Xu, and Ying Wu. Semi-supervised transfer learning for image rain removal. In CVPR, pages 3877-3886, 2019.","title":"Semi-supervised transfer learning for image rain removal","authors":"Wei Wei, Deyu Meng, Qian Zhao, Zongben Xu, Ying Wu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Xiao et al., 2021] Tete Xiao, Mannat Singh, Eric Mintun, Trevor Darrell, Piotr Dollár, and Ross Girshick. Early convolutions help transformers see better. arXiv preprint arXiv:2106.14881, 2021.","title":"Early convolutions help transformers see better","authors":"Tete Xiao, Mannat Singh, Eric Mintun, Trevor Darrell, Piotr Dollár, Ross Girshick","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Yang et al., 2019] Wenhan Yang, Robby T Tan, Jiashi Feng, Zongming Guo, Shuicheng Yan, and Jiaying Liu. Joint rain detection and removal from a single image with con- textualized deep networks. PAMI, 42(6):1377-1393, 2019.","title":"Joint rain detection and removal from a single image with contextualized deep networks","authors":"Yang","year":"2019","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Zamir et al., 2022] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, pages 5728- 5739, 2022.","title":"Restormer: Efficient transformer for high-resolution image restoration","authors":"Aditya Syed Waqas Zamir, Salman Arora, Munawar Khan, Fahad Hayat, Ming-Hsuan Shahbaz Khan, Yang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zhang et al., 2017] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian de- noiser: Residual learning of deep cnn for image denoising. TIP, 26(7):3142-3155, 2017.","title":"Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising","authors":"Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, Lei Zhang","year":"2017","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhang et al., 2018a] Kai Zhang, Wangmeng Zuo, and Lei Zhang. Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. TIP, 27(9):4608-4622, 2018.","title":"Ffdnet: Toward a fast and flexible solution for cnn-based image denoising","authors":"Zhang","year":"2018","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zhang et al., 2018b] Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. Image super- resolution using very deep residual channel attention net- works. In ECCV, pages 286-301, 2018.","title":"Image superresolution using very deep residual channel attention networks","authors":"Zhang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zhang et al., 2021] Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Luc Van Gool, and Radu Timofte. Plug-and- play image restoration with deep denoiser prior. PAMI, 2021.","title":"Plug-andplay image restoration with deep denoiser prior","authors":"Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Luc Van Gool, Radu Timofte","year":"2021","venue":"PAMI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}