{"type":"ScholarlyArticle","title":"Source-Adaptive Discriminative Kernels based Network for Remote Sensing Pansharpening","authors":[{"name":"Siran Peng","slug":"siran-peng","given":"Siran","family":"Peng","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"},{"name":"Liang-Jian Deng","slug":"liang-jian-deng","given":"Liang-Jian","family":"Deng","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"},{"name":"Jin-Fan Hu","slug":"jin-fan-hu","given":"Jin-Fan","family":"Hu","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"},{"name":"Yuwei Zhuo","slug":"yuwei-zhuo","given":"Yuwei","family":"Zhuo","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"}],"abstract":"For the pansharpening problem, previous convolutional neural networks (CNNs) mainly concatenate high-resolution panchromatic (PAN) images and low-resolution multispectral (LR-MS) images in their architectures, which ignores the distinctive attributes of different sources. In this paper, we propose a convolution network with source-adaptive discriminative kernels, called ADKNet, for the pansharpening task. Those kernels consist of spatial kernels generated from PAN images containing rich spatial details and spectral kernels generated from LR-MS images containing abundant spectral information. The kernel generating process is specially designed to extract information discriminately and effectively. Furthermore, the kernels are learned in a pixel-by-pixel manner to characterize different information in distinct areas. Extensive experimental results indicate that ADKNet outperforms current state-of-the-art (SOTA) pansharpening methods in both quantitative and qualitative assessments, in the meanwhile only with about 60,000 network parameters. Also, the proposed network is extended to the hyperspectral image super-resolution (HSISR) problem, still yields SOTA performance, proving the universality of our model. The code is available at http://github.com/liangjiandeng/ADKNet.","doi":"10.24963/ijcai.2022/179","url":"https://ijcai.org/proceedings/2022/179","pdf":"https://ijcai.org/proceedings/2022/0179.pdf","markdown":"https://ijcai.org/proceedings/2022/179.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/179","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":179,"pages":{"first":1283,"last":1289},"keywords":["Machine Learning: Applications","Constraint Satisfaction and Optimization: Applications","Computer Vision: Applications","Computer Vision: Machine Learning for Vision","Machine Learning: Hyperparameter Optimization"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986809","length_sec":80,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984952","length_sec":340,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Alparone et al., 2007] Luciano Alparone, Lucien Wald, Jo- celyn Chanussot, Claire Thomas, Paolo Gamba, and Lori Mann Bruce. Comparison of pansharpening algo- rithms: Outcome of the 2006 grs-s data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10):3012-3021, 2007.","title":"Comparison of pansharpening algorithms: Outcome of the 2006 grs-s data-fusion contest","authors":"Luciano Alparone, Lucien Wald, Jocelyn Chanussot, Claire Thomas, Paolo Gamba, Lori Mann Bruce","year":"2007","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Choi et al., 2010] Jaewan Choi, Kiyun Yu, and Yongil Kim. A new adaptive component-substitution-based satellite im- age fusion by using partial replacement. IEEE Transac- tions on Geoscience and Remote Sensing, 2010.","title":"A new adaptive component-substitution-based satellite image fusion by using partial replacement","authors":"Choi","year":"2010","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2021] Liang-Jian Deng, Gemine Vivone, Cheng Jin, and Jocelyn Chanussot. Detail injection-based deep convolutional neural networks for pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 59(8):6995-7010, 2021.","title":"Detail injection-based deep convolutional neural networks for pansharpening","authors":"Liang-Jian Deng, Gemine Vivone, Jin Cheng, Jocelyn Chanussot","year":"2021","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fu et al., 2021] Xueyang Fu, Wu Wang, Yue Huang, Xing- hao Ding, and John Paisley. Deep multiscale detail net- works for multiband spectral image sharpening. IEEE Transactions on Neural Networks and Learning Systems, 32(5):2090-2104, 2021.","title":"Deep multiscale detail networks for multiband spectral image sharpening","authors":"Xueyang Fu, Wu Wang, Yue Huang, Xinghao Ding, John Paisley","year":"2021","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Garzelli and Nencini, 2009] Andrea Garzelli and Fil- ippo Nencini. Hypercomplex quality assessment of multi/hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 6(4):662-665, 2009.","title":"Andrea Garzelli and Filippo Nencini. Hypercomplex quality assessment of multi/hyperspectral images","authors":"Nencini Garzelli","year":"2009","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Garzelli et al., 2008] Andrea Garzelli, Filippo Nencini, and Luca Capobianco. Optimal mmse pan sharpening of very high resolution multispectral images. IEEE Transactions on Geoscience and Remote Sensing, 46(1):228-236, 2008.","title":"Optimal mmse pan sharpening of very high resolution multispectral images","authors":"Andrea Garzelli, Filippo Nencini, Luca Capobianco","year":"2008","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Giuseppe et al., 2016] Masi Giuseppe, Cozzolino Davide, Verdoliva Luisa, and Scarpa Giuseppe. Pansharpening by convolutional neural networks. Remote Sensing, 8(7):594, 2016.","title":"Pansharpening by convolutional neural networks","authors":"Masi Giuseppe, Cozzolino Davide, Luisa Verdoliva, Scarpa Giuseppe","year":"2016","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2019] Lin He, Yizhou Rao, Jun Li, Jocelyn Chanussot, Antonio Plaza, Jiawei Zhu, and Bo Li. Pan- sharpening via detail injection based convolutional neu- ral networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(4):1188- 1204, 2019.","title":"Pansharpening via detail injection based convolutional neural networks","authors":"Lin He, Yizhou Rao, Jun Li, Jocelyn Chanussot, Antonio Plaza, Jiawei Zhu, Bo Li","year":"2019","venue":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jia et al., 2016] Xu Jia, Brabandere De Brabandere, Tinne Tuytelaars, and Luc Van Gool. Dynamic filter net- works. Advances in neural information processing sys- tems, 29:667-675, 2016.","title":"Dynamic filter networks","authors":"Xu Jia, Brabandere De Brabandere, Tinne Tuytelaars, Luc Van Gool","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Laben and Brower, 2000] Craig A. Laben and Bernard V. Brower. Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening, 2000.","title":"Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening","authors":"Craig A Laben, Bernard V Brower","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu, 2000] Jianguo Liu. Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details. International Journal of Re- mote Sensing, 21(18):3461-3472, 2000.","title":"Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details","authors":"Jianguo Liu","year":"2000","venue":"International Journal of Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Meng et al., 2018] Xiangchao Meng, Huanfeng Shen, Huifang Li, Liangpei Zhang, and Randi Fu. Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges. Information Fusion, 46:102-113, 2018.","title":"Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges","authors":"Meng","year":"2018","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Su et al., 2019] Hang Su, Varun Jampani, Deqing Sun, Orazio Gallo, Erik Learned-Miller, and Jan Kautz. Pixel-adaptive convolutional neural networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11158-11167, 2019.","title":"Pixel-adaptive convolutional neural networks","authors":"Su","year":"2019","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Vivone et al., 2014] Gemine Vivone, Rocco Restaino, Mauro Dalla Mura, Giorgio Licciardi, and Jocelyn Chanussot. Contrast and error-based fusion schemes for multispectral image pansharpening. IEEE Geoscience and Remote Sensing Letters, 11(5):930-934, 2014.","title":"Contrast and error-based fusion schemes for multispectral image pansharpening","authors":"Vivone","year":"2014","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Vivone et al., 2015] Gemine Vivone, Luciano Alparone, Jo- celyn Chanussot, Mauro Dalla Mura, Andrea Garzelli, Giorgio A. Licciardi, Rocco Restaino, and Lucien Wald. A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5):2565-2586, 2015.","title":"A critical comparison among pansharpening algorithms","authors":"Vivone","year":"2015","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Vivone, 2019] Gemine Vivone. Robust band-dependent spatial-detail approaches for panchromatic sharpening. IEEE Transactions on Geoscience and Remote Sensing, 57(9):6421-6433, 2019.","title":"Gemine Vivone. Robust band-dependent spatial-detail approaches for panchromatic sharpening","authors":"Vivone","year":"2019","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wald et al., 1997] Lucien Wald, Thierry Ranchin, and Marc Mangolini. Fusion of satellite images of different spa- tial resolutions: Assessing the quality of resulting im- ages. Photogrammetric Engineering and Remote Sensing, 63:691-699, 1997.","title":"Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images","authors":"Wald","year":"1997","venue":"Photogrammetric Engineering and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wald, 2002] Lucien Wald. Data fusion. definitions and ar- chitectures -fusion of images of different spatial resolu- tions. Presses des MINES, 2002.","title":"Data fusion. definitions and architectures -fusion of images of different spatial resolutions","authors":"Wald ;, Lucien Wald","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang and Bovik, 2002] Zhou Wang and Alan C. Bovik. A universal image quality index. IEEE Signal Processing Letters, 9(3):81-84, 2002.","title":"A universal image quality index","authors":"Zhou Wang, Alan C Bovik","year":"2002","venue":"IEEE Signal Processing Letters","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2018] Jialin Wu, Dai Li, Yu Yang, Chandrajit Ba- jaj, and Xiangyang Ji. Dynamic filtering with large sam- pling field for convnets. In European Conference on Com- puter Vision (ECCV), 2018.","title":"Dynamic filtering with large sampling field for convnets","authors":"Jialin Wu, Dai Li, Yu Yang, Chandrajit Bajaj, Xiangyang Ji","year":"2018","venue":"European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yang et al., 2017] Junfeng Yang, Xueyang Fu, Yuwen Hu, Yue Huang, Xinghao Ding, and John Paisley. Pan- net: A deep network architecture for pan-sharpening. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 1753-1761, 2017.","title":"Pannet: A deep network architecture for pan-sharpening","authors":"Junfeng Yang, Xueyang Fu, Yuwen Hu, Yue Huang, Xinghao Ding, John Paisley","year":"2017","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2019] Yongjun Zhang, Chi Liu, Mingwei Sun, and Yangjun Ou. Pan-sharpening using an efficient bidi- rectional pyramid network. IEEE Transactions on Geo- science and Remote Sensing, 57(8):5549-5563, 2019.","title":"Pan-sharpening using an efficient bidirectional pyramid network","authors":"Yongjun Zhang, Chi Liu, Mingwei Sun, Yangjun Ou","year":"2019","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhou et al., 1998] Jie Zhou, Daniel L. Civco, and John A. Silander. A wavelet transform method to merge landsat tm and spot panchromatic data. International Journal of Remote Sensing, 19(4):743-757, 1998.","title":"A wavelet transform method to merge landsat tm and spot panchromatic data","authors":"Jie Zhou, Daniel L Civco, John A Silander","year":"1998","venue":"International Journal of Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2021] Jingkai Zhou, Varun Jampani, Zhixiong Pi, Qiong Liu, and Ming-Hsuan Yang. Decoupled dy- namic filter networks. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6643-6652, 2021.","title":"Decoupled dynamic filter networks","authors":"Zhou","year":"2021","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}