{"type":"ScholarlyArticle","title":"SpanConv: A New Convolution via Spanning Kernel Space for Lightweight Pansharpening","authors":[{"name":"Zhi-Xuan Chen","slug":"zhi-xuan-chen","given":"Zhi-Xuan","family":"Chen","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"},{"name":"Cheng Jin","slug":"cheng-jin","given":"Cheng","family":"Jin","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"},{"name":"Tian-Jing Zhang","slug":"tian-jing-zhang","given":"Tian-Jing","family":"Zhang","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"},{"name":"Xiao Wu","slug":"xiao-wu","given":"Xiao","family":"Wu","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"},{"name":"Liang-Jian Deng","slug":"liang-jian-deng","given":"Liang-Jian","family":"Deng","affiliation":"University of Electronic Science and Technology of China , Chengdu , 611731"}],"abstract":"Standard convolution operations can effectively perform feature extraction and representation but result in high computational cost, largely due to the generation of the original convolution kernel corresponding to the channel dimension of the feature map, which will cause unnecessary redundancy. In this paper, we focus on kernel generation and present an interpretable span strategy, named SpanConv, for the effective construction of kernel space. Specifically, we first learn two navigated kernels with single channel as bases, then extend the two kernels by learnable coefficients, and finally span the two sets of kernels by their linear combination to construct the so-called SpanKernel. The proposed SpanConv is realized by replacing plain convolution kernel by SpanKernel. To verify the effectiveness of SpanConv, we design a simple network with SpanConv. Experiments demonstrate the proposed network significantly reduces parameters comparing with benchmark networks for remote sensing pansharpening, while achieving competitive performance and excellent generalization. Code is available at https://github.com/zhi-xuan-chen/IJCAI-2022 SpanConv.","doi":"10.24963/ijcai.2022/118","url":"https://ijcai.org/proceedings/2022/118","pdf":"https://ijcai.org/proceedings/2022/0118.pdf","markdown":"https://ijcai.org/proceedings/2022/118.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/118","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":118,"pages":{"first":841,"last":847},"keywords":["Computer Vision: Machine Learning for Vision","Machine Learning: Convolutional Networks","Computer Vision: Computational photography","Machine Learning: Applications"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987008","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985151","length_sec":641,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Aiazzi et al., 2002] Bruno Aiazzi, Luciano Alparone, Ste- fano Baronti, and Andrea Garzelli. Context-driven fu- sion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Trans- actions on geoscience and remote sensing, 40(10):2300- 2312, 2002.","title":"Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis","authors":"Bruno Aiazzi, Luciano Alparone, Stefano Baronti, Andrea Garzelli","year":"2002","venue":"IEEE Transactions on geoscience and remote sensing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alparone et al., 2004] Luciano Alparone, Stefano Baronti, Andrea Garzelli, and Filippo Nencini. A global qual- ity measurement of pan-sharpened multispectral imagery. IEEE Geoscience and Remote Sensing Letters, 1(4):313- 317, 2004.","title":"A global quality measurement of pan-sharpened multispectral imagery","authors":"Luciano Alparone, Stefano Baronti, Andrea Garzelli, Filippo Nencini","year":"2004","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Alparone et al., 2008] Luciano Alparone, Bruno Aiazzi, Stefano Baronti, Andrea Garzelli, Filippo Nencini, and Massimo Selva. Multispectral and panchromatic data fu- sion assessment without reference. Photogrammetric En- gineering & Remote Sensing, 74(2):193-200, 2008.","title":"Multispectral and panchromatic data fusion assessment without reference","authors":"Luciano Alparone, Bruno Aiazzi, Stefano Baronti, Andrea Garzelli, Filippo Nencini, Massimo Selva","year":"2008","venue":"Photogrammetric Engineering & Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boardman, 1993] Joseph Boardman. Automating spectral unmixing of aviris data using convex geometry concepts. In Summaries of the 4th Annual JPL Airborne Geoscience Workshop, volume 1, pages 11-14, 1993.","title":"Automating spectral unmixing of aviris data using convex geometry concepts","authors":"Joseph Boardman","year":"1993","venue":"Summaries of the 4th Annual JPL Airborne Geoscience Workshop","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2021] Liang-Jian Deng, Gemine Vivone, Cheng Jin, and Jocelyn Chanussot. Detail injection-based deep convolutional neural networks for pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 59(8):6995-7010, 2021.","title":"Detail injection-based deep convolutional neural networks for pansharpening","authors":"Liang-Jian Deng, Gemine Vivone, Jin Cheng, Jocelyn Chanussot","year":"2021","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Garzelli and Nencini, 2009] Andrea Garzelli and Fil- ippo Nencini. Hypercomplex quality assessment of multi/hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 6(4):662-665, 2009.","title":"Andrea Garzelli and Filippo Nencini. Hypercomplex quality assessment of multi/hyperspectral images","authors":"Nencini Garzelli","year":"2009","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ghazali et al., 2011] Rozaida Ghazali, Abir Jaafar Hussain, and Panos Liatsis. Dynamic ridge polynomial neural net- work: Forecasting the univariate non-stationary and sta- tionary trading signals. Expert Systems with Applications, 38(4):3765-3776, 2011.","title":"Dynamic ridge polynomial neural network: Forecasting the univariate non-stationary and stationary trading signals","authors":"Ghazali","year":"2011","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Haase and Amthor, 2020] Daniel Haase and Manuel Amthor. Rethinking depthwise separable convolutions: How intra-kernel correlations lead to improved Mo- bileNets. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.","title":"Rethinking depthwise separable convolutions: How intra-kernel correlations lead to improved Mo-bileNets","authors":"Amthor ; ] Daniel Haase, Manuel Haase, Amthor","year":"2020","venue":"The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2019] Lin He, Yizhou Rao, Jun Li, Jocelyn Chanussot, Antonio Plaza, Jiawei Zhu, and Bo Li. Pan- sharpening via detail injection based convolutional neu- ral networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(4):1188- 1204, 2019.","title":"Pansharpening via detail injection based convolutional neural networks","authors":"Lin He, Yizhou Rao, Jun Li, Jocelyn Chanussot, Antonio Plaza, Jiawei Zhu, Bo Li","year":"2019","venue":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Howard et al., 2017] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mo- bilenets: Efficient convolutional neural networks for mo- bile vision applications. arXiv preprint arXiv:1704.04861, 2017.","title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications","authors":"Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jin et al., 2022] Cheng Jin, Liang-Jian Deng, Ting-Zhu Huang, and Gemine Vivone. Laplacian pyramid networks: A new approach for multispectral pansharpening. Infor- mation Fusion, 78:158-170, 2022.","title":"Laplacian pyramid networks: A new approach for multispectral pansharpening","authors":"Jin Cheng, Liang-Jian Deng, Ting-Zhu Huang, Gemine Vivone","year":"2022","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural infor- mation processing systems, 25:1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Krizhevsky","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lolli et al., 2017] Simone Lolli, Luciano Alparone, An- drea Garzelli, and Gemine Vivone. Haze correction for contrast-based multispectral pansharpening. IEEE Geo- science and Remote Sensing Letters, 14(12):2255-2259, 2017.","title":"Haze correction for contrast-based multispectral pansharpening","authors":"Simone Lolli, Luciano Alparone, Andrea Garzelli, Gemine Vivone","year":"2017","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Masi et al., 2016] Giuseppe Masi, Davide Cozzolino, Luisa Verdoliva, and Giuseppe Scarpa. Pansharpening by convo- lutional neural networks. Remote Sensing, 8(7):594, 2016.","title":"Pansharpening by convolutional neural networks","authors":"Giuseppe Masi, Davide Cozzolino, Luisa Verdoliva, Giuseppe Scarpa","year":"2016","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Meyer, 2000] C. D. Meyer. Matrix Analysis and Applied Linear Algebra Book and Solutions Manual. Matrix Anal- ysis and Applied Linear Algebra Book and Solutions Man- ual, 2000.","title":"Matrix Analysis and Applied Linear Algebra Book and Solutions Manual","authors":"C D Meyer","year":"2000","venue":"Matrix Analysis and Applied Linear Algebra Book and Solutions Manual","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[T.-J. Zhang and Vivone, 2022] T.-Z. Huang J. Chanussot T.- J. Zhang, L.-J. Deng and G. Vivone. A triple-double con- volutional neural network for panchromatic sharpening. IEEE Transactions on Neural Networks and Learning Sys- tems, 2022.","title":"A triple-double convolutional neural network for panchromatic sharpening","authors":"T.-Z Huang, J Chanussot, T.-J Zhang, L.-J Deng, G Vivone","year":"2022","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vivone et al., 2018] Gemine Vivone, Rocco Restaino, and Jocelyn Chanussot. Full scale regression-based injection coefficients for panchromatic sharpening. IEEE Transac- tions on Image Processing, 27(7):3418-3431, 2018.","title":"Gemine Vivone, Rocco Restaino, and Jocelyn Chanussot. Full scale regression-based injection coefficients for panchromatic sharpening","authors":"Vivone","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vivone et al., 2020] Gemine Vivone, Mauro Dalla Mura, Andrea Garzelli, Rocco Restaino, Giuseppe Scarpa, Mag- nus Orn Ulfarsson, Luciano Alparone, and Jocelyn Chanussot. A new benchmark based on recent advances in multispectral pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods. IEEE Geoscience and Remote Sensing Magazine, 2020.","title":"A new benchmark based on recent advances in multispectral pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods","authors":"Vivone","year":"2020","venue":"IEEE Geoscience and Remote Sensing Magazine","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vivone, 2019] Gemine Vivone. Robust band-dependent spatial-detail approaches for panchromatic sharpening. IEEE transactions on Geoscience and Remote Sensing, 57(9):6421-6433, 2019.","title":"Gemine Vivone. Robust band-dependent spatial-detail approaches for panchromatic sharpening","authors":"Vivone","year":"2019","venue":"IEEE transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wald, 2002] Lucien Wald. Data fusion: definitions and ar- chitectures: fusion of images of different spatial resolu- tions. Presses des MINES, 2002.","title":"Data fusion: definitions and architectures: fusion of images of different spatial resolutions","authors":"Wald ;, Lucien Wald","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wei et al., 2017] Yancong Wei, Qiangqiang Yuan, Xi- angchao Meng, Huanfeng Shen, Liangpei Zhang, and Michael Ng. Multi-scale-and-depth convolutional neural network for remote sensed imagery pan-sharpening. In 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pages 3413-3416. IEEE, 2017.","title":"Multi-scale-and-depth convolutional neural network for remote sensed imagery pan-sharpening","authors":"Wei","year":"2017","venue":"IEEE International Geoscience and Remote Sensing Symposium (IGARSS)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2017] Junfeng Yang, Xueyang Fu, Yuwen Hu, Yue Huang, Xinghao Ding, and John Paisley. Pannet: A deep network architecture for pan-sharpening. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 5449-5457, 2017.","title":"Pannet: A deep network architecture for pan-sharpening","authors":"Junfeng Yang, Xueyang Fu, Yuwen Hu, Yue Huang, Xinghao Ding, John Paisley","year":"2017","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2019] Yongjun Zhang, Chi Liu, Mingwei Sun, and Yangjun Ou. Pan-sharpening using an efficient bidi- rectional pyramid network. IEEE Transactions on Geo- science and Remote Sensing, 57(8):5549-5563, 2019.","title":"Pan-sharpening using an efficient bidirectional pyramid network","authors":"Yongjun Zhang, Chi Liu, Mingwei Sun, Yangjun Ou","year":"2019","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}