{"type":"ScholarlyArticle","title":"DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion","authors":[{"name":"Zixiang Zhao","slug":"zixiang-zhao","given":"Zixiang","family":"Zhao","affiliation":"School of Mathematics and Statistics , Xi'an Jiaotong University , China"},{"name":"Shuang Xu","slug":"shuang-xu","given":"Shuang","family":"Xu","affiliation":"School of Mathematics and Statistics , Xi'an Jiaotong University , China"},{"name":"Chunxia Zhang","slug":"chunxia-zhang","given":"Chunxia","family":"Zhang","affiliation":"School of Mathematics and Statistics , Xi'an Jiaotong University , China"},{"name":"Junmin Liu","slug":"junmin-liu","given":"Junmin","family":"Liu","affiliation":"School of Mathematics and Statistics , Xi'an Jiaotong University , China"},{"name":"Jiangshe Zhang","slug":"jiangshe-zhang","given":"Jiangshe","family":"Zhang","affiliation":"School of Mathematics and Statistics , Xi'an Jiaotong University , China"},{"name":"Pengfei Li","slug":"pengfei-li","given":"Pengfei","family":"Li","affiliation":"Hikvision , China"}],"abstract":"Infrared and visible image fusion, a hot topic in the field of image processing, aims at obtaining fused images keeping the advantages of source images. This paper proposes a novel auto-encoder (AE) based fusion network. The core idea is that the encoder decomposes an image into background and detail feature maps with low- and high-frequency information, respectively, and that the decoder recovers the original image. To this end, the loss function makes the background/detail feature maps of source images similar/dissimilar. In the test phase, background and detail feature maps are respectively merged via a fusion module, and the fused image is recovered by the decoder. Qualitative and quantitative results illustrate that our method can generate fusion images containing highlighted targets and abundant detail texture information with strong reproducibility and meanwhile surpass state-of-the-art (SOTA) approaches.","doi":"10.24963/ijcai.2020/135","url":"https://ijcai.org/proceedings/2020/135","pdf":"https://ijcai.org/proceedings/2020/0135.pdf","markdown":"https://ijcai.org/proceedings/2020/135.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/135","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":135,"pages":{"first":970,"last":976},"keywords":["Computer Vision: 2D and 3D Computer Vision","Computer Vision: Computational Photography, Photometry, Shape from X","Machine Learning: Deep Learning: Convolutional networks"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25154/file_25154.m3u8","length_sec":257,"provider":"hls"}],"references":[{"position":1,"raw":"[Bavirisetti and Dhuli, 2015] Durga Prasad Bavirisetti and Ravindra Dhuli. Fusion of infrared and visible sensor im- ages based on anisotropic diffusion and karhunen-loeve transform. IEEE Sensors Journal, 16(1):203-209, 2015.","title":"Durga Prasad Bavirisetti and Ravindra Dhuli. Fusion of infrared and visible sensor images based on anisotropic diffusion and karhunen-loeve transform","authors":"Dhuli Bavirisetti","year":"2015","venue":"IEEE Sensors Journal","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bavirisetti and Dhuli, 2016] Durga Prasad Bavirisetti and Ravindra Dhuli. Two-scale image fusion of visible and infrared images using saliency detection. Infrared Physics & Technology, 76:52-64, 2016.","title":"Durga Prasad Bavirisetti and Ravindra Dhuli. Two-scale image fusion of visible and infrared images using saliency detection","authors":"Dhuli Bavirisetti","year":"2016","venue":"Infrared Physics & Technology","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhatnagar and Liu, 2015] Gaurav Bhatnagar and Zheng Liu. A novel image fusion framework for night-vision naviga- tion and surveillance. Signal, Image and Video Processing, 9(1):165-175, 2015.","title":"A novel image fusion framework for night-vision navigation and surveillance","authors":"Gaurav Bhatnagar, Zheng Liu","year":"2015","venue":"Signal, Image and Video Processing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown and Süsstrunk, 2011] Matthew Brown and Sabine Süsstrunk. Multi-spectral sift for scene category recog- nition. In CVPR 2011, pages 177-184. IEEE, 2011.","title":"Multi-spectral sift for scene category recognition","authors":"Matthew Brown, Sabine Süsstrunk","year":"2011","venue":"CVPR 2011","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guo et al., 2017] Hanqi Guo, Yong Ma, Xiaoguang Mei, and Jiayi Ma. Infrared and visible image fusion based on total variation and augmented lagrangian. Journal of the Optical Society of America A, 34(11):1961-1968, 2017.","title":"Infrared and visible image fusion based on total variation and augmented lagrangian","authors":"Guo","year":"2017","venue":"Journal of the Optical Society of America A","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2017] Hai-Miao Hu, Jiawei Wu, Bo Li, Qiang Guo, and Jin Zheng. An adaptive fusion algorithm for visible and infrared videos based on entropy and the cumulative distri- bution of gray levels. IEEE Transactions on Multimedia, 19(12):2706-2719, 2017.","title":"An adaptive fusion algorithm for visible and infrared videos based on entropy and the cumulative distribution of gray levels","authors":"Hai-Miao Hu, Jiawei Wu, Bo Li, Qiang Guo, Jin Zheng","year":"2017","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Lahoud and Susstrunk, 2018] F. Lahoud and S. Susstrunk. Ar in vr: Simulating infrared augmented vision. In 2018 25th IEEE International Conference on Image Processing (ICIP), pages 3893-3897, Oct 2018.","title":"Ar in vr: Simulating infrared augmented vision","authors":"F Lahoud, S Susstrunk","year":"2018","venue":"25th IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lahoud and Süsstrunk, 2019] Fayez Lahoud and Sabine Süsstrunk. Fast and efficient zero-learning image fusion. arXiv preprint arXiv:1905.03590, 2019.","title":"Fast and efficient zero-learning image fusion","authors":"Fayez Lahoud, Sabine Süsstrunk","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li and Wu, 2018] Hui Li and Xiao-Jun Wu. Densefuse: A fusion approach to infrared and visible images. IEEE Trans- actions on Image Processing, 28(5):2614-2623, 2018.","title":"Densefuse: A fusion approach to infrared and visible images","authors":"Hui Li, Xiao-Jun Wu","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2011] Shutao Li, Bin Yang, and Jianwen Hu. Perfor- mance comparison of different multi-resolution transforms for image fusion. Information Fusion, 12(2):74-84, 2011.","title":"Performance comparison of different multi-resolution transforms for image fusion","authors":"Li","year":"2011","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2018] Hui Li, Xiao-Jun Wu, and Josef Kittler. In- frared and visible image fusion using a deep learning frame- work. In 2018 24th International Conference on Pattern Recognition (ICPR), pages 2705-2710. IEEE, 2018.","title":"Infrared and visible image fusion using a deep learning framework","authors":"Hui Li, Xiao-Jun Wu, Josef Kittler","year":"2018","venue":"24th International Conference on Pattern Recognition (ICPR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2016] Yu Liu, Xun Chen, Rabab K Ward, and Z Jane Wang. Image fusion with convolutional sparse rep- resentation. IEEE Signal Processing Letters, 23(12):1882- 1886, 2016.","title":"Image fusion with convolutional sparse representation","authors":"Yu Liu, Xun Chen, Rabab K Ward, Jane Wang","year":"2016","venue":"IEEE Signal Processing Letters","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ma et al., 2016] Jiayi Ma, Chen Chen, Chang Li, and Jun Huang. Infrared and visible image fusion via gradient trans- fer and total variation minimization. Information Fusion, 31:100-109, 2016.","title":"Infrared and visible image fusion via gradient transfer and total variation minimization","authors":"Jiayi Ma, Chen Chen, Chang Li, Jun Huang","year":"2016","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ma et al., 2019a] Jiayi Ma, Yong Ma, and Chang Li. In- frared and visible image fusion methods and applications: A survey. Information Fusion, 45:153-178, 2019.","title":"Infrared and visible image fusion methods and applications: A survey","authors":"Ma","year":"2019","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2019b] Jiayi Ma, Wei Yu, Pengwei Liang, Chang Li, and Junjun Jiang. Fusiongan: A generative adversarial network for infrared and visible image fusion. Information Fusion, 48:11-26, 2019.","title":"Fusiongan: A generative adversarial network for infrared and visible image fusion","authors":"Ma","year":"2019","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ma et al., 2020] Jiayi Ma, Pengwei Liang, Wei Yu, Chen Chen, Xiaojie Guo, Jia Wu, and Junjun Jiang. Infrared and visible image fusion via detail preserving adversarial learning. Information Fusion, 54:85-98, 2020.","title":"Infrared and visible image fusion via detail preserving adversarial learning","authors":"Jiayi Ma, Pengwei Liang, Wei Yu, Chen Chen, Xiaojie Guo, Jia Wu, Junjun Jiang","year":"2020","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mao et al., 2016] Xiaojiao Mao, Chunhua Shen, and Yu-Bin Yang. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connec- tions. In Advances in neural information processing sys- tems, pages 2802-2810, 2016.","title":"Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections","authors":"Mao","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Meher et al., 2019] Bikash Meher, Sanjay Agrawal, Rutu- parna Panda, and Ajith Abraham. A survey on region based image fusion methods. Information Fusion, 48:119-132, 2019.","title":"A survey on region based image fusion methods","authors":"Bikash Meher, Sanjay Agrawal, Rutuparna Panda, Ajith Abraham","year":"2019","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mirza and Osindero, 2014] Mehdi Mirza and Simon Osin- dero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.","title":"Conditional generative adversarial nets","authors":"Mehdi Mirza, Simon Osindero","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Patil and Mudengudi, 2011] Ujwala Patil and Uma Muden- gudi. Image fusion using hierarchical pca. In 2011 Interna- tional Conference on Image Information Processing, pages 1-6. IEEE, 2011.","title":"Image fusion using hierarchical pca","authors":"Ujwala Patil, Uma Mudengudi","year":"2011","venue":"2011 International Conference on Image Information Processing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Prabhakar et al., 2017] K Ram Prabhakar, V Sai Srikar, and R Venkatesh Babu. Deepfuse: A deep unsupervised ap- proach for exposure fusion with extreme exposure image pairs. In ICCV, pages 4724-4732, 2017.","title":"Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs","authors":"Ram Prabhakar, V Sai Srikar, R Venkatesh, Babu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Confer- ence on Medical image computing and computer-assisted intervention, pages 234-241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical image computing and computer-assisted intervention","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Toet and Hogervorst, 2012] Alexander Toet and Maarten A. Hogervorst. Progress in color night vision. Optical Engi- neering, 51(1):1 -20, 2012.","title":"Progress in color night vision","authors":"Hogervorst Toet, Alexander Toet, Maarten A Hogervorst","year":"2012","venue":"Optical Engineering","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2004] Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. Image quality assess- ment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-612, 2004.","title":"Image quality assessment: from error visibility to structural similarity","authors":"Zhou Wang, Alan C Bovik, Eero P Hamid R Sheikh, Simoncelli","year":"2004","venue":"IEEE transactions on image processing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2017] Xiaoye Zhang, Yong Ma, Fan Fan, Ying Zhang, and Jun Huang. Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition. Journal of the Optical Society of America A, 34(8):1400-1410, 2017.","title":"Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition","authors":"Xiaoye Zhang, Yong Ma, Fan Fan, Ying Zhang, Jun Huang","year":"2017","venue":"Journal of the Optical Society of America A","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zong and Qiu, 2017] Jing-jing Zong and Tian-shuang Qiu. Medical image fusion based on sparse representation of classified image patches. Biomedical Signal Processing and Control, 34:195-205, 2017.","title":"Jing-jing Zong and Tian-shuang Qiu. Medical image fusion based on sparse representation of classified image patches","authors":"Qiu Zong","year":"2017","venue":"Biomedical Signal Processing and Control","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}