{"type":"ScholarlyArticle","title":"Align, Perturb and Decouple: Toward Better Leverage of Difference Information for RSI Change Detection","authors":[{"name":"Supeng Wang","slug":"supeng-wang","given":"Supeng","family":"Wang","affiliation":"School of Computer Science , Shanghai Key Laboratory of Data Science , Fudan University"},{"name":"Yuxi Li","slug":"yuxi-li","given":"Yuxi","family":"Li","affiliation":"YouTu Lab , Tencent , Shanghai"},{"name":"Ming Xie","slug":"ming-xie","given":"Ming","family":"Xie","affiliation":"School of Computer Science , Shanghai Key Laboratory of Data Science , Fudan University"},{"name":"Mingmin Chi","slug":"mingmin-chi","given":"Mingmin","family":"Chi","affiliation":"School of Computer Science , Shanghai Key Laboratory of Data Science , Fudan University; Zhengzhou Zhongke Institute of Integrated circuit and System Application , China; Zhongshan PoolNet Technology Ltd , China"},{"name":"Yabiao Wang","slug":"yabiao-wang","given":"Yabiao","family":"Wang","affiliation":"YouTu Lab , Tencent , Shanghai"},{"name":"Chengjie Wang","slug":"chengjie-wang","given":"Chengjie","family":"Wang","affiliation":"YouTu Lab , Tencent , Shanghai; Shanghai Jiao Tong University , China"},{"name":"Wenbing Zhu","slug":"wenbing-zhu","given":"Wenbing","family":"Zhu","affiliation":"Rongcheer , China"}],"abstract":"Change detection is a widely adopted technique in remote sense imagery (RSI) analysis in the discovery of long-term geomorphic evolution. To highlight the areas of semantic changes, previous effort mostly pays attention to learning representative feature descriptors of a single image, while the difference information is either modeled with simple difference operations or implicitly embedded via feature interactions. Nevertheless, such difference modeling can be noisy since it suffers from non-semantic changes and lacks explicit guidance from image content or context. In this paper, we revisit the importance of feature difference for change detection in RSI, and propose a series of operations to fully exploit the difference information: Alignment, Perturbation and Decoupling (APD). Firstly, alignment leverages contextual similarity to compensate for the non-semantic difference in feature space. Next, a difference module trained with semantic-wise perturbation is adopted to learn more generalized change estimators, which reversely bootstraps feature extraction and prediction. Finally, a decoupled dual-decoder structure is designed to predict semantic changes in both content-aware and content-agnostic manners. Extensive experiments are conducted on benchmarks of LEVIR-CD, WHU-CD and DSIFN-CD, demonstrating our proposed operations bring significant improvement and achieve competitive results under similar comparative conditions. Code is available at https://github.com/wangsp1999/CD-Research/tree/main/openAPD","doi":"10.24963/ijcai.2023/166","url":"https://ijcai.org/proceedings/2023/166","pdf":"https://ijcai.org/proceedings/2023/0166.pdf","markdown":"https://ijcai.org/proceedings/2023/166.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/166","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":166,"pages":{"first":1497,"last":1505},"keywords":["Computer Vision: CV: Scene analysis and understanding"],"videos":[],"references":[{"position":1,"raw":"[Bandara and Patel, 2022] Wele Gedara Chaminda Bandara and Vishal M Patel. A transformer-based siamese network for change detection. arXiv preprint arXiv:2201.01293, 2022.","title":"A transformer-based siamese network for change detection","authors":"Patel ; Wele Bandara, Chaminda Gedara, Bandara, M Vishal, Patel","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen and Shi, 2020a] Hao Chen and Zhenwei Shi. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sens- ing, 12(10):1662, 2020.","title":"A spatial-temporal attention-based method and a new dataset for remote sensing image change detection","authors":"Chen, Shi Hao, Chen, Zhenwei Shi","year":"2020","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Shi, 2020b] Hao Chen and Zhenwei Shi. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote. Sens., 12:1662, 2020.","title":"A spatial-temporal attention-based method and a new dataset for remote sensing image change detection","authors":"Chen, Shi ; B] Hao, Chen, Zhenwei Shi","year":"2020","venue":"Remote. Sens","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2018] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. In Proceedings of the Euro- pean conference on computer vision (ECCV), pages 801- 818, 2018.","title":"Encoder-decoder with atrous separable convolution for semantic image segmentation","authors":"Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, Hartwig Adam","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022a] Hao Chen, Wenyuan Li, Song Chen, and Zhenwei Shi. Semantic-aware dense representation learning for remote sensing image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60:1- 18, 2022.","title":"Semantic-aware dense representation learning for remote sensing image change detection","authors":"Chen","year":"2022","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2022b] Hao Chen, Zipeng Qi, and Zhenwei Shi. Remote sensing image change detection with transform- ers. IEEE Transactions on Geoscience and Remote Sens- ing, 60:3095166, 2022.","title":"Remote sensing image change detection with transformers","authors":"Chen","year":"2022","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Daudt et al., 2018a] Rodrigo Caye Daudt, Bertr Le Saux, and Alexandre Boulch. Fully convolutional siamese net- works for change detection. In 2018 25th IEEE Inter- national Conference on Image Processing (ICIP), pages 4063-4067. IEEE, 2018.","title":"Alexandre Boulch. Fully convolutional siamese networks for change detection","authors":"Daudt","year":"2018","venue":"25th IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Daudt et al., 2018b] Rodrigo Caye Daudt, Bertr Le Saux, Alexandre Boulch, and Yann Gousseau. Urban change de- tection for multispectral earth observation using convolu- tional neural networks. In IGARSS 2018-2018 IEEE In- ternational Geoscience and Remote Sensing Symposium, pages 2115-2118. Ieee, 2018.","title":"Yann Gousseau. Urban change detection for multispectral earth observation using convolutional neural networks","authors":"Daudt","year":"2018","venue":"IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 2021] Sheng Fang, Kaiyu Li, Jinyuan Shao, and Zhe Li. Snunet-cd: A densely connected siamese network for change detection of vhr images. IEEE Geoscience and Remote Sensing Letters, 19:1-5, 2021.","title":"Snunet-cd: A densely connected siamese network for change detection of vhr images","authors":"Sheng Fang, Kaiyu Li, Jinyuan Shao, Zhe Li","year":"2021","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2017] Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine In- telligence, 42:2011-2023, 2017.","title":"Squeeze-and-excitation networks","authors":"Jie Hu, Li Shen, Samuel Albanie, Gang Sun, Enhua Wu","year":"2017","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hussain et al., 2013] Masroor Hussain, Dongmei Chen, Angela Cheng, Hui Wei, and David Stanley. Change de- tection from remotely sensed images: From pixel-based to object-based approaches. ISPRS Journal of photogramme- try and remote sensing, 80:91-106, 2013.","title":"Change detection from remotely sensed images: From pixel-based to object-based approaches","authors":"Masroor Hussain, Dongmei Chen, Angela Cheng, Hui Wei, David Stanley","year":"2013","venue":"ISPRS Journal of photogrammetry and remote sensing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jaderberg et al., 2015] Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial transformer networks. In NIPS, 2015.","title":"Spatial transformer networks","authors":"Jaderberg","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ji et al., 2018] Shunping Ji, Shiqing Wei, and Meng Lu. Fully convolutional networks for multisource building ex- traction from an open aerial and satellite imagery data set. IEEE Transactions on Geoscience and Remote Sensing, 57(1):574-586, 2018.","title":"Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set","authors":"Ji","year":"2018","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ji et al., 2019] Shunping Ji, Yanyun Shen, Meng Lu, and Yongjun Zhang. Building instance change detection from large-scale aerial images using convolutional neural net- works and simulated samples. Remote. Sens., 11:1343, 2019.","title":"Building instance change detection from large-scale aerial images using convolutional neural networks and simulated samples","authors":"Ji","year":"2019","venue":"Remote. Sens","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krizhevsky et al., 2017] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84-90, 2017.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2017","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2021] Pan Li, Da Li, Wei Li, Shaogang Gong, Yan- wei Fu, and Timothy M Hospedales. A simple feature augmentation for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8886-8895, 2021.","title":"A simple feature augmentation for domain generalization","authors":"Li","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2022] Qingyang Li, Ruofei Zhong, Xin Du, and Yuying Du. Transunetcd: A hybrid transformer network for change detection in optical remote-sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60:1-19, 2022.","title":"Transunetcd: A hybrid transformer network for change detection in optical remote-sensing images","authors":"Qingyang Li, Ruofei Zhong, Xin Du, Yuying Du","year":"2022","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2019a] Ruoyun Liu, Monika Kuffer, and Clau- dio Persello. The temporal dynamics of slums employing a cnn-based change detection approach. Remote. Sens., 11:2844, 2019.","title":"The temporal dynamics of slums employing a cnn-based change detection approach","authors":"Liu","year":"2019","venue":"Remote. Sens","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2019b] Yi Liu, Chao Pang, Zongqian Zhan, Xi- aomeng Zhang, and Xue Yang. Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model. IEEE Geo- science and Remote Sensing Letters, 18:811-815, 2019.","title":"Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model","authors":"Liu","year":"2019","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nemoto et al., 2017] Keisuke Nemoto, Ryuhei Hamaguchi, M. Sato, Aito Fujita, Tomoyuki Imaizumi, and Shuhei Hikosaka. Building change detection via a combination of cnns using only rgb aerial imageries. In Remote Sensing, 2017.","title":"Building change detection via a combination of cnns using only rgb aerial imageries","authors":"Keisuke Nemoto, Ryuhei Hamaguchi, M Sato, Aito Fujita, Tomoyuki Imaizumi, Shuhei Hikosaka","year":"2017","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Verma et al., 2019] Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better rep- resentations by interpolating hidden states. In Interna- tional Conference on Machine Learning, pages 6438- 6447. PMLR, 2019.","title":"Manifold mixup: Better representations by interpolating hidden states","authors":"Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz, Yoshua Bengio","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2019] Yulin Wang, Xuran Pan, Shiji Song, Hong Zhang, Gao Huang, and Cheng Wu. Implicit se- mantic data augmentation for deep networks. Advances in Neural Information Processing Systems, 32, 2019.","title":"Implicit semantic data augmentation for deep networks","authors":"Yulin Wang, Xuran Pan, Shiji Song, Hong Zhang, Gao Huang, Cheng Wu","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xie et al., 2021] Enze Xie, Wenhai Wang, Zhiding Yu, An- ima Anandkumar, Jose M Alvarez, and Ping Luo. Seg- former: Simple and efficient design for semantic segmen- tation with transformers. In Neural Information Process- ing Systems (NeurIPS), 2021.","title":"Segformer: Simple and efficient design for semantic segmentation with transformers","authors":"Xie","year":"2021","venue":"Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2022] Zhendong Yang, Zhe Li, Mingqi Shao, Dachuan Shi, Zehuan Yuan, and Chun Yuan. Masked generative distillation. arXiv preprint arXiv:2205.01529, 2022.","title":"Masked generative distillation","authors":"Zhendong Yang, Zhe Li, Mingqi Shao, Dachuan Shi, Zehuan Yuan, Chun Yuan","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang and Shi, 2020] Min Zhang and Wenzhong Shi. A feature difference convolutional neural network-based change detection method. IEEE Transactions on Geo- science and Remote Sensing, 58:7232-7246, 2020.","title":"A feature difference convolutional neural network-based change detection method","authors":"Shi ; ] Zhang, Min Zhang, Wenzhong Shi","year":"2020","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2020a] Chenxiao Zhang, Peng Yue, Deodato Tapete, Liangcun Jiang, Boyi Shangguan, Li Huang, and Guangchao Liu. A deeply supervised image fusion net- work for change detection in high resolution bi-temporal remote sensing images. ISPRS Journal of Photogramme- try and Remote Sensing, 166:183-200, 2020.","title":"A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images","authors":"Zhang","year":"2020","venue":"ISPRS Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2020b] Chenxiao Zhang, Peng Yue, Deodato Tapete, Liangcun Jiang, Boyi Shangguan, Li Huang, and Guangchao Liu. A deeply supervised image fusion net- work for change detection in high resolution bi-temporal remote sensing images. Isprs Journal of Photogrammetry and Remote Sensing, 166:183-200, 2020.","title":"A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images","authors":"Zhang","year":"2020","venue":"Isprs Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zheng et al., 2021] Zhuo Zheng, Ailong Ma, Liangpei Zhang, and Yanfei Zhong. Change is everywhere: Single- temporal supervised object change detection in remote sensing imagery. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 15193- 15202, 2021.","title":"Change is everywhere: Singletemporal supervised object change detection in remote sensing imagery","authors":"Zhuo Zheng, Ailong Ma, Liangpei Zhang, Yanfei Zhong","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}