{"type":"ScholarlyArticle","title":"EvaNet: Elevation-Guided Flood Extent Mapping on Earth Imagery","authors":[{"name":"Mirza Tanzim Sami","slug":"mirza-tanzim-sami","given":"Mirza Tanzim","family":"Sami","affiliation":"University of Alabama at Birmingham"},{"name":"Da Yan","slug":"da-yan","given":"Da","family":"Yan","affiliation":"Indiana University Bloomington"},{"name":"Saugat Adhikari","slug":"saugat-adhikari","given":"Saugat","family":"Adhikari","affiliation":"Indiana University Bloomington"},{"name":"Lyuheng Yuan","slug":"lyuheng-yuan","given":"Lyuheng","family":"Yuan","affiliation":"Indiana University Bloomington"},{"name":"Jiao Han","slug":"jiao-han","given":"Jiao","family":"Han","affiliation":"Indiana University Bloomington"},{"name":"Zhe Jiang","slug":"zhe-jiang","given":"Zhe","family":"Jiang","affiliation":"University of Florida"},{"name":"Jalal Khalil","slug":"jalal-khalil","given":"Jalal","family":"Khalil","affiliation":"St . Cloud State University"},{"name":"Yang Zhou","slug":"yang-zhou","given":"Yang","family":"Zhou","affiliation":"Auburn University"}],"abstract":"Accurate and timely mapping of flood extent from high resolution satellite imagery plays a crucial role in disaster management such as damage assessment and relief activities. However, current state-of-the-art solutions are based on U-Net, which cannot segment the flood pixels accurately due to the ambiguous pixels (e.g., tree canopies, clouds) that prevent a direct judgement from only the spectral features. Thanks to the digital elevation model (DEM) data readily available from sources such as United States Geological Survey (USGS), this work explores the use of an elevation map to improve flood extent mapping. We propose, EvaNet, an elevation-guided segmentation model based on the encoder-decoder architecture with two novel techniques: (1) a loss function encoding the physical law of gravity that if a location is flooded (resp. dry), then its adjacent locations with a lower (resp. higher) elevation must also be flooded (resp. dry); (2) a new (de)convolution operation that integrates the elevation map by a location-sensitive gating mechanism to regulate how much spectral features flow through adjacent layers. Extensive experiments show that EvaNet significantly outperforms the U-Net baselines, and works as a perfect drop-in replacement for U-Net in existing solutions to flood extent mapping. EvaNet is open-sourced at https://github.com/MTSami/EvaNet.","doi":"10.24963/ijcai.2024/133","url":"https://ijcai.org/proceedings/2024/133","pdf":"https://ijcai.org/proceedings/2024/0133.pdf","markdown":"https://ijcai.org/proceedings/2024/133.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/133","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":133,"pages":{"first":1200,"last":1208},"keywords":["Computer Vision: CV: 3D computer vision","Data Mining: DM: Mining spatial and/or temporal data","Computer Vision: CV: Applications","Computer Vision: CV: Segmentation"],"videos":[],"references":[{"position":1,"raw":"[Adhikari et al., 2022] Saugat Adhikari, Da Yan, Mirza Tanzim Sami, Jalal Khalil, Lyuheng Yuan, Bhadhan Roy Joy, Zhe Jiang, and Arpan Man Sainju. An elevation-guided annotation tool for flood extent mapping on earth imagery. In SIGSPATIAL. ACM, 2022.","title":"An elevation-guided annotation tool for flood extent mapping on earth imagery","authors":"Adhikari","year":"2022","venue":"SIGSPATIAL. ACM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arevalo et al., 2017] John Arevalo, Thamar Solorio, Manuel Montes-y Gómez, and Fabio A González. Gated multimodal units for information fusion. arXiv preprint arXiv:1702.01992, 2017.","title":"Gated multimodal units for information fusion","authors":"John Arevalo, Thamar Solorio, Manuel Montes-Y Gómez, Fabio A González","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beni and Gebrehiwot, 2021] Leila Hashemi Beni and As- mamaw Gebrehiwot. Flood extent mapping: An inte- grated method using deep learning and region growing us- ing UAV optical data. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens., 14:2127-2135, 2021.","title":"Flood extent mapping: An integrated method using deep learning and region growing using UAV optical data","authors":"Leila Hashemi, Beni, Asmamaw Gebrehiwot","year":"2021","venue":"IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bonafilia et al., 2020] Derrick Bonafilia, Beth Tellman, Tyler Anderson, and Erica Issenberg. Sen1floods11: A georeferenced dataset to train and test deep learning flood algorithms for sentinel-1. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 210-211, 2020. [Copernicus, 2014] Copernicus. Sentinel-1 -missions - sentinel online -sentinel online. https://sentinel.esa.int/ web/sentinel/missions/sentinel-1, 2014. (Accessed on 07/29/2023).","title":"Sen1floods11: A georeferenced dataset to train and test deep learning flood algorithms for sentinel-1","authors":"Derrick Bonafilia, Beth Tellman, Tyler Anderson, Erica Issenberg","year":"2014","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops","doi":null,"url":"https://sentinel.esa.int/web/sentinel/missions/sentinel-1,2014","internal_url":null},{"position":5,"raw":"[Copernicus, 2015] Copernicus. Sentinel-2 -missions - sentinel online -sentinel online. https://sentinel.esa.int/ web/sentinel/missions/sentinel-2, 2015. (Accessed on 07/29/2023).","title":"Copernicus. Sentinel-2 -missionssentinel online -sentinel online","authors":null,"year":"2015","venue":null,"doi":null,"url":"https://sentinel.esa.int/web/sentinel/missions/sentinel-2,2015","internal_url":null},{"position":6,"raw":"[Dauphin et al., 2017] Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In Doina Precup and Yee Whye Teh, editors, ICML, volume 70 of Proceedings of Machine Learning Research, pages 933-941. PMLR, 2017.","title":"Language modeling with gated convolutional networks","authors":"N Yann, Angela Dauphin, Michael Fan, David Auli, Grangier","year":"2017","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Drakonakis et al., 2022] Georgios I Drakonakis, Grigo- rios Tsagkatakis, Konstantina Fotiadou, and Panagio- tis Tsakalides. Ombrianet-supervised flood mapping via convolutional neural networks using multitemporal sentinel-1 and sentinel-2 data fusion. IEEE Journal of Se- lected Topics in Applied Earth Observations and Remote Sensing, 15:2341-2356, 2022.","title":"Ombrianet-supervised flood mapping via convolutional neural networks using multitemporal sentinel-1 and sentinel-2 data fusion","authors":"Grigorios Georgios I Drakonakis, Konstantina Tsagkatakis, Panagiotis Fotiadou, Tsakalides","year":"2022","venue":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fraccaro et al., 2022] Paolo Fraccaro, Nikola Stoyanov, Za- heed Gaffoor, Laura Elena Cue La Rosa, Jitendra Singh, Tatsuya Ishikawa, Blair Edwards, Anne Jones, and Kom- minist Weldermariam. Deploying an artificial intelligence application to detect flood from sentinel 1 data. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12489-12495, 2022.","title":"Deploying an artificial intelligence application to detect flood from sentinel 1 data","authors":"Paolo Fraccaro, Nikola Stoyanov, Zaheed Gaffoor, Laura Elena Cue, La Rosa, Jitendra Singh, Tatsuya Ishikawa, Blair Edwards, Anne Jones, Komminist Weldermariam","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ghosh et al., 2022] Rahul Ghosh, Arvind Renganathan, Kshitij Tayal, Xiang Li, Ankush Khandelwal, Xiaowei Jia, Christopher J. Duffy, John Nieber, and Vipin Kumar. Robust inverse framework using knowledge-guided self- supervised learning: An application to hydrology. In KDD, pages 465-474. ACM, 2022.","title":"Robust inverse framework using knowledge-guided selfsupervised learning: An application to hydrology","authors":"Rahul Ghosh, Arvind Renganathan, Kshitij Tayal, Xiang Li, Ankush Khandelwal, Xiaowei Jia, Christopher J Duffy, John Nieber, Vipin Kumar","year":"2022","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. Mask R-CNN. In ICCV, pages 2980-2988. IEEE Computer Society, 2017.","title":"Mask R-CNN","authors":"He","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hui et al., 2021a] Bo Hui, Da Yan, Haiquan Chen, and Wei- Shinn Ku. Trajectory wavenet: A trajectory-based model for traffic forecasting. In ICDM, pages 1114-1119. IEEE, 2021.","title":"Trajectory wavenet: A trajectory-based model for traffic forecasting","authors":"Hui","year":"2021","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hui et al., 2021b] Bo Hui, Da Yan, Haiquan Chen, and Wei- Shinn Ku. Trajnet: A trajectory-based deep learning model for traffic prediction. In KDD, pages 716-724. ACM, 2021.","title":"Trajnet: A trajectory-based deep learning model for traffic prediction","authors":"Hui","year":"2021","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jiang and Sainju, 2019] Zhe Jiang and Arpan Man Sainju. Hidden markov contour tree: A spatial structured model for hydrological applications. In Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis, editors, KDD, pages 804-813. ACM, 2019.","title":"Hidden markov contour tree: A spatial structured model for hydrological applications","authors":"Zhe Jiang, Arpan, Man Sainju","year":"2019","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jiang et al., 2021] Peishi Jiang, Nis Meinert, Helga Jordão, Constantin Weisser, Simon Holgate, Alexander Lavin, Björn Lütjens, Dava Newman, Haruko Wainwright, Catherine Walker, and Patrick Barnard. Digital twin earth -coasts: Developing a fast and physics-informed surro- gate model for coastal floods via neural operators. arXiv preprint arXiv:2110.07100, 2021.","title":"Digital twin earth -coasts: Developing a fast and physics-informed surrogate model for coastal floods via neural operators","authors":"Jiang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kim et al., 2018] Jaekyum Kim, Junho Koh, Yecheol Kim, Jaehyung Choi, Youngbae Hwang, and Jun Won Choi. Ro- bust deep multi-modal learning based on gated informa- tion fusion network. In Asian Conference on Computer Vision, pages 90-106. Springer, 2018.","title":"Robust deep multi-modal learning based on gated information fusion network","authors":"Kim","year":"2018","venue":"Asian Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kirillov et al., 2019] Alexander Kirillov, Kaiming He, Ross B. Girshick, Carsten Rother, and Piotr Dollár. Panoptic segmentation. In CVPR, pages 9404-9413. Computer Vision Foundation / IEEE, 2019.","title":"Panoptic segmentation","authors":"Alexander Kirillov, Kaiming He, Ross B Girshick, Carsten Rother, Piotr Dollár","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Konapala et al., 2021] Goutam Konapala, Sujay V Kumar, and Shahryar Khalique Ahmad. Exploring sentinel-1 and sentinel-2 diversity for flood inundation mapping using deep learning. ISPRS Journal of Photogrammetry and Re- mote Sensing, 180:163-173, 2021.","title":"Exploring sentinel-1 and sentinel-2 diversity for flood inundation mapping using deep learning","authors":"Goutam Konapala, V Sujay, Shahryar Kumar, Ahmad Khalique","year":"2021","venue":"ISPRS Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Long et al., 2015a] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431-3440. IEEE Com- puter Society, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Long","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Long et al., 2015b] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431- 3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Long","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mahesh et al., 2022] Ragini Bal Mahesh, Jorge Leandro, and Qing Lin. Physics informed neural network for spatial-temporal flood forecasting. In Climate Change and Water Security: Select Proceedings of VCDRR 2021, pages 77-91. Springer, 2022.","title":"Physics informed neural network for spatial-temporal flood forecasting","authors":"Ragini Bal, Mahesh, Jorge Leandro, Qing Lin","year":"2022","venue":"Climate Change and Water Security: Select Proceedings of VCDRR 2021","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mateo-Garcia et al., 2021] Gonzalo Mateo-Garcia, Joshua Veitch-Michaelis, Lewis Smith, Silviu Vlad Oprea, Guy Schumann, Yarin Gal, Atılım Günes ¸Baydin, and Diet- mar Backes. Towards global flood mapping onboard low cost satellites with machine learning. Scientific reports, 11(1):1-12, 2021.","title":"Towards global flood mapping onboard low cost satellites with machine learning","authors":"Gonzalo Mateo-Garcia, Joshua Veitch-Michaelis, Lewis Smith, Silviu Vlad Oprea, Guy Schumann, Yarin Gal, Atılım Günes ¸baydin, Dietmar Backes","year":"2021","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Matgen et al., 2020] Patrick Matgen, Sandro Martinis, Wolfgang Wagner, Vahid Freeman, Peter Zeil, Niall Mc- Cormick, et al. Feasibility assessment of an automated, global, satellite-based flood-monitoring product for the copernicus emergency management service. Luxembourg: Publications Office of the European Union, 2020.","title":"Feasibility assessment of an automated, global, satellite-based flood-monitoring product for the copernicus emergency management service","authors":"Patrick Matgen, Sandro Martinis, Wolfgang Wagner, Vahid Freeman, Peter Zeil, Niall Mc-Cormick","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[NCSU, 2023] NCSU. NCSU Libraries. https://www.lib. ncsu.edu/gis/elevation, 2023.","title":"NCSU. NCSU Libraries","authors":null,"year":"2023","venue":null,"doi":null,"url":"https://www.lib.ncsu.edu/gis/elevation,2023","internal_url":null},{"position":24,"raw":"[Nearing et al., 2020] Grey Nearing, Frederik Kratzert, Daniel Klotz, Pieter-Jan Hoedt, Günter Klambauer, Sepp Hochreiter, Hoshin Gupta, Sella Nevo, and Yossi Matias. A deep learning architecture for conservative dynamical systems: Application to rainfall-runoff modeling. In AI for Earth Sciences Workshop at NeurIPS, 2020.","title":"A deep learning architecture for conservative dynamical systems: Application to rainfall-runoff modeling","authors":"Nearing","year":"2020","venue":"AI for Earth Sciences Workshop at NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[NOAA, 2016] NOAA. Hurricane matthew imagery. https: //geodesy.noaa.gov/storm archive/storms/matthew/, 2016. (Accessed on 08/02/2023).","title":"NOAA. Hurricane matthew imagery","authors":null,"year":"2016","venue":null,"doi":null,"url":"https://geodesy.noaa.gov/stormarchive/storms/matthew/,2016","internal_url":null},{"position":26,"raw":"[NOAA, 2017] NOAA. Hurricane harvey imagery. https://storms.ngs.noaa.gov/storms/harvey/index.html#7/ 28.411/-96.691, 2017. (Accessed on 08/02/2023).","title":"NOAA. Hurricane harvey imagery","authors":null,"year":"2017","venue":null,"doi":null,"url":"https://storms.ngs.noaa.gov/storms/harvey/index.html#7/28.411/-96.691,2017","internal_url":null},{"position":27,"raw":"[Oddo and Bolten, 2019] PC Oddo and JD Bolten. The value of near real-time earth observations for improved flood disaster response. frontiers in environmental science, 2019.","title":"The value of near real-time earth observations for improved flood disaster response","authors":"Bolten ; ] Pc Oddo, Oddo, Bolten","year":"2019","venue":"frontiers in environmental science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, volume 9351 of Lecture Notes in Computer Science, pages 234- 241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Rudner et al., 2019] Tim GJ Rudner, Marc Rußwurm, Jakub Fil, Ramona Pelich, Benjamin Bischke, Veronika Kopačková, and Piotr Biliński. Multi3net: segmenting flooded buildings via fusion of multiresolution, multisen- sor, and multitemporal satellite imagery. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 702-709, 2019.","title":"Multi3net: segmenting flooded buildings via fusion of multiresolution, multisensor, and multitemporal satellite imagery","authors":"G J Tim, Marc Rudner, Jakub Rußwurm, Ramona Fil, Benjamin Pelich, Veronika Bischke, Piotr Kopačková, Biliński","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sainju et al., 2020] Arpan Man Sainju, Wenchong He, Zhe Jiang, and Da Yan. Spatial classification with limited ob- servations based on physics-aware structural constraint. In AAAI, pages 898-905. AAAI Press, 2020.","title":"Spatial classification with limited observations based on physics-aware structural constraint","authors":"Sainju","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sainju et al., 2022] Arpan Man Sainju, Wenchong He, and Zhe Jiang. A hidden markov contour tree model for spa- tial structured prediction. IEEE Trans. Knowl. Data Eng., 34(4):1530-1543, 2022.","title":"A hidden markov contour tree model for spatial structured prediction","authors":"Man Arpan, Wenchong Sainju, Zhe He, Jiang","year":"2022","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sami et al., 2024] Mirza Tanzim Sami, Da Yan, Saugat Adhikari, Lyuheng Yuan, Jiao Han, Zhe Jiang, Jalal Khalil, and Yang Zhou. Evanet: Elevation-guided flood extent mapping on earth imagery. arXiv preprint arXiv:2404.17917, 2024.","title":"Evanet: Elevation-guided flood extent mapping on earth imagery","authors":"Tanzim Mirza, Da Sami, Saugat Yan, Lyuheng Adhikari, Jiao Yuan, Zhe Han, Jalal Jiang, Yang Khalil, Zhou","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Sun et al., 2023] Alexander Y. Sun, Zhi Li, Wonhyun Lee, Qixing Huang, Bridget R. Scanlon, and Clint Dawson. Rapid flood inundation forecast using fourier neural op- erator. In ICCV, pages 3735-3741. IEEE, 2023. [USGS, 2023] USGS. Tnm download v2. https:// apps.nationalmap.gov/downloader/, 2023. (Accessed on 08/02/2023).","title":"Rapid flood inundation forecast using fourier neural operator","authors":"Alexander Y Sun, Zhi Li, Wonhyun Lee, Qixing Huang, Bridget R Scanlon, Clint Dawson","year":"2023","venue":"ICCV","doi":null,"url":"https://apps.nationalmap.gov/downloader/,2023","internal_url":null},{"position":34,"raw":"[Wahlstrom et al., 2015] Margareta Wahlstrom, Debarati Guha-Sapir, et al. The human cost of weather-related disasters 1995-2015. Geneva, Switzerland: UNISDR, 2015.","title":"The human cost of weather-related disasters 1995-2015","authors":"Margareta Wahlstrom, Debarati Guha-Sapir","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wu et al., 2019] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In IJCAI, pages 1907- 1913. ijcai.org, 2019.","title":"Graph wavenet for deep spatial-temporal graph modeling","authors":"Wu","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/264"},{"position":36,"raw":"[Xie et al., 2018] Miao Xie, Zhe Jiang, and Arpan Man Sainju. Geographical hidden markov tree for flood extent mapping. In KDD, pages 2545-2554. ACM, 2018.","title":"Geographical hidden markov tree for flood extent mapping","authors":"Miao Xie, Zhe Jiang, Arpan Man Sainju","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}