{"type":"ScholarlyArticle","title":"Confidence-based Self-Corrective Learning: An Application in Height Estimation Using Satellite LiDAR and Imagery","authors":[{"name":"Zhili Li","slug":"zhili-li","given":"Zhili","family":"Li","affiliation":"University of Maryland"},{"name":"Yiqun Xie","slug":"yiqun-xie","given":"Yiqun","family":"Xie","affiliation":"University of Maryland"},{"name":"Xiaowei Jia","slug":"xiaowei-jia","given":"Xiaowei","family":"Jia","affiliation":"University of Pittsburgh"}],"abstract":"Widespread, and rapid, environmental transformation is underway on Earth driven by human activities. Climate shifts such as global warming have led to massive and alarming loss of ice and snow in the high-latitude regions including the Arctic, causing many natural disasters due to sea-level rise, etc. Mitigating the impacts of climate change has also become a United Nations' Sustainable Development Goal for 2030. The recent launch of the ICESat-2 satellites target on heights in the polar regions. However, the observations are only available along very narrow scan lines, leaving large no-data gaps in-between. We aim to fill the gaps by combining the height observations with high-resolution satellite imagery that have large footprints (spatial coverage). The data expansion is a challenging task as the height data are often constrained on one or a few lines per image in real applications, and the images are highly noisy for height estimation. Related work on image-based height prediction and interpolation relies on specific types of images or does not consider the highly-localized height distribution. We propose a spatial self-corrective learning framework, which explicitly uses confidence-based pseudo-interpolation, recurrent self-refinement, and truth-based correction with a regression layer to address the challenges. We carry out experiments on different landscapes in the high-latitude regions and the proposed method shows stable improvements compared to the baseline methods.","doi":"10.24963/ijcai.2023/671","url":"https://ijcai.org/proceedings/2023/671","pdf":"https://ijcai.org/proceedings/2023/0671.pdf","markdown":"https://ijcai.org/proceedings/2023/671.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/671","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":671,"pages":{"first":6049,"last":6057},"keywords":["AI for Good: Multidisciplinary Topics and Applications","AI for Good: Machine Learning","AI for Good: Uncertainty in AI","AI for Good: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Amirkolaee and Arefi, 2019] Hamed Amini Amirkolaee and Hossein Arefi. Height estimation from single aerial images using a deep convolutional encoder-decoder network. ISPRS journal of photogrammetry and remote sensing, 149:50-66, 2019.","title":"Height estimation from single aerial images using a deep convolutional encoder-decoder network","authors":"Hamed Amini, Amirkolaee, Hossein Arefi","year":"2019","venue":"ISPRS journal of photogrammetry and remote sensing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Appleby et al., 2020] Gabriel Appleby, Linfeng Liu, and Li-Ping Liu. Kriging convolutional networks. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 34, pages 3187-3194, 2020.","title":"Kriging convolutional networks","authors":"Linfeng Gabriel Appleby, Li-Ping Liu, Liu","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bao et al., 2022] Han Bao, Xun Zhou, Yiqun Xie, Yingxue Zhang, and Yanhua Li. Covid-gan+: Estimating human mobility responses to covid-19 through spatio-temporal generative adversarial networks with enhanced features. ACM Transactions on Intelligent Systems and Technology (TIST), 13(2):1-23, 2022.","title":"Covid-gan+: Estimating human mobility responses to covid-19 through spatio-temporal generative adversarial networks with enhanced features","authors":"Han Bao, Xun Zhou, Yiqun Xie, Yingxue Zhang, Yanhua Li","year":"2022","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bertinetto et al., 2018] Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-learning with differentiable closed-form solvers. arXiv preprint arXiv:1805.08136, 2018.","title":"Meta-learning with differentiable closed-form solvers","authors":"Luca Bertinetto, Joao F Henriques, Philip Hs Torr, Andrea Vedaldi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Caballero et al., 2013] William Caballero, Ramón Giraldo, and Jorge Mateu. A universal kriging approach for spa- tial functional data. Stochastic environmental research and risk assessment, 27:1553-1563, 2013.","title":"A universal kriging approach for spatial functional data","authors":"William Caballero, Ramón Giraldo, Jorge Mateu","year":"2013","venue":"Stochastic environmental research and risk assessment","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2018] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. In Proceedings of the Euro- pean conference on computer vision (ECCV), pages 801- 818, 2018.","title":"Encoder-decoder with atrous separable convolution for semantic image segmentation","authors":"Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, Hartwig Adam","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cressie, 2015] Noel Cressie. Statistics for spatial data. John Wiley & Sons, 2015.","title":"Statistics for spatial data","authors":"Noel Cressie","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Farr and Kobrick, 2000] Tom G Farr and Mike Kobrick. Shuttle radar topography mission produces a wealth of data. Eos, Transactions American Geophysical Union, 81(48):583-585, 2000.","title":"Shuttle radar topography mission produces a wealth of data","authors":"Kobrick ; Farr, G Tom, Mike Farr, Kobrick","year":"2000","venue":"Eos, Transactions American Geophysical Union","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2020] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial networks. Communications of the ACM, 63(11):139-144, 2020.","title":"Generative adversarial networks","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2020","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing sys- tems, 30, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[KCN, 2019] KCN. The implementation of Kriging Convo- lutional Networks algorithm. https://github.com/tufts-ml/ KCN, 2019. Accessed: 2/26/2023.","title":"KCN. The implementation of Kriging Convolutional Networks algorithm","authors":"Kcn","year":"2019","venue":null,"doi":null,"url":"https://github.com/tufts-ml/KCN","internal_url":null},{"position":12,"raw":"[Li et al., 2023] Zhili Li, Yiqun Xie, Xiaowei Jia, Kara Stu- art, Caroline Delaire, and Sergii Skakun. Point-to-region co-learning for poverty mapping at high resolution using satellite imagery. In AAAI Conference on Artificial Intelli- gence, 2023.","title":"Point-to-region co-learning for poverty mapping at high resolution using satellite imagery","authors":"Zhili Li, Yiqun Xie, Xiaowei Jia, Kara Stuart, Caroline Delaire, Sergii Skakun","year":"2023","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Chun Liu, Xin Huang, Dawei Wen, Huijun Chen, and Jianya Gong. Assessing the quality of build- ing height extraction from ziyuan-3 multi-view imagery. Remote Sensing Letters, 8(9):907-916, 2017.","title":"Assessing the quality of building height extraction from ziyuan-3 multi-view imagery","authors":"Chun Liu, Xin Huang, Dawei Wen, Huijun Chen, Jianya Gong","year":"2017","venue":"Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2022] Xiaoqiang Liu, Yanjun Su, Tianyu Hu, Qi- uli Yang, Bingbing Liu, Yufei Deng, Hao Tang, Zhiyao Tang, Jingyun Fang, and Qinghua Guo. Neural network guided interpolation for mapping canopy height of china's forests by integrating gedi and icesat-2 data. Remote Sens- ing of Environment, 269:112844, 2022.","title":"Neural network guided interpolation for mapping canopy height of china's forests by integrating gedi and icesat-2 data","authors":"Xiaoqiang Liu, Yanjun Su, Tianyu Hu, Qiuli Yang, Bingbing Liu, Yufei Deng, Hao Tang, Zhiyao Tang, Jingyun Fang, Qinghua Guo","year":"2022","venue":"Remote Sensing of Environment","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[McKinley and Levine, 1998] Sky McKinley and Megan Levine. Cubic spline interpolation. College of the Red- woods, 45(1):1049-1060, 1998.","title":"Sky McKinley and Megan Levine. Cubic spline interpolation","authors":"Levine Mckinley","year":"1998","venue":"College of the Redwoods","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pörtner et al., 2019] Hans-Otto Pörtner, Debra C Roberts, Valérie Masson-Delmotte, Panmao Zhai, Melinda Tig- nor, Elvira Poloczanska, and NM Weyer. The ocean and cryosphere in a changing climate. IPCC special report on the ocean and cryosphere in a changing climate, 1155, 2019.","title":"The ocean and cryosphere in a changing climate","authors":"Pörtner","year":"2019","venue":"IPCC special report on the ocean and cryosphere in a changing climate","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Qin, 2019] Rongjun Qin. A critical analysis of satellite stereo pairs for digital surface model generation and a matching quality prediction model. ISPRS Journal of Pho- togrammetry and Remote Sensing, 154:139-150, 2019.","title":"Rongjun Qin. A critical analysis of satellite stereo pairs for digital surface model generation and a matching quality prediction model","authors":"Qin","year":"2019","venue":"ISPRS Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Richter-Menge et al., 2019] J Richter-Menge, ML Druck- enmiller, MO Jeffries, et al. Arctic ecosystems and com- munities are increasingly at risk due to continued warming and declining sea ice. 2019.","title":"Arctic ecosystems and communities are increasingly at risk due to continued warming and declining sea ice","authors":"Richter-Menge","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234- 241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shepard, 1968] Donald Shepard. A two-dimensional inter- polation function for irregularly-spaced data. In Proceed- ings of the 1968 23rd ACM national conference, pages 517-524, 1968.","title":"A two-dimensional interpolation function for irregularly-spaced data","authors":"Donald Shepard","year":"1968","venue":"Proceedings of the 1968 23rd ACM national conference","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Soergel et al., 2009] Uwe Soergel, Eckart Michaelsen, An- tje Thiele, Erich Cadario, and Ulrich Thoennessen. Stereo analysis of high-resolution SAR images for building height estimation in cases of orthogonal aspect directions. ISPRS Journal of Photogrammetry and Remote Sensing, 64(5):490-500, 2009.","title":"Stereo analysis of high-resolution SAR images for building height estimation in cases of orthogonal aspect directions","authors":"Soergel","year":"2009","venue":"ISPRS Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tzeng et al., 2017] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE confer- ence on computer vision and pattern recognition, pages 7167-7176, 2017.","title":"Adversarial discriminative domain adaptation","authors":"Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[UN, 2022] UN. United Nations' sustainable develop- ment goals. https://sdgs.un.org/goals, 2022. Accessed: 11/30/2022.","title":"UN. United Nations' sustainable development goals","authors":null,"year":"2022","venue":null,"doi":null,"url":"https://sdgs.un.org/goals,2022","internal_url":null},{"position":24,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wackernagel and Wackernagel, 2003] Hans Wackernagel and Hans Wackernagel. Ordinary kriging. Multivariate Geostatistics: An Introduction with Applications, pages 79-88, 2003.","title":"Ordinary kriging","authors":"Wackernagel Wackernagel, Hans Wackernagel, Hans Wackernagel","year":"2003","venue":"Multivariate Geostatistics: An Introduction with Applications","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2021] Yuankai Wu, Dingyi Zhuang, Aurelie Labbe, and Lijun Sun. Inductive graph neural networks for spatiotemporal kriging. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 35, pages 4478- 4485, 2021.","title":"Inductive graph neural networks for spatiotemporal kriging","authors":"Wu","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xie et al., 2021] Yiqun Xie, Erhu He, Xiaowei Jia, Han Bao, Xun Zhou, Rahul Ghosh, and Praveen Ravirathi- nam. A statistically-guided deep network transformation and moderation framework for data with spatial hetero- geneity. In 2021 IEEE International Conference on Data Mining (ICDM), pages 767-776. IEEE, 2021.","title":"A statistically-guided deep network transformation and moderation framework for data with spatial heterogeneity","authors":"Yiqun Xie, Erhu He, Xiaowei Jia, Han Bao, Xun Zhou, Rahul Ghosh, Praveen Ravirathinam","year":"2021","venue":"IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xie et al., 2023a] Yiqun Xie, Weiye Chen, Erhu He, Xi- aowei Jia, Han Bao, Xun Zhou, Rahul Ghosh, and Praveen Ravirathinam. Harnessing heterogeneity in space with sta- tistically guided meta-learning. Knowledge and informa- tion systems, 65(6):2699-2729, 2023.","title":"Harnessing heterogeneity in space with statistically guided meta-learning","authors":"Xie","year":"2023","venue":"Knowledge and information systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xie et al., 2023b] Yiqun Xie, Zhili Li, Han Bao, Xiaowei Jia, Dongkuan Xu, Xun Zhou, and Sergii Skakun. Auto- CM: Unsupervised deep learning for satellite imagery composition and cloud masking using spatio-temporal dy- namics. In AAAI Conference on Artificial Intelligence, 2023.","title":"Auto-CM: Unsupervised deep learning for satellite imagery composition and cloud masking using spatio-temporal dynamics","authors":"Xie","year":"2023","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}