{"type":"ScholarlyArticle","title":"Monitoring Vegetation From Space at Extremely Fine Resolutions via Coarsely-Supervised Smooth U-Net","authors":[{"name":"Joshua Fan","slug":"joshua-fan","given":"Joshua","family":"Fan","affiliation":"Department of Computer Science , Cornell University"},{"name":"Di Chen","slug":"di-chen","given":"Di","family":"Chen","affiliation":"Department of Computer Science , Cornell University"},{"name":"Jiaming Wen","slug":"jiaming-wen","given":"Jiaming","family":"Wen","affiliation":"School of Integrative Plant Science -Soil and Crop Sciences Section , Cornell University"},{"name":"Ying Sun","slug":"ying-sun","given":"Ying","family":"Sun","affiliation":"School of Integrative Plant Science -Soil and Crop Sciences Section , Cornell University"},{"name":"Carla Gomes","slug":"carla-gomes","given":"Carla","family":"Gomes","affiliation":"Department of Computer Science , Cornell University"}],"abstract":"Monitoring vegetation productivity at extremely fine resolutions is valuable for real-world agricultural applications, such as detecting crop stress and providing early warning of food insecurity. Solar-Induced Chlorophyll Fluorescence (SIF) provides a promising way to directly measure plant productivity from space. However, satellite SIF observations are only available at a coarse spatial resolution, making it impossible to monitor how individual crop types or farms are doing. This poses a challenging coarsely-supervised regression (or downscaling) task; at training time, we only have SIF labels at a coarse resolution (3km), but we want to predict SIF at much finer spatial resolutions (e.g. 30m, a 100x increase). We also have additional fine-resolution input features, but the relationship between these features and SIF is unknown. To address this, we propose Coarsely-Supervised Smooth U-Net (CS-SUNet), a novel method for this coarse supervision setting. CS-SUNet combines the expressive power of deep convolutional networks with novel regularization methods based on prior knowledge (such as a smoothness loss) that are crucial for preventing overfitting. Experiments show that CS-SUNet resolves fine-grained variations in SIF more accurately than existing methods.","doi":"10.24963/ijcai.2022/703","url":"https://ijcai.org/proceedings/2022/703","pdf":"https://ijcai.org/proceedings/2022/0703.pdf","markdown":"https://ijcai.org/proceedings/2022/703.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/703","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Research Papers","track":"AI for Good","number":703,"pages":{"first":5066,"last":5072},"keywords":["Machine Learning: Weakly Supervised Learning","Multidisciplinary Topics and Applications: Sustainable Development Goals","Computer Vision: Applications","Machine Learning: Multi-instance","Computer Vision: Transfer, low-shot, semi- and un- supervised learning","Multidisciplinary Topics and Applications: Computational Sustainability"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987142","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985285","length_sec":700,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Duveiller et al., 2020] G. Duveiller, F. Filipponi, S. Walther, P. Köhler, C. Frankenberg, L. Guanter, and A. Cescatti. A spatially downscaled sun-induced fluorescence global product for enhanced monitoring of vegetation productivity. Earth System Science Data, 12(2):1101-1116, 2020.","title":"A spatially downscaled sun-induced fluorescence global product for enhanced monitoring of vegetation productivity","authors":"G Duveiller, F Filipponi, S Walther, P Köhler, C Frankenberg, L Guanter, A Cescatti","year":"2020","venue":"Earth System Science Data","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Kira and Sun, 2020] O. Kira and Y. Sun. Extraction of sub- pixel c3/c4 emissions of solar-induced chlorophyll fluores- cence (sif) using artificial neural network. ISPRS Journal of Photogrammetry and Remote Sensing, 161:135-146, 2020.","title":"Extraction of subpixel c3/c4 emissions of solar-induced chlorophyll fluorescence (sif) using artificial neural network","authors":"O Kira, Y Sun","year":"2020","venue":"ISPRS Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Kotzias et al., 2015] D. Kotzias, M. Denil, N. De Freitas, and P. Smyth. From group to individual labels using deep features. In Proceedings of the 21th ACM SIGKDD in- ternational conference on knowledge discovery and data mining, pages 597-606, 2015.","title":"From group to individual labels using deep features","authors":"D Kotzias, M Denil, N De Freitas, P Smyth","year":"2015","venue":"Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Li and Xiao, 2019] X. Li and J. Xiao. A global, 0.05-degree product of solar-induced chlorophyll fluorescence derived from oco-2, modis, and reanalysis data. Remote Sensing, 11(5):517, 2019.","title":"A global, 0.05-degree product of solar-induced chlorophyll fluorescence derived from oco-2, modis, and reanalysis data","authors":"Xiao ; Li, X Li, J Xiao","year":"2019","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Liu et al., 2020] Y. Liu, A. R. Ganguly, and J. Dy. Climate downscaling using ynet: A deep convolutional network with skip connections and fusion. In Proceedings of the 26th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, pages 3145-3153, 2020.","title":"Climate downscaling using ynet: A deep convolutional network with skip connections and fusion","authors":"Y Liu, A R Ganguly, J Dy","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Loshchilov and Hutter, 2018] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2018.","title":"Decoupled weight decay regularization","authors":"Hutter Loshchilov, I Loshchilov, F Hutter","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Musicant et al., 2007] D. R. Musicant, J. M. Christensen, and J. F. Olson. Supervised learning by training on aggre- gate outputs. In Seventh IEEE International Conference on Data Mining (ICDM 2007), pages 252-261. IEEE, 2007.","title":"Supervised learning by training on aggregate outputs","authors":"D R Musicant, J M Christensen, J F Olson","year":"2007","venue":"Seventh IEEE International Conference on Data Mining (ICDM 2007)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ortiz-Bobea et al., 2021] A. Ortiz-Bobea, T. R. Ault, C. M. Carrillo, R. G. Chambers, and D. B. Lobell. Anthro- pogenic climate change has slowed global agricultural pro- ductivity growth. Nature Climate Change, 11(4):306-312, 2021.","title":"Anthropogenic climate change has slowed global agricultural productivity growth","authors":"A Ortiz-Bobea, T R Ault, C M Carrillo, R G Chambers, D B Lobell","year":"2021","venue":"Nature Climate Change","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Peng et al., 2017] J. Peng, A. Loew, O. Merlin, and N. E. C. Verhoest. A review of spatial downscaling of satellite remotely sensed soil moisture. Reviews of Geophysics, 55(2):341-366, 2017.","title":"A review of spatial downscaling of satellite remotely sensed soil moisture","authors":"J Peng, A Loew, O Merlin, N E C Verhoest","year":"2017","venue":"Reviews of Geophysics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Peng et al., 2020] B. Peng, K. Guan, W. Zhou, C. Jiang, C. Frankenberg, Y. Sun, L. He, and P. Köhler. Assess- ing the benefit of satellite-based solar-induced chlorophyll fluorescence in crop yield prediction. International Jour- nal of Applied Earth Observation and Geoinformation, 90:102126, 2020.","title":"Assessing the benefit of satellite-based solar-induced chlorophyll fluorescence in crop yield prediction","authors":"B Peng, K Guan, W Zhou, C Jiang, C Frankenberg, Y Sun, L He, P Köhler","year":"2020","venue":"International Journal of Applied Earth Observation and Geoinformation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ronneberger et al., 2015] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Med- ical image computing and computer-assisted intervention, pages 234-241. Springer, 2015.","title":"U-net: Convolutional networks for biomedical image segmentation","authors":"O Ronneberger, P Fischer, T Brox","year":"2015","venue":"International Conference on Medical image computing and computer-assisted intervention","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Rosca et al., 2020] M. Rosca, T. Weber, A. Gretton, and S. Mohamed. A case for new neural networks smoothness constraints. In \"I Can't Believe It's Not Better!\"NeurIPS 2020 workshop, 2020.","title":"A case for new neural networks smoothness constraints","authors":"M Rosca, T Weber, A Gretton, S Mohamed","year":"2020","venue":"I Can't Believe It's Not Better!","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Song et al., 2020] Y. Song, J. Wang, and L. Wang. Satel- lite solar-induced chlorophyll fluorescence reveals heat stress impacts on wheat yield in india. Remote Sensing, 12(20):3277, 2020.","title":"Satellite solar-induced chlorophyll fluorescence reveals heat stress impacts on wheat yield in india","authors":"Y Song, J Wang, L Wang","year":"2020","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sturrock et al., 2014] H. J. W. Sturrock, J. M. Cohen, P. Keil, A. J. Tatem, A. Le Menach, N. E. Ntshalintshali, M. S. Hsiang, and R. D. Gosling. Fine-scale malaria risk mapping from routine aggregated case data. Malaria jour- nal, 13(1):1-9, 2014.","title":"Fine-scale malaria risk mapping from routine aggregated case data","authors":"H J W Sturrock, J M Cohen, P Keil, A J Tatem, A Le Menach, N E Ntshalintshali, M S Hsiang, R D Gosling","year":"2014","venue":"Malaria journal","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sun et al., 2017] Y. Sun, C. Frankenberg, J. D. Wood, D. S. Schimel, M. Jung, L. Guanter, D. T. Drewry, M. Verma, A. Porcar-Castell, T. J. Griffis, et al. Oco-2 advances photosynthesis observation from space via solar-induced chlorophyll fluorescence. Science, 358(6360):eaam5747, 2017.","title":"Oco-2 advances photosynthesis observation from space via solar-induced chlorophyll fluorescence","authors":"Y Sun, C Frankenberg, J D Wood, D S Schimel, M Jung, L Guanter, D T Drewry, M Verma, A Porcar-Castell, T J Griffis","year":"2017","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tanaka et al., 2019] Y. Tanaka, T. Iwata, T. Tanaka, T. Kurashima, M. Okawa, and H. Toda. Refining coarse- grained spatial data using auxiliary spatial data sets with various granularities. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 33, pages 5091- 5099, 2019.","title":"Refining coarsegrained spatial data using auxiliary spatial data sets with various granularities","authors":"Y Tanaka, T Iwata, T Tanaka, T Kurashima, M Okawa, H Toda","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang and Yang, 2018] Z. Wang and J. Yang. Diabetic retinopathy detection via deep convolutional networks for discriminative localization and visual explanation. In Workshops at the thirty-second AAAI conference on arti- ficial intelligence, 2018.","title":"Diabetic retinopathy detection via deep convolutional networks for discriminative localization and visual explanation","authors":"Wang, Yang ;, ] Z Wang, J Yang","year":"2018","venue":"Workshops at the thirty-second AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2020] S. Wang, W. Chen, S. M. Xie, G. Azzari, and D. B. Lobell. Weakly supervised deep learning for segmentation of remote sensing imagery. Remote Sensing, 12(2):207, 2020.","title":"Weakly supervised deep learning for segmentation of remote sensing imagery","authors":"S Wang, W Chen, S M Xie, G Azzari, D B Lobell","year":"2020","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wen et al., 2020] J. Wen, P. Köhler, G. Duveiller, N. C. Parazoo, T. S. Magney, G. Hooker, L. Yu, C. Y. Chang, and Y. Sun. A framework for harmonizing multiple satellite instruments to generate a long-term global high spatial-resolution solar-induced chlorophyll fluorescence (sif). Remote Sensing of Environment, 239:111644, 2020.","title":"A framework for harmonizing multiple satellite instruments to generate a long-term global high spatial-resolution solar-induced chlorophyll fluorescence (sif)","authors":"J Wen, P Köhler, G Duveiller, N C Parazoo, T S Magney, G Hooker, L Yu, C Y Chang, Y Sun","year":"2020","venue":"Remote Sensing of Environment","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wood et al., 2004] A. W. Wood, L. R. Leung, V. Sridhar, and D. P. Lettenmaier. Hydrologic implications of dy- namical and statistical approaches to downscaling climate model outputs. Climatic change, 62(1):189-216, 2004.","title":"Hydrologic implications of dynamical and statistical approaches to downscaling climate model outputs","authors":"A W Wood, L R Leung, V Sridhar, D P Lettenmaier","year":"2004","venue":"Climatic change","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yu et al., 2019] L. Yu, J. Wen, C. Y. Chang, C. Frankenberg, and Y. Sun. High-resolution global contiguous sif of oco- 2. Geophysical Research Letters, 46(3):1449-1458, 2019.","title":"High-resolution global contiguous sif of oco-2","authors":"L Yu, J Wen, C Y Chang, C Frankenberg, Y Sun","year":"2019","venue":"Geophysical Research Letters","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2020] Y. Zhang, N. Charoenphakdee, Z. Wu, and M. Sugiyama. Learning from aggregate observa- tions. Advances in Neural Information Processing Sys- tems, 33:7993-8005, 2020.","title":"Learning from aggregate observations","authors":"Y Zhang, N Charoenphakdee, Z Wu, M Sugiyama","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}