{"type":"ScholarlyArticle","title":"Recurrent Generative Networks for Multi-Resolution Satellite Data: An Application in Cropland Monitoring","authors":[{"name":"Xiaowei Jia","slug":"xiaowei-jia","given":"Xiaowei","family":"Jia","affiliation":"Department of Computer Science and Engineering , University of Minnesota"},{"name":"Mengdie Wang","slug":"mengdie-wang","given":"Mengdie","family":"Wang","affiliation":"Department of Computer Science and Engineering , University of Minnesota"},{"name":"Ankush Khandelwal","slug":"ankush-khandelwal","given":"Ankush","family":"Khandelwal","affiliation":"Department of Computer Science and Engineering , University of Minnesota"},{"name":"Anuj Karpatne","slug":"anuj-karpatne","given":"Anuj","family":"Karpatne","affiliation":"Department of Computer Science , Virginia Tech"},{"name":"Vipin Kumar","slug":"vipin-kumar","given":"Vipin","family":"Kumar","affiliation":"Department of Computer Science and Engineering , University of Minnesota"}],"abstract":"Effective and timely monitoring of croplands is critical for managing food supply. While remote sensing data from earth-observing satellites can be used to monitor croplands over large regions, this task is challenging for small-scale croplands as they cannot be captured precisely using coarse-resolution data. On the other hand, the remote sensing data in higher resolution are collected less frequently and contain missing or disturbed data. Hence, traditional sequential models cannot be directly applied on high-resolution data to extract temporal patterns, which are essential to identify crops. In this work, we propose a generative model to combine multi-scale remote sensing data to detect croplands at high resolution. During the learning process, we leverage the temporal patterns learned from coarse-resolution data to generate missing high-resolution data. Additionally, the proposed model can track classification confidence in real time and potentially lead to an early detection. The evaluation in an intensively cultivated region demonstrates the effectiveness of the proposed method in cropland detection.","doi":"10.24963/ijcai.2019/365","url":"https://ijcai.org/proceedings/2019/365","pdf":"https://ijcai.org/proceedings/2019/0365.pdf","markdown":"https://ijcai.org/proceedings/2019/365.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/365","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":365,"pages":{"first":2628,"last":2634},"keywords":["Machine Learning: Deep Learning","Machine Learning Applications: Environmental","Machine Learning: Classification","Multidisciplinary Topics and Applications: Computational Sustainability"],"videos":[],"references":[{"position":1,"raw":"[Audebert et al., 2017] Nicolas Audebert, Bertrand Le Saux, and Sébastien Lefèvre. Joint learning from earth observa- tion and openstreetmap data to get faster better semantic maps. In EARTHVISION 2017 CVPR Workshop, 2017.","title":"Joint learning from earth observation and openstreetmap data to get faster better semantic maps","authors":"Audebert","year":"2017","venue":"EARTHVISION 2017 CVPR Workshop","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Che et al., 2016] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. arXiv preprint arXiv:1606.01865, 2016.","title":"Recurrent neural networks for multivariate time series with missing values","authors":"Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, Yan Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Stow, 2003] DongMei Chen and Douglas Stow. Strategies for integrating information from multiple spa- tial resolutions into land-use/land-cover classification rou- tines. PE&RS, 2003.","title":"Strategies for integrating information from multiple spatial resolutions into land-use/land-cover classification routines","authors":"Chen, Stow ;, Dongmei Chen, Douglas Stow","year":"2003","venue":"PE&RS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chung et al., 2015] Junyoung Chung, Kyle Kastner, Lau- rent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. A recurrent latent variable model for sequential data. In Advances in neural information processing sys- tems, pages 2980-2988, 2015.","title":"A recurrent latent variable model for sequential data","authors":"Chung","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gueguen and Hamid, 2015] Lionel Gueguen and Raffay Hamid. Large-scale damage detection using satellite im- agery. In CVPR, 2015.","title":"Large-scale damage detection using satellite imagery","authors":"Hamid ; Gueguen, Lionel Gueguen, Raffay Hamid","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guindin-Garcia et al., 2012] Noemi Guindin-Garcia, Ana- toly A Gitelson, Timothy J Arkebauer, John Shanahan, and Albert Weiss. An evaluation of modis 8-and 16-day com- posite products for monitoring maize green leaf area index. Agricultural and Forest Meteorology, 161:15-25, 2012.","title":"An evaluation of modis 8-and 16-day composite products for monitoring maize green leaf area index","authors":"Noemi Guindin-Garcia, Anatoly A Gitelson, Timothy J Arkebauer, John Shanahan, Albert Weiss","year":"2012","venue":"Agricultural and Forest Meteorology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Habibie et al., 2017] Ikhsanul Habibie, Daniel Holden, Jonathan Schwarz, Joe Yearsley, Taku Komura, Jun Saito, Ikuo Kusajima, Xi Zhao, Myung-Geol Choi, Ruizhen Hu, et al. A recurrent variational autoencoder for human mo- tion synthesis. IEEE CG&A, 2017.","title":"A recurrent variational autoencoder for human motion synthesis","authors":"Habibie","year":"2017","venue":"IEEE CG&A","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Inglada et al., 2016] Jordi Inglada, Arthur Vincent, Marcela Arias, and Claire Marais-Sicre. Improved early crop type identification by joint use of high temporal resolution sar and optical image time series. Remote Sensing, 2016.","title":"Improved early crop type identification by joint use of high temporal resolution sar and optical image time series","authors":"Jordi Inglada, Arthur Vincent, Marcela Arias, Claire Marais-Sicre","year":"2016","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jia et al., 2017a] Xiaowei Jia, Ankush Khandelwal, Gu- ruprasad Nayak, James Gerber, Kimberly Carlson, Paul West, and Vipin Kumar. Incremental dual-memory lstm in land cover prediction. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining, pages 867-876. ACM, 2017.","title":"Incremental dual-memory lstm in land cover prediction","authors":"Jia","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jia et al., 2017b] Xiaowei Jia, Ankush Khandelwal, Gu- ruprasad Nayak, James Gerber, Kimberly Carlson, Paul West, and Vipin Kumar. Predict land covers with transition modeling and incremental learning. In SIAM International Conference on Data Mining. SIAM, 2017.","title":"Predict land covers with transition modeling and incremental learning","authors":"Jia","year":"2017","venue":"SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jia et al., 2019a] Xiaowei Jia, Sheng Li, Ankush Khandel- wal, Guruprasad Nayak, Anuj Karpatne, and Vipin Ku- mar. Spatial context-aware networks for mining temporal discriminative period in land cover detection. In SIAM In- ternational Conference on Data mining. SIAM, 2019.","title":"Spatial context-aware networks for mining temporal discriminative period in land cover detection","authors":"Jia","year":"2019","venue":"SIAM International Conference on Data mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jia et al., 2019b] Xiaowei Jia, Guruprasad Nayak, Ankush Khandelwal, Anuj Karpatne, and Vipin Kumar. Classify- ing heterogeneous sequential data by cyclic domain adap- tation: An application in land cover detection. In SIAM International Conference on Data mining. SIAM, 2019.","title":"Classifying heterogeneous sequential data by cyclic domain adap-tation: An application in land cover detection","authors":"Jia","year":"2019","venue":"SIAM International Conference on Data mining","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jia et al., 2019c] Xiaowei Jia, Jared Willard, Anuj Karpatne, Jordan Read, Jacob Zwart, Michael Stein- bach, and Vipin Kumar. Physics guided rnns for modeling dynamical systems: A case study in simulating lake temperature profiles. In Proceedings of the 2019 SIAM In- ternational Conference on Data Mining, pages 558-566.","title":"Physics guided rnns for modeling dynamical systems: A case study in simulating lake temperature profiles","authors":"Jia","year":"2019","venue":"Proceedings of the 2019 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kurtz et al., 2012] Camille Kurtz, Nicolas Passat, Pierre Gancarski, and Anne Puissant. Extraction of complex pat- terns from multiresolution remote sensing images: A hi- erarchical top-down methodology. Pattern Recognition, 45(2):685-706, 2012.","title":"Extraction of complex patterns from multiresolution remote sensing images: A hierarchical top-down methodology","authors":"Camille Kurtz, Nicolas Passat, Pierre Gancarski, Anne Puissant","year":"2012","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lyu et al., 2016] Haobo Lyu, Hui Lu, and Lichao Mou. Learning a transferable change rule from a recurrent neural network for land cover change detection. Remote Sensing, 2016.","title":"Learning a transferable change rule from a recurrent neural network for land cover change detection","authors":"Lyu","year":"2016","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mathieu et al., 2015] Michael Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction be- yond mean square error. arXiv preprint:1511.05440, 2015.","title":"Deep multi-scale video prediction beyond mean square error","authors":"Michael Mathieu, Camille Couprie, Yann Lecun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Myint et al., 2011] Soe W Myint, Patricia Gober, Anthony Brazel, Susanne Grossman-Clarke, and Qihao Weng. Per- pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery. Remote sensing of environment, 115(5):1145-1161, 2011.","title":"Perpixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery","authors":"Patricia Soe W Myint, Anthony Gober, Susanne Brazel, Qihao Grossman-Clarke, Weng","year":"2011","venue":"Remote sensing of environment","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Waske and Braun, 2009] Björn Waske and Matthias Braun. Classifier ensembles for land cover mapping using multi- temporal sar imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 64(5):450-457, 2009.","title":"Classifier ensembles for land cover mapping using multitemporal sar imagery","authors":"Braun Waske, Björn Waske, Matthias Braun","year":"2009","venue":"ISPRS Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yang et al., 2018] Xi Yang, Yuan Zhang, and Min Chi. Time-aware subgroup matrix decomposition: Imputing missing data using forecasting events. In 2018 IEEE In- ternational Conference on Big Data. IEEE, 2018.","title":"Time-aware subgroup matrix decomposition: Imputing missing data using forecasting events","authors":"Xi Yang, Yuan Zhang, Min Chi","year":"2018","venue":"IEEE International Conference on Big Data","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2017a] Yihang Zhang, Peter M Atkinson, Xi- aodong Li, Feng Ling, Qunming Wang, and Yun Du. Learning-based spatial-temporal superresolution mapping of forest cover with modis images. IEEE Transactions on Geoscience and Remote Sensing, 55(1):600-614, 2017.","title":"Learning-based spatial-temporal superresolution mapping of forest cover with modis images","authors":"Zhang","year":"2017","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2017b] Yuan Zhang, Chen Lin, Min Chi, Julie Ivy, Muge Capan, and Jeanne M Huddleston. Lstm for septic shock: Adding unreliable labels to reliable predic- tions. In IEEE Big Data. IEEE, 2017.","title":"Lstm for septic shock: Adding unreliable labels to reliable predictions","authors":"Zhang","year":"2017","venue":"IEEE Big Data. IEEE","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhong et al., 2014] Liheng Zhong, Peng Gong, and Gre- gory S Biging. Efficient corn and soybean mapping with temporal extendability: A multi-year experiment using landsat imagery. RSE, 2014.","title":"Efficient corn and soybean mapping with temporal extendability: A multi-year experiment using landsat imagery","authors":"Zhong","year":"2014","venue":"RSE","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhong et al., 2016] Liheng Zhong, Lina Hu, Le Yu, Peng Gong, and Gregory S Biging. Automated mapping of soy- bean and corn using phenology. ISPRS Journal of Pho- togrammetry and Remote Sensing, 2016.","title":"Automated mapping of soybean and corn using phenology","authors":"Zhong","year":"2016","venue":"ISPRS Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}