{"type":"ScholarlyArticle","title":"Generating Interpretable Poverty Maps using Object Detection in Satellite Images","authors":[{"name":"Kumar Ayush","slug":"kumar-ayush","given":"Kumar","family":"Ayush","affiliation":"Department of Computer Science , Stanford University"},{"name":"Burak Uzkent","slug":"burak-uzkent","given":"Burak","family":"Uzkent","affiliation":"Department of Computer Science , Stanford University"},{"name":"Marshall Burke","slug":"marshall-burke","given":"Marshall","family":"Burke","affiliation":"Department of Earth System Science , Stanford University"},{"name":"David Lobell","slug":"david-lobell","given":"David","family":"Lobell","affiliation":"Department of Earth System Science , Stanford University"},{"name":"Stefano Ermon","slug":"stefano-ermon","given":"Stefano","family":"Ermon","affiliation":"Department of Computer Science , Stanford University"}],"abstract":"Accurate local-level poverty measurement is an essential task for governments and humanitarian organizations to track the progress towards improving livelihoods and distribute scarce resources. Recent computer vision advances in using satellite imagery to predict poverty have shown increasing accuracy, but they do not generate features that are interpretable to policymakers, inhibiting adoption by practitioners. Here we demonstrate an interpretable computational framework to accurately predict poverty at a local level by applying object detectors to high resolution (30cm) satellite images.  Using the weighted counts of objects as features,  we achieve  0.539 Pearson's r^2 in predicting village-level poverty in Uganda, a 31% improvement over existing (and less interpretable) benchmarks. Feature importance and ablation analysis reveal intuitive relationships between object counts and poverty predictions. Our results suggest that interpretability does not have to come at the cost of performance, at least in this important domain.","doi":"10.24963/ijcai.2020/608","url":"https://ijcai.org/proceedings/2020/608","pdf":"https://ijcai.org/proceedings/2020/0608.pdf","markdown":"https://ijcai.org/proceedings/2020/608.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/608","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Special track on AI for CompSust and Human well-being","track":"Special track on AI for CompSust and Human well-being","number":608,"pages":{"first":4410,"last":4416},"keywords":["Machine Learning: Deep Learning: Convolutional networks","Machine Learning: Interpretability","Machine Learning Applications: Applications of Supervised Learning","AI Ethics: Societal Impact of AI"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25268/file_25268.m3u8","length_sec":302,"provider":"hls"}],"references":[{"position":1,"raw":"[Assembly, 2015] General Assembly. Sustainable develop- ment goals. SDGs Transform Our World, 2030, 2015.","title":"General Assembly. Sustainable development goals. SDGs Transform Our World","authors":null,"year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blumenstock et al., 2015] Joshua Blumenstock, Gabriel Cadamuro, and Robert On. Predicting poverty and wealth from mobile phone metadata. Science, 350(6264):1073- 1076, 2015.","title":"Predicting poverty and wealth from mobile phone metadata","authors":"Joshua Blumenstock, Gabriel Cadamuro, Robert On","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Donaldson and Storeygard, 2016] Dave Donaldson and Adam Storeygard. The view from above: Applications of satellite data in economics. Journal of Economic Perspectives, 30(4):171-98, 2016.","title":"The view from above: Applications of satellite data in economics","authors":"Storeygard Donaldson, Dave Donaldson, Adam Storeygard","year":"2016","venue":"Journal of Economic Perspectives","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Doshi-Velez and Kim, 2017] Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608, 2017.","title":"Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning","authors":"Kim Doshi-Velez","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Engstrom et al., 2017] Ryan Engstrom, Jonathan Hersh, and David Newhouse. Poverty from space: Using high- resolution satellite imagery for estimating economic well- being, 2017.","title":"Poverty from space: Using highresolution satellite imagery for estimating economic wellbeing","authors":"Ryan Engstrom, Jonathan Hersh, David Newhouse","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Filmer and Pritchett, 2001] Deon Filmer and Lant H Pritch- ett. Estimating wealth effects without expenditure data-or tears: an application to educational enrollments in states of india. Demography, 38(1):115-132, 2001.","title":"Estimating wealth effects without expenditure data-or tears: an application to educational enrollments in states of india","authors":"Deon Filmer, H Lant, Pritchett","year":"2001","venue":"Demography","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fu et al., 2017] Cheng-Yang Fu, Wei Liu, Ananth Ranga, Ambrish Tyagi, and Alexander C Berg. Dssd: De- convolutional single shot detector. arXiv preprint arXiv:1701.06659, 2017.","title":"Dssd: Deconvolutional single shot detector","authors":"Cheng-Yang Fu, Wei Liu, Ananth Ranga, Ambrish Tyagi, Alexander C Berg","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gebru et al., 2017] Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, Erez Lieberman Aiden, and Li Fei-Fei. Using deep learning and google street view to estimate the demographic makeup of neighbor- hoods across the united states. Proceedings of the National Academy of Sciences, 114(50):13108-13113, 2017.","title":"Using deep learning and google street view to estimate the demographic makeup of neighborhoods across the united states","authors":"Gebru","year":"2017","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jean et al., 2016] Neal Jean, Marshall Burke, Michael Xie, W Matthew Davis, David B Lobell, and Stefano Ermon. Combining satellite imagery and machine learning to pre- dict poverty. Science, 353(6301):790-794, 2016.","title":"Combining satellite imagery and machine learning to predict poverty","authors":"Neal Jean, Marshall Burke, Michael Xie, Matthew Davis, David B Lobell, Stefano Ermon","year":"2016","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jerven, 2017] Morten Jerven. How much will a data revo- lution in development cost? In Forum for Development Studies, volume 44, pages 31-50.","title":"How much will a data revolution in development cost?","authors":"Morten Jerven, Jerven","year":"2017","venue":"Forum for Development Studies","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Taylor & Francis, 2017.","title":null,"authors":"Francis Taylor","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lai et al., 2011] Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox. A large-scale hierarchical multi-view rgb-d object dataset. In 2011 IEEE international conference on robotics and automation, pages 1817-1824. IEEE, 2011.","title":"A large-scale hierarchical multi-view rgb-d object dataset","authors":"Kevin Lai, Liefeng Bo, Xiaofeng Ren, Dieter Fox","year":"2011","venue":"IEEE international conference on robotics and automation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lam et al., 2018] Darius Lam, Richard Kuzma, Kevin McGee, Samuel Dooley, Michael Laielli, Matthew Klaric, Yaroslav Bulatov, and Brendan McCord. xview: Ob- jects in context in overhead imagery. arXiv preprint arXiv:1802.07856, 2018.","title":"xview: Objects in context in overhead imagery","authors":"Darius Lam, Richard Kuzma, Kevin Mcgee, Samuel Dooley, Michael Laielli, Matthew Klaric, Yaroslav Bulatov, Brendan Mccord","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2017a] Tsung-Yi Lin, Piotr Dollár, Ross Gir- shick, Kaiming He, Bharath Hariharan, and Serge Be- longie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117-2125, 2017.","title":"Feature pyramid networks for object detection","authors":"Lin","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2017b] Tsung-Yi Lin, Priya Goyal, Ross Gir- shick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Lin","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2016] Wei Liu, Dragomir Anguelov, Dumitru Er- han, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European conference on computer vision, pages 21-37. Springer, 2016.","title":"Ssd: Single shot multibox detector","authors":"Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C Berg","year":"2016","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lundberg and Lee, 2017] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fer- gus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 4765- 4774.","title":"A unified approach to interpreting model predictions","authors":"M Scott, Su-In Lundberg, Lee","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lundberg et al., 2018] Scott M Lundberg, Gabriel G Erion, and Su-In Lee. Consistent individualized feature attribu- tion for tree ensembles. arXiv preprint arXiv:1802.03888, 2018.","title":"Consistent individualized feature attribution for tree ensembles","authors":"Scott M Lundberg, Su-In Gabriel G Erion, Lee","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Murdoch et al., 2019] W James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. Definitions, methods, and applications in interpretable machine learn- ing. Proceedings of the National Academy of Sciences, 116(44):22071-22080, 2019.","title":"Definitions, methods, and applications in interpretable machine learning","authors":"James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, Bin Yu","year":"2019","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Perez et al., 2017] Anthony Perez, Christopher Yeh, George Azzari, Marshall Burke, David Lobell, and Stefano Er- mon. Poverty prediction with public landsat 7 satel- lite imagery and machine learning. arXiv preprint arXiv:1711.03654, 2017.","title":"Poverty prediction with public landsat 7 satellite imagery and machine learning","authors":"Anthony Perez, Christopher Yeh, George Azzari, Marshall Burke, David Lobell, Stefano Ermon","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Redmon and Farhadi, 2018] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018.","title":"Yolov3: An incremental improvement","authors":"Joseph Redmon, Ali Farhadi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sheehan et al., 2019] Evan Sheehan, Chenlin Meng, Matthew Tan, Burak Uzkent, Neal Jean, Marshall Burke, David Lobell, and Stefano Ermon. Predicting economic development using geolocated wikipedia articles. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2698-2706, 2019.","title":"Predicting economic development using geolocated wikipedia articles","authors":"Evan Sheehan, Chenlin Meng, Matthew Tan, Burak Uzkent, Neal Jean, Marshall Burke, David Lobell, Stefano Ermon","year":"2019","venue":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shrivastava et al., 2016] Abhinav Shrivastava, Rahul Suk- thankar, Jitendra Malik, and Abhinav Gupta. Beyond skip connections: Top-down modulation for object detection. arXiv preprint arXiv:1612.06851, 2016.","title":"Beyond skip connections: Top-down modulation for object detection","authors":"Abhinav Shrivastava, Rahul Sukthankar, Jitendra Malik, Abhinav Gupta","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[UBOS, 2012] Uganda Bureau of Statistics UBOS. Uganda national panel survey 2011/2012. Uganda, 2012.","title":"Uganda Bureau of Statistics UBOS. Uganda national panel survey","authors":"Ubos","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}