{"type":"ScholarlyArticle","title":"Fairness and Representation in Satellite-Based Poverty Maps: Evidence of Urban-Rural Disparities and Their Impacts on Downstream Policy","authors":[{"name":"Emily Aiken","slug":"emily-aiken","given":"Emily","family":"Aiken","affiliation":"UC Berkeley , School of Information"},{"name":"Esther Rolf","slug":"esther-rolf","given":"Esther","family":"Rolf","affiliation":"Harvard University , Data Science Initiative & Center for Research on Computation and Society"},{"name":"Joshua Blumenstock","slug":"joshua-blumenstock","given":"Joshua","family":"Blumenstock","affiliation":"UC Berkeley , School of Information"}],"abstract":"Poverty maps derived from satellite imagery are increasingly used to inform high-stakes policy decisions, such as the allocation of humanitarian aid and the distribution of government resources. Such poverty maps are typically constructed by training machine learning algorithms on a relatively modest amount of ``ground truth\" data from surveys, and then predicting poverty levels in areas where imagery exists but surveys do not. Using survey and satellite data from ten countries, this paper investigates disparities in representation, systematic biases in prediction errors, and fairness concerns in satellite-based poverty mapping across urban and rural lines, and shows how these phenomena affect the validity of policies based on predicted maps. Our findings highlight the importance of careful error and bias analysis before using satellite-based poverty maps in real-world policy decisions.","doi":"10.24963/ijcai.2023/653","url":"https://ijcai.org/proceedings/2023/653","pdf":"https://ijcai.org/proceedings/2023/0653.pdf","markdown":"https://ijcai.org/proceedings/2023/653.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/653","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":653,"pages":{"first":5888,"last":5896},"keywords":["AI for Good: AI Ethics, Trust, Fairness","AI for Good: Multidisciplinary Topics and Applications"],"videos":[],"references":[{"position":1,"raw":"[Abbasi et al., 2019] Mohsen Abbasi, Sorelle A Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Fairness in representation: Quantifying stereotyping as a representational harm. In Proceedings of the 2019 SIAM International Conference on Data Mining, pages 801-809.","title":"Fairness in representation: Quantifying stereotyping as a representational harm","authors":"Mohsen Abbasi, A Sorelle, Carlos Friedler, Suresh Scheidegger, Venkatasubramanian","year":null,"venue":"Proceedings of the 2019 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aiken et al., 2022] Emily Aiken, Suzanne Bellue, Dean Karlan, Chris Udry, and Joshua E Blumenstock. Machine learning and phone data can improve targeting of humani- tarian aid. Nature, 603(7903):864-870, 2022.","title":"Machine learning and phone data can improve targeting of humanitarian aid","authors":"Emily Aiken, Suzanne Bellue, Dean Karlan, Chris Udry, Joshua E Blumenstock","year":"2022","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Asher et al., 2021] Sam Asher, Tobias Lunt, Ryu Matsuura, and Paul Novosad. Development research at high geo- graphic resolution: An analysis of night lights, firms, and poverty in India using the SHRUG open data platform. The World Bank Economic Review, 2021.","title":"Development research at high geographic resolution: An analysis of night lights, firms, and poverty in India using the SHRUG open data platform","authors":"Sam Asher, Tobias Lunt, Ryu Matsuura, Paul Novosad","year":"2021","venue":"The World Bank Economic Review","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blumenstock et al., 2015] Joshua Blumenstock, Gabriel Cadamuro, and Robert On. Predicting poverty and wealth from mobile phone metadata. Science, 350(6264):1073- 1076, 2015.","title":"Predicting poverty and wealth from mobile phone metadata","authors":"Joshua Blumenstock, Gabriel Cadamuro, Robert On","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Blumenstock, 2018] Joshua Blumenstock. Don't forget people in the use of big data for development. Nature, 561(7722), 2018.","title":"Don't forget people in the use of big data for development","authors":"Joshua Blumenstock, Blumenstock","year":"2018","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bolliger et al., 2017] Ian Bolliger, Tamma Carleton, Solomon Hsiang, Jonathan Kadish, Jonathan Proctor, Benjamin Recht, Esther Rolf, and Vaishaal Shankar. Ground control to Major Tom: The importance of field surveys in remotely sensed data analysis. arXiv preprint arXiv:1710.09342, 2017.","title":"Ground control to Major Tom: The importance of field surveys in remotely sensed data analysis","authors":"Bolliger","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Boyarskaya et al., 2020] Margarita Boyarskaya, Alexandra Olteanu, and Kate Crawford. Overcoming failures of imagination in AI infused system development and de- ployment. arXiv preprint arXiv:2011.13416, 2020.","title":"Overcoming failures of imagination in AI infused system development and deployment","authors":"Margarita Boyarskaya, Alexandra Olteanu, Kate Crawford","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Brown et al., 2018] Caitlin Brown, Martin Ravallion, and Dominique Van de Walle. A poor means test? Econo- metric targeting in Africa. Journal of Development Eco- nomics, 134:109-124, 2018.","title":"A poor means test? Econometric targeting in Africa","authors":"Caitlin Brown, Martin Ravallion, Dominique Van De Walle","year":"2018","venue":"Journal of Development Economics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Buolamwini and Gebru, 2018] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy dispari- ties in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77- 91. PMLR, 2018.","title":"Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification","authors":"Gebru Buolamwini","year":"2018","venue":"Conference on fairness, accountability and transparency","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Burke et al., 2021] Marshall Burke, Anne Driscoll, David B Lobell, and Stefano Ermon. Using satellite imagery to un- derstand and promote sustainable development. Science, 371(6535), 2021.","title":"Using satellite imagery to understand and promote sustainable development","authors":"Marshall Burke, Anne Driscoll, David B Lobell, Stefano Ermon","year":"2021","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Carleton et al., 2022] Tamma Carleton, Trinetta Chong, Hannah Druckenmiller, Eugenio Noda, Jonathan Proctor, Esther Rolf, and Solomon Hsiang. Multi-Task Observation Using Satellite Imagery and Kitchen Sinks (MOSAIKS) API. https://siml.berkeley.edu, 2022.","title":"Multi-Task Observation Using Satellite Imagery and Kitchen Sinks (MOSAIKS) API","authors":"Tamma Carleton, Trinetta Chong, Hannah Druckenmiller, Eugenio Noda, Jonathan Proctor, Esther Rolf, Solomon Hsiang","year":"2022","venue":null,"doi":null,"url":"https://siml.berkeley.edu","internal_url":null},{"position":12,"raw":"[Chi et al., 2022] Guanghua Chi, Han Fang, Sourav Chatter- jee, and Joshua E Blumenstock. Microestimates of wealth for all low-and middle-income countries. Proceedings of the National Academy of Sciences, 119(3):e2113658119, 2022.","title":"Microestimates of wealth for all low-and middle-income countries","authors":"Chi","year":"2022","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Chouldechova and G'Sell, 2017] Alexandra Chouldechova and Max G'Sell. Fairer and more accurate, but for whom? arXiv preprint arXiv:1707.00046, 2017.","title":"Alexandra Chouldechova and Max G'Sell. Fairer and more accurate, but for whom","authors":"G' Chouldechova, Sell","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Engstrom et al., 2022] Ryan Engstrom, Jonathan Hersh, and David Newhouse. Poverty from space: Using high resolution satellite imagery for estimating economic well- being. The World Bank Economic Review, 36(2):382-412, 2022.","title":"Poverty from space: Using high resolution satellite imagery for estimating economic wellbeing","authors":"Ryan Engstrom, Jonathan Hersh, David Newhouse","year":"2022","venue":"The World Bank Economic Review","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Fatehkia et al., 2020] Masoomali Fatehkia, Isabelle Tingzon, Ardie Orden, Stephanie Sy, Vedran Sekara, Manuel Garcia-Herranz, and Ingmar Weber. Mapping socioeconomic indicators using social media advertising data. EPJ Data Science, 9(1):22, 2020.","title":"Mapping socioeconomic indicators using social media advertising data","authors":"Fatehkia","year":"2020","venue":"EPJ Data Science","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gentilini et al., 2022] Ugo Gentilini, Mohamed Bubaker Alsafi Almenfi, TMM Iyengar, Yuko Oka- mura, John Austin Downes, Pamela Dale, Michael Weber, David Locke Newhouse, Claudia P Rodriguez Alas, Ma- reeha Kamran, et al. Social protection and jobs responses to COVID-19. 2022.","title":"Social protection and jobs responses to COVID-19","authors":"Ugo Gentilini, Mohamed Bubaker, Alsafi Almenfi, Tmm Iyengar, Yuko Okamura, John Austin Downes, Pamela Dale, Michael Weber, David Locke Newhouse, Claudia P Rodriguez Alas, Mareeha Kamran","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ghosh and Roy, 1997] Rabindra Nath Ghosh and KC Roy. The changing status of women in India: Impact of urban- ization and development. International Journal of Social Economics, 24(7/8/9):902-917, 1997.","title":"The changing status of women in India: Impact of urbanization and development","authors":"Roy ; Rabindra Ghosh, Ghosh Nath, Roy","year":"1997","venue":"International Journal of Social Economics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hofer et al., 2020] Martin Hofer, Tomas Sako, Arturo Mar- tinez Jr, Mildred Addawe, Joseph Bulan, Ron Lester Du- rante, and Marymell Martillan. Applying artificial intel- ligence on satellite imagery to compile granular poverty statistics. Asian Development Bank Economics Working Paper Series, (629), 2020.","title":"Applying artificial intelligence on satellite imagery to compile granular poverty statistics","authors":"Martin Hofer, Tomas Sako, Arturo Martinez, Mildred Addawe, Joseph Bulan, Ron Lester Durante, Marymell Martillan","year":"2020","venue":"Asian Development Bank Economics Working Paper Series","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Huang et al., 2021] Luna Yue Huang, Solomon M Hsiang, and Marco Gonzalez-Navarro. Using satellite imagery and deep learning to evaluate the impact of anti-poverty pro- grams. Technical report, National Bureau of Economic Research, 2021.","title":"Using satellite imagery and deep learning to evaluate the impact of anti-poverty programs","authors":"Luna Yue Huang, Solomon M Hsiang, Marco Gonzalez-Navarro","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Jean et al., 2016] Neal Jean, Marshall Burke, Michael Xie, W Matthew Davis, David B Lobell, and Stefano Ermon. Combining satellite imagery and machine learning to pre- dict poverty. Science, 353(6301):790-794, 2016.","title":"Combining satellite imagery and machine learning to predict poverty","authors":"Neal Jean, Marshall Burke, Michael Xie, Matthew Davis, David B Lobell, Stefano Ermon","year":"2016","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Jerven, 2013] Morten Jerven. Poor numbers. In Poor Num- bers. Cornell University Press, 2013.","title":"Poor numbers","authors":"Morten Jerven","year":"2013","venue":"Poor Numbers","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kondmann and Zhu, 2021] Lukas Kondmann and Xiao Xi- ang Zhu. Under the radar -auditing fairness in ML for humanitarian mapping. arXiv preprint arXiv:2108.02137, 2021.","title":"Under the radar -auditing fairness in ML for humanitarian mapping","authors":"Zhu Kondmann, Lukas Kondmann, Xiao Xiang, Zhu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kuper, 2013] Leo Kuper. Religion and urbanization in Africa. Reading in Race and Ethnic Relations: The Com- monwealth and International Library: Reading in Sociol- ogy, pages 129-148, 2013.","title":"Religion and urbanization in Africa","authors":"Leo Kuper, Kuper","year":"2013","venue":"Reading in Race and Ethnic Relations: The Commonwealth and International Library: Reading in Sociology","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lindert et al., 2020] Kathy Lindert, Tina George Karip- pacheril, Inés Rodríguez Caillava, and Kenichi Nishikawa Chávez. Sourcebook on the foundations of social protec- tion delivery systems. World Bank Publications, 2020.","title":"Sourcebook on the foundations of social protection delivery systems","authors":"Kathy Lindert, Tina George Karippacheril, Inés Rodríguez Caillava, Kenichi Nishikawa, Chávez","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liu et al., 2018] Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair ma- chine learning. In International Conference on Machine Learning, pages 3150-3158. PMLR, 2018.","title":"Delayed impact of fair machine learning","authors":"Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, Moritz Hardt","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":"/proceedings/2019/862"},{"position":26,"raw":"[McGovern et al., 2022] Amy McGovern, Imme Ebert- Uphoff, David John Gagne, and Ann Bostrom. Why we need to focus on developing ethical, responsible, and trust- worthy artificial intelligence approaches for environmental science. Environmental Data Science, 1:e6, 2022.","title":"Why we need to focus on developing ethical, responsible, and trustworthy artificial intelligence approaches for environmental science","authors":"Amy Mcgovern, Imme Ebert-Uphoff, David John Gagne, Ann Bostrom","year":"2022","venue":"Environmental Data Science","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Obermeyer et al., 2019] Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464):447-453, 2019.","title":"Dissecting racial bias in an algorithm used to manage the health of populations","authors":"Ziad Obermeyer, Brian Powers, Christine Vogeli, Sendhil Mullainathan","year":"2019","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Paul et al., 2021] Boban Varghese Paul, Chipo Msowoya, Edward Archibald, Massimo Sichinga, Alejandra Campero Peredo, and Muhammad Abdullah Ali Malik. Malawi COVID-19 urban cash intervention process evaluation report. 2021.","title":"Malawi COVID-19 urban cash intervention process evaluation report","authors":"Boban Varghese, Paul, Chipo Msowoya, Edward Archibald, Massimo Sichinga, Alejandra Campero Peredo, Muhammad Abdullah, Ali Malik","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Proctor et al., 2023] Jonathan Proctor, Tamma Carleton, and Sandy Sum. Parameter recovery using remotely sensed variables. Technical report, National Bureau of Economic Research, 2023.","title":"Parameter recovery using remotely sensed variables","authors":"Jonathan Proctor, Tamma Carleton, Sandy Sum","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Rao and Molina, 2015] John NK Rao and Isabel Molina. Small area estimation. John Wiley & Sons, 2015.","title":"Small area estimation","authors":"Molina ; John Nk Rao, Isabel Rao, Molina","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Ratledge et al., 2022] Nathan Ratledge, Gabe Cadamuro, Brandon de la Cuesta, Matthieu Stigler, and Marshall Burke. Using machine learning to assess the livelihood impact of electricity access. Nature, 611(7936):491-495, 2022.","title":"Using machine learning to assess the livelihood impact of electricity access","authors":"Nathan Ratledge, Gabe Cadamuro, Brandon De La Cuesta, Matthieu Stigler, Marshall Burke","year":"2022","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Rolf et al., 2021] Esther Rolf, Jonathan Proctor, Tamma Carleton, Ian Bolliger, Vaishaal Shankar, Miyabi Ishihara, Benjamin Recht, and Solomon Hsiang. A generalizable and accessible approach to machine learning with global satellite imagery. Nature communications, 12(1):1-11, 2021.","title":"A generalizable and accessible approach to machine learning with global satellite imagery","authors":"Esther Rolf, Jonathan Proctor, Tamma Carleton, Ian Bolliger, Vaishaal Shankar, Miyabi Ishihara, Benjamin Recht, Solomon Hsiang","year":"2021","venue":"Nature communications","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Rolf, 2023] Esther Rolf. Evaluation challenges for geospa- tial ml. arXiv preprint arXiv:2303.18087, 2023.","title":"Esther Rolf. Evaluation challenges for geospatial ml","authors":"Rolf","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Sherman et al., 2023] Luke Sherman, Jonathan Proctor, Hannah Druckenmiller, Heriberto Tapia, and Solomon M Hsiang. Global high-resolution estimates of the United Nations Human Development Index using satellite im- agery and machine-learning. Working Paper 31044, Na- tional Bureau of Economic Research, March 2023.","title":"Global high-resolution estimates of the United Nations Human Development Index using satellite imagery and machine-learning","authors":"Sherman","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Smythe and Blumenstock, 2022] Isabella S Smythe and Joshua E Blumenstock. Geographic microtargeting of social assistance with high-resolution poverty maps. Proceedings of the National Academy of Sciences, 119(32):e2120025119, 2022.","title":"Geographic microtargeting of social assistance with high-resolution poverty maps","authors":"S Isabella, Joshua E Smythe, Blumenstock","year":"2022","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Steele et al., 2017] Jessica E Steele, Pål Roe Sundsøy, Carla Pezzulo, Victor A Alegana, Tomas J Bird, Joshua Blu- menstock, Johannes Bjelland, Kenth Engø-Monsen, Yves- Alexandre De Montjoye, Asif M Iqbal, et al. Mapping poverty using mobile phone and satellite data. Journal of The Royal Society Interface, 14(127):20160690, 2017.","title":"Mapping poverty using mobile phone and satellite data","authors":"Jessica E Steele, Pål Roe Sundsøy, Carla Pezzulo, Tomas J Victor A Alegana, Joshua Bird, Johannes Blumenstock, Kenth Bjelland, Yves-Alexandre Engø-Monsen, Asif M De Montjoye, Iqbal","year":"2017","venue":"Journal of The Royal Society Interface","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Tingzon et al., 2019] Isabelle Tingzon, Ardie Orden, KT Go, S Sy, V Sekara, Ingmar Weber, M Fatehkia, M García-Herranz, and D Kim. Mapping poverty in the Philippines using machine learning, satellite imagery, and crowd-sourced geospatial information. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 2019.","title":"Mapping poverty in the Philippines using machine learning, satellite imagery, and crowd-sourced geospatial information","authors":"Isabelle Tingzon, Ardie Orden, S Go, V Sy, Ingmar Sekara, M Weber, M Fatehkia, García-Herranz, Kim","year":"2019","venue":"International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wadoux et al., 2021] Alexandre MJ-C Wadoux, Gerard BM Heuvelink, Sytze De Bruin, and Dick J Brus. Spatial cross- validation is not the right way to evaluate map accuracy. Ecological Modelling, 457:109692, 2021.","title":"Spatial crossvalidation is not the right way to evaluate map accuracy","authors":"Mj-C Alexandre, Wadoux, B M Gerard, Sytze Heuvelink, Dick J De Bruin, Brus","year":"2021","venue":"Ecological Modelling","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yeh et al., 2020] Christopher Yeh, Anthony Perez, Anne Driscoll, George Azzari, Zhongyi Tang, David Lobell, Ste- fano Ermon, and Marshall Burke. Using publicly avail- able satellite imagery and deep learning to understand economic well-being in Africa. Nature communications, 11(1):1-11, 2020.","title":"Using publicly available satellite imagery and deep learning to understand economic well-being in Africa","authors":"Christopher Yeh, Anthony Perez, Anne Driscoll, George Azzari, Zhongyi Tang, David Lobell, Stefano Ermon, Marshall Burke","year":"2020","venue":"Nature communications","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang et al., 2022] Miao Zhang, Harvineet Singh, Lazarus Chok, and Rumi Chunara. Segmenting across places: The need for fair transfer learning with satellite imagery. In 2022 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition Workshops (CVPRW), pages 2915-2924, 2022.","title":"Segmenting across places: The need for fair transfer learning with satellite imagery","authors":"Miao Zhang, Harvineet Singh, Lazarus Chok, Rumi Chunara","year":"2022","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}