{"type":"ScholarlyArticle","title":"Find Rhinos without Finding Rhinos: Active Learning with Multimodal Imagery of South African Rhino Habitats","authors":[{"name":"Lucia Gordon","slug":"lucia-gordon","given":"Lucia","family":"Gordon","affiliation":"Harvard University"},{"name":"Nikhil Behari","slug":"nikhil-behari","given":"Nikhil","family":"Behari","affiliation":"Masssachusetts Institute of Technology"},{"name":"Samuel Collier","slug":"samuel-collier","given":"Samuel","family":"Collier","affiliation":"Harvard University"},{"name":"Elizabeth Bondi-Kelly","slug":"elizabeth-bondi-kelly","given":"Elizabeth","family":"Bondi-Kelly","affiliation":"Masssachusetts Institute of Technology"},{"name":"Jackson A. Killian","slug":"jackson-a-killian","given":"Jackson A.","family":"Killian","affiliation":"Harvard University"},{"name":"Catherine Ressijac","slug":"catherine-ressijac","given":"Catherine","family":"Ressijac","affiliation":"Harvard University"},{"name":"Peter Boucher","slug":"peter-boucher","given":"Peter","family":"Boucher","affiliation":"Harvard University"},{"name":"Andrew Davies","slug":"andrew-davies","given":"Andrew","family":"Davies","affiliation":"Harvard University"},{"name":"Milind Tambe","slug":"milind-tambe","given":"Milind","family":"Tambe","affiliation":"Harvard University"}],"abstract":"Much of Earth's charismatic megafauna is endangered by human activities, particularly the rhino, which is at risk of extinction due to the poaching crisis in Africa. Monitoring rhinos' movement is crucial to their protection but has unfortunately proven difficult because rhinos are elusive. Therefore, instead of tracking rhinos, we propose the novel approach of mapping communal defecation sites, called middens, which give information about rhinos' spatial behavior valuable to anti-poaching, management, and reintroduction efforts. This paper provides the first-ever mapping of rhino midden locations by building classifiers to detect them using remotely sensed thermal, RGB, and LiDAR imagery in passive and active learning settings. As existing active learning methods perform poorly due to the extreme class imbalance in our dataset, we design MultimodAL, an active learning system employing a ranking technique and multimodality to achieve competitive performance with passive learning models with 94% fewer labels. Our methods could therefore save over 76 hours in labeling time when used on a similarly-sized dataset. Unexpectedly, our midden map reveals that rhino middens are not randomly distributed throughout the landscape; rather, they are clustered. Consequently, rangers should be targeted at areas with high midden densities to strengthen anti-poaching efforts, in line with UN Target 15.7.","doi":"10.24963/ijcai.2023/663","url":"https://ijcai.org/proceedings/2023/663","pdf":"https://ijcai.org/proceedings/2023/0663.pdf","markdown":"https://ijcai.org/proceedings/2023/663.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/663","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":663,"pages":{"first":5977,"last":5985},"keywords":["AI for Good: Computer Vision","AI for Good: Machine Learning","AI for Good: Multidisciplinary Topics and Applications","AI for Good: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Alexander et al., 2022] Quincy Alexander, Vedhus Hoskere, Yasutaka Narazaki, Andrew Maxwell, and Billie Spencer. Fusion of thermal and RGB images for automated deep learning based crack detection in civil infrastructure. AI in Civil Engineering, 1(3), 2022.","title":"Fusion of thermal and RGB images for automated deep learning based crack detection in civil infrastructure","authors":"Alexander","year":"2022","venue":"AI in Civil Engineering","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anderson and Gaston, 2013] Karen Anderson and Kevin J Gaston. Lightweight unmanned aerial vehicles will rev- olutionize spatial ecology. Frontiers in Ecology and the Environment, 11(3):138-146, 2013.","title":"Lightweight unmanned aerial vehicles will revolutionize spatial ecology","authors":"Karen Anderson, Kevin J Gaston","year":"2013","venue":"Frontiers in Ecology and the Environment","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beery et al., 2018] Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.","title":"Recognition in terra incognita","authors":"Sara Beery, Grant Van Horn, Pietro Perona","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ceballos et al., 2015] Gerardo Ceballos, Paul R. Ehrlich, Anthony D. Barnosky, Andrés García, Robert M. Pringle, and Todd M. Palmer. Accelerated modern human-induced species losses: Entering the sixth mass extinction. Science Advances, 1(5):e1400253, 2015.","title":"Accelerated modern human-induced species losses: Entering the sixth mass extinction","authors":"Gerardo Ceballos, Paul R Ehrlich, Anthony D Barnosky, Andrés García, Robert M Pringle, Todd M Palmer","year":"2015","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[De Oliveira and Wehrmeister, 2016] Diulhio Candido De Oliveira and Marco Aurelio Wehrmeister. Towards real-time people recognition on aerial imagery using convolutional neural networks. In 2016 IEEE 19th Inter- national Symposium on Real-Time Distributed Computing (ISORC), pages 27-34, 2016.","title":"Diulhio Candido De Oliveira and Marco Aurelio Wehrmeister. Towards real-time people recognition on aerial imagery using convolutional neural networks","authors":"De Oliveira, Wehrmeister","year":"2016","venue":"IEEE 19th International Symposium on Real-Time Distributed Computing (ISORC)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ferreira et al., 2015] Sam M Ferreira, Cathy Greaver, Grant A Knight, Mike H Knight, Izak PJ Smit, and Danie Pienaar. Disruption of rhino demography by poachers may lead to population declines in Kruger National Park, South Africa. PLoS One, 10(6):e0127783, 2015.","title":"Disruption of rhino demography by poachers may lead to population declines in Kruger National Park, South Africa","authors":"Cathy Sam M Ferreira, Grant A Greaver, Mike H Knight, Izak Pj Knight, Danie Smit, Pienaar","year":"2015","venue":"PLoS One","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Johnson and Gillingham, 2005] Chris J. Johnson and Michael P. Gillingham. An evaluation of mapped species distribution models used for conservation planning. Environmental Conservation, 32(2):117-128, 2005.","title":"An evaluation of mapped species distribution models used for conservation planning","authors":"Chris J Johnson, Michael P Gillingham","year":"2005","venue":"Environmental Conservation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kellenberger et al., 2019] Benjamin Kellenberger, Diego Marcos, Sylvain Lobry, and Devis Tuia. Half a percent of labels is enough: Efficient animal detection in UAV im- agery using deep CNNs and active learning. IEEE Trans- actions on Geoscience and Remote Sensing, 57(12):9524- 9533, 2019.","title":"Half a percent of labels is enough: Efficient animal detection in UAV imagery using deep CNNs and active learning","authors":"Benjamin Kellenberger, Diego Marcos, Sylvain Lobry, Devis Tuia","year":"2019","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kirillov et al., 2023] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment Anything. https://arxiv.org/pdf/2304.02643.pdf, 2023. Accessed: 2023-05-24.","title":"Segment Anything","authors":"Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, Piotr Dollár, Ross Girshick","year":"2023","venue":null,"doi":null,"url":"https://arxiv.org/pdf/2304.02643.pdf","internal_url":null},{"position":10,"raw":"[Lewis and Gale, 1994] David D. Lewis and William A. Gale. A sequential algorithm for training text classifiers. In Bruce W. Croft and C. J. van Rijsbergen, editors, SIGIR '94, pages 3-12, London, 1994. Springer London.","title":"A sequential algorithm for training text classifiers","authors":"David D Lewis, William A Gale","year":"1994","venue":"SIGIR '94","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Linklater et al., 2013] Wayne L Linklater, Katharina Mayer, and Ronald R Swaisgood. Chemical signals of age, sex and identity in black rhinoceros. Animal behaviour, 85(3):671-677, 2013.","title":"Chemical signals of age, sex and identity in black rhinoceros","authors":"Katharina Wayne L Linklater, Ronald R Mayer, Swaisgood","year":"2013","venue":"Animal behaviour","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Marneweck, 2013] Courtney Marneweck. What is a Rhino Midden? https://www.wildlifeact.com/blog/ what-is-a-rhino-midden/, 2013. Accessed: 2023-05-24.","title":"What is a Rhino Midden?","authors":"Courtney Marneweck","year":"2023","venue":null,"doi":null,"url":"https://www.wildlifeact.com/blog/what-is-a-rhino-midden/,2013","internal_url":null},{"position":13,"raw":"[Marvin et al., 2016] David C. Marvin, Lian Pin Koh, Antony J. Lynam, Serge Wich, Andrew B. Davies, Ramesh Krishnamurthy, Emma Stokes, Ruth Starkey, and Gre- gory P. Asner. Integrating technologies for scalable ecol- ogy and conservation. Global Ecology and Conservation, 7:262-275, 2016.","title":"Integrating technologies for scalable ecology and conservation","authors":"David C Marvin, Lian Pin Koh, Antony J Lynam, Serge Wich, Andrew B Davies, Ramesh Krishnamurthy, Emma Stokes, Ruth Starkey, Gregory P Asner","year":"2016","venue":"Global Ecology and Conservation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Owen-Smith and Smith, 1973] Rupert Norman Owen- Smith and Rupert Norman Owen Smith. The behavioural ecology of the white rhinoceros. PhD thesis, University of Wisconsin Madison, 1973.","title":"Rupert Norman Owen-Smith and Rupert Norman Owen Smith. The behavioural ecology of the white rhinoceros","authors":"Owen-Smith, Smith","year":"1973","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Settles, 2009] Burr Settles. Active learning literature sur- vey. https://burrsettles.com/pub/settles.activelearning.pdf, 2009. Accessed: 2023-05-24.","title":"Burr Settles. Active learning literature survey","authors":"Settles","year":"2009","venue":null,"doi":null,"url":"https://burrsettles.com/pub/settles.activelearning.pdf","internal_url":null},{"position":16,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. https://arxiv.org/pdf/1409. 1556.pdf, 2015. Accessed: 2023-05-24.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":null,"doi":null,"url":"https://arxiv.org/pdf/1409.1556.pdf","internal_url":null},{"position":17,"raw":"[Speth et al., 2022] Simon Speth, Artur Gonc ¸alves, Bastien Rigault, Satoshi Suzuki, Mondher Bouazizi, Yutaka Mat- suo, and Helmut Prendinger. Deep learning with RGB and thermal images onboard a drone for monitoring op- erations. Journal of Field Robotics, 39(6):840-868, 2022.","title":"Deep learning with RGB and thermal images onboard a drone for monitoring operations","authors":"Simon Speth, Artur Gonc ¸alves, Bastien Rigault, Satoshi Suzuki, Mondher Bouazizi, Yutaka Matsuo, Helmut Prendinger","year":"2022","venue":"Journal of Field Robotics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Save the Rhino International, 2022] Save the Rhino International. Poaching statistics. https://www.savetherhino.org/rhino-info/poaching-stats/, 2022. Accessed: 2023-05-24.","title":"Save the Rhino International. Poaching statistics","authors":null,"year":"2022","venue":null,"doi":null,"url":"https://www.savetherhino.org/rhino-info/poaching-stats/,2022","internal_url":null},{"position":19,"raw":"[Tuia et al., 2022] Devis Tuia, Benjamin Kellenberger, Sara Beery, Blair R. Costelloe, Silvia Zuffi, Benjamin Risse, Alexander Mathis, Mackenzie W. Mathis, Frank van Langevelde, Tilo Burghardt, and et al. Perspectives in ma- chine learning for wildlife conservation, Feb 2022.","title":"Perspectives in machine learning for wildlife conservation","authors":"Devis Tuia, Benjamin Kellenberger, Sara Beery, Blair R Costelloe, Silvia Zuffi, Benjamin Risse, Alexander Mathis, Mackenzie W Mathis, Frank Van Langevelde, Tilo Burghardt","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[UN, 2023a] Sustainable Development Group UN. Leave No One Behind. https://unsdg.un.org/2030-agenda/ universal-values/leave-no-one-behind, 2023. Accessed: 2023-05-24.","title":"Sustainable Development Group UN. Leave No One Behind","authors":null,"year":"2023","venue":null,"doi":null,"url":"https://unsdg.un.org/2030-agenda/universal-values/leave-no-one-behind,2023","internal_url":null},{"position":21,"raw":"[UN, 2023b] Department of Economic and Social Affairs UN. Goal 15. https://sdgs.un.org/goals/goal15, 2023. Accessed: 2023-05-24.","title":"Department of Economic and Social Affairs UN","authors":null,"year":"2023","venue":null,"doi":null,"url":"https://sdgs.un.org/goals/goal15,2023","internal_url":null},{"position":22,"raw":"[WWF, 2023] WWF. Rhino. https://www.worldwildlife.org/ species/rhino, 2023. Accessed: 2023-05-24.","title":"WWF. Rhino","authors":null,"year":"2023","venue":null,"doi":null,"url":"https://www.worldwildlife.org/species/rhino,2023","internal_url":null},{"position":23,"raw":"[Zhang et al., 2021] Heng Zhang, Elisa Fromont, Sébastien Lefevre, and Bruno Avignon. Deep active learning from multispectral data through cross-modality prediction in- consistency. In 2021 IEEE International Conference on Image Processing (ICIP), pages 449-453, 2021.","title":"Deep active learning from multispectral data through cross-modality prediction inconsistency","authors":"Heng Zhang, Elisa Fromont, Sébastien Lefevre, Bruno Avignon","year":"2021","venue":"IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}