{"type":"ScholarlyArticle","title":"Long-term Monitoring of Bird Flocks in the Wild","authors":[{"name":"Kshitiz","slug":"kshitiz","given":"","family":"Kshitiz","affiliation":"IIT Jodhpur , India"},{"name":"Sonu Shreshtha","slug":"sonu-shreshtha","given":"Sonu","family":"Shreshtha","affiliation":"IIT Jodhpur , India"},{"name":"Ramy Mounir","slug":"ramy-mounir","given":"Ramy","family":"Mounir","affiliation":"University of South Florida , Tampa , Florida , USA"},{"name":"Mayank Vatsa","slug":"mayank-vatsa","given":"Mayank","family":"Vatsa","affiliation":"IIT Jodhpur , India"},{"name":"Richa Singh","slug":"richa-singh","given":"Richa","family":"Singh","affiliation":"IIT Jodhpur , India"},{"name":"Saket Anand","slug":"saket-anand","given":"Saket","family":"Anand","affiliation":"IIIT Delhi , India"},{"name":"Sudeep Sarkar","slug":"sudeep-sarkar","given":"Sudeep","family":"Sarkar","affiliation":"University of South Florida , Tampa , Florida , USA"},{"name":"Sevaram Mali Parihar","slug":"sevaram-mali-parihar","given":"Sevaram","family":"Mali Parihar","affiliation":"Crane Conservationist , Khichan , India"}],"abstract":"Monitoring and analysis of wildlife are key to conservation planning and conflict management. The widespread use of camera traps coupled with AI-based analysis tools serves as an excellent example of successful and non-invasive use of technology for design, planning, and evaluation of conservation policies. As opposed to the typical use of camera traps that capture still images or short videos, in this project, we propose to analyze longer term videos monitoring a large flock of birds. This project, which is part of the NSF-TIH Indo-US joint R&D partnership, focuses on solving challenges associated with the analysis of long-term videos captured at feeding grounds and nesting sites, among other such locations that host large flocks of migratory birds. We foresee that the objectives of this project would lead to datasets and benchmarking tools as well as novel algorithms that would be instrumental in developing automated video analysis tools that could in turn help understand individual and social behavior of birds. The first of the key outcomes of this research will include the curation of challenging, real-world datasets for benchmarking various image and video analytics algorithms for tasks such as counting, detection, segmentation, and tracking. Our recent efforts towards this outcome is a curated dataset of 812 high-resolution, point-annotated, images (4K - 32MP) of a flock of Demoiselle cranes (Anthropoides virgo) taken from their feeding site at Khichan, Rajasthan, India. The average number of birds in each image is about 207, with a maximum count of 1500. The benchmark experiments show that state-of-the-art vision techniques struggle with tasks such as segmentation, detection, localization, and density estimation for the proposed dataset. Over the execution of this open science research, we will be scaling this dataset for segmentation and tracking in videos, as well as developing novel techniques for video analytics for wildlife monitoring.","doi":"10.24963/ijcai.2023/704","url":"https://ijcai.org/proceedings/2023/704","pdf":"https://ijcai.org/proceedings/2023/0704.pdf","markdown":"https://ijcai.org/proceedings/2023/704.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/704","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good - Projects","track":"AI for Good - Projects","number":704,"pages":{"first":6344,"last":6352},"keywords":["AI for Good - Projects: Machine Learning","AI for Good - Projects: Computer Vision","AI for Good - Projects: Multidisciplinary Topics and Applications"],"videos":[],"references":[{"position":1,"raw":"[Beery et al., 2019] Sara Beery, Dan Morris, and Siyu Yang. Efficient pipeline for camera trap image review. arXiv preprint arXiv:1907.06772, 2019.","title":"Efficient pipeline for camera trap image review","authors":"Sara Beery, Dan Morris, Siyu Yang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Computer Vision-ECCV 2020, pages 213-229, 2020.","title":"End-to-end object detection with transformers","authors":"Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko","year":"2020","venue":"Computer Vision-ECCV 2020","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2011] I-Ching Chen, Jane K Hill, Ralf Ohlemüller, David B Roy, and Chris D Thomas. Rapid range shifts of species associated with high levels of climate warming. Science, 333(6045):1024-1026, 2011.","title":"Rapid range shifts of species associated with high levels of climate warming","authors":"Chen","year":"2011","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Delany, 2010] Simon Delany. Guidance on waterbird moni- toring methodology: field protocol for waterbird counting. Wetlands international, 25:4-5, 2010.","title":"Guidance on waterbird monitoring methodology: field protocol for waterbird counting","authors":"Simon Delany","year":"2010","venue":"Wetlands international","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[for Conservation of Nature (IUCN), 2010] International Union for Conservation of Nature (IUCN). Iucn red list of threatened species. version 2010.2, 2010.","title":"International Union for Conservation of Nature (IUCN). Iucn red list of threatened species","authors":null,"year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Galtbalt et al., 2022] Batbayar Galtbalt, Nyambayar Batba- yar, Tuvshintugs Sukhbaatar, Bernd Vorneweg, Georg Heine, Uschi Müller, Martin Wikelski, and Marcel Klaassen. Differences in on-ground and aloft conditions explain seasonally different migration paths in demoiselle crane. Movement Ecology, 10(1):1-11, 2022.","title":"Differences in on-ground and aloft conditions explain seasonally different migration paths in demoiselle crane","authors":"Galtbalt","year":"2022","venue":"Movement Ecology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.","title":"Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn","authors":"He","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Higuchi, 2012] Hiroyoshi Higuchi. Bird migration and the conservation of the global environment. Journal of Or- nithology, 153(Suppl 1):3-14, 2012.","title":"Bird migration and the conservation of the global environment","authors":"Hiroyoshi Higuchi, Higuchi","year":"2012","venue":"Journal of Ornithology","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim and Kim, 2020] Saehun Kim and Munchurl Kim. Learning of counting crowded birds of various scales via novel density activation maps. IEEE Access, 8:155296- 155305, 2020.","title":"Learning of counting crowded birds of various scales via novel density activation maps","authors":"Kim ; Kim, Saehun Kim, Munchurl Kim","year":"2020","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2018] Yuhong Li, Xiaofan Zhang, and Deming Chen. Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes. In Proceed- ings of the IEEE conference on computer vision and pat- tern recognition, pages 1091-1100, 2018.","title":"Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes","authors":"Yuhong Li, Xiaofan Zhang, Deming Chen","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2019] Weizhe Liu, Mathieu Salzmann, and Pas- cal Fua. Context-aware crowd counting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5099-5108, 2019.","title":"Context-aware crowd counting","authors":"Liu","year":"2019","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mounir et al., 2022] Ramy Mounir, Sathyanarayanan Aakur, and Sudeep Sarkar. Self-supervised temporal event segmentation inspired by cognitive theories. In Advanced Methods and Deep Learning in Computer Vision, pages 405-448. Elsevier, 2022.","title":"Self-supervised temporal event segmentation inspired by cognitive theories","authors":"Ramy Mounir, Sathyanarayanan Aakur, Sudeep Sarkar","year":"2022","venue":"Advanced Methods and Deep Learning in Computer Vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mounir et al., 2023] Ramy Mounir, Ahmed Shahabaz, Ro- man Gula, Jörn Theuerkauf, and Sudeep Sarkar. Towards automated ethogramming: Cognitively-inspired event seg- mentation for streaming wildlife video monitoring. Inter- national journal of computer vision, pages 1-31, 2023.","title":"Towards automated ethogramming: Cognitively-inspired event segmentation for streaming wildlife video monitoring","authors":"Ramy Mounir, Ahmed Shahabaz, Roman Gula, Jörn Theuerkauf, Sudeep Sarkar","year":"2023","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nichols and Williams, 2006] James D Nichols and Byron K Williams. Monitoring for conservation. Trends in ecology & evolution, 21(12):668-673, 2006.","title":"Monitoring for conservation","authors":"Williams ; Nichols, D James, Byron K Nichols, Williams","year":"2006","venue":"Trends in ecology & evolution","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Norouzzadeh et al., 2021] Mohammad Sadegh Norouz- zadeh, Dan Morris, Sara Beery, Neel Joshi, Nebojsa Jojic, and Jeff Clune. A deep active learning system for species identification and counting in camera trap images. Methods in ecology and evolution, 12(1):150-161, 2021.","title":"A deep active learning system for species identification and counting in camera trap images","authors":"Mohammad Sadegh Norouzzadeh, Dan Morris, Sara Beery, Neel Joshi, Nebojsa Jojic, Jeff Clune","year":"2021","venue":"Methods in ecology and evolution","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pantazis et al., 2021] Omiros Pantazis, Gabriel J Brostow, Kate E Jones, and Oisin Mac Aodha. Focus on the pos- itives: Self-supervised learning for biodiversity monitor- ing. In Proceedings of the IEEE/CVF International con- ference on computer vision, pages 10583-10592, 2021.","title":"Focus on the positives: Self-supervised learning for biodiversity monitoring","authors":"Omiros Pantazis, J Gabriel, Kate E Brostow, Oisin Jones, Aodha Mac","year":"2021","venue":"Proceedings of the IEEE/CVF International conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention-MICCAI 2015, pages 234-241, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"Medical image computing and computer-assisted intervention-MICCAI 2015","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Savner and Kanhangad, 2023] Siddharth Singh Savner and Vivek Kanhangad. Crowdformer: Weakly-supervised crowd counting with improved generalizability. Journal of Visual Communication and Image Representation, page 103853, 2023.","title":"Crowdformer: Weakly-supervised crowd counting with improved generalizability","authors":"Siddharth Singh, Savner, Vivek Kanhangad","year":"2023","venue":"Journal of Visual Communication and Image Representation","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[S ¸ekercioglu et al., 2004] C ¸agan H S ¸ekercioglu, Gretchen C Daily, and Paul R Ehrlich. Ecosystem consequences of bird declines. Proceedings of the National Academy of Sciences, 101(52):18042-18047, 2004.","title":"Ecosystem consequences of bird declines","authors":"Gretchen C C ¸agan H S ¸ekercioglu, Paul R Daily, Ehrlich","year":"2004","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Van Horn et al., 2015] Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis, Pietro Perona, and Serge Belongie. Building a bird recog- nition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pages 595-604, 2015.","title":"Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection","authors":"Van Horn","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. Technical Report CNS-TR- 2011-001, California Institute of Technology, 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang and Breckon, 2022] Qian Wang and Toby P Breckon. Crowd counting via segmentation guided attention net- works and curriculum loss. IEEE Transactions on Intel- ligent Transportation Systems, 23(9):15233-15243, 2022.","title":"Crowd counting via segmentation guided attention networks and curriculum loss","authors":"Qian Wang, Toby P Breckon","year":"2022","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2020] Boyu Wang, Huidong Liu, Dimitris Samaras, and Minh Hoai Nguyen. Distribution matching for crowd counting. Advances in neural information pro- cessing systems, 33:1595-1607, 2020.","title":"Distribution matching for crowd counting","authors":"Boyu Wang, Huidong Liu, Dimitris Samaras, Minh Hoai Nguyen","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zakaria et al., 2005] Mohamed Zakaria, Puan Chong Leong, and Muhammad Ezhar Yusuf. Comparison of species composition in three forest types: Towards using bird as indicator of forest ecosystem health. Journal of biological sciences, 5(6):734-737, 2005.","title":"Comparison of species composition in three forest types: Towards using bird as indicator of forest ecosystem health","authors":"Mohamed Zakaria, Puan, Chong Leong, Muhammad Ezhar Yusuf","year":"2005","venue":"Journal of biological sciences","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2016] Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, and Yi Ma. Single-image crowd counting via multi-column convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 589-597, 2016.","title":"Single-image crowd counting via multi-column convolutional neural network","authors":"Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, Yi Ma","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}