{"type":"ScholarlyArticle","title":"SiamBOMB: A Real-time AI-based System for Home-cage Animal Tracking, Segmentation and Behavioral Analysis","authors":[{"name":"Xi Chen","slug":"xi-chen","given":"Xi","family":"Chen","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Hao Zhai","slug":"hao-zhai","given":"Hao","family":"Zhai","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences; School of Automation and Electrical Engineering , University of Science and Technology Beijing , China"},{"name":"Danqian Liu","slug":"danqian-liu","given":"Danqian","family":"Liu","affiliation":"Howard Hughes Medical Institute , University of California , Berkeley , CA , USA; Center of Excellence in Brain Science and Intelligence Technology , Chinese Academy of Sciences"},{"name":"Weifu Li","slug":"weifu-li","given":"Weifu","family":"Li","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences; College of Science , Huazhong Agricultural University , Wuhan , China"},{"name":"Chaoyue Ding","slug":"chaoyue-ding","given":"Chaoyue","family":"Ding","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Qiwei Xie","slug":"qiwei-xie","given":"Qiwei","family":"Xie","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"},{"name":"Hua Han","slug":"hua-han","given":"Hua","family":"Han","affiliation":"National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences; Center of Excellence in Brain Science and Intelligence Technology , Chinese Academy of Sciences"}],"abstract":"Biologists often need to handle numerous video-based home-cage animal behavior analysis tasks that require massive workloads. Therefore, we develop an AI-based multi-species tracking and segmentation system, SiamBOMB, for real-time and automatic home-cage animal behavioral analysis. In this system, a background-enhanced Siamese-based network with replaceable modular design ensures the flexibility and generalizability of the system, and a user-friendly interface makes it convenient to use for biologists. This real-time AI system will effectively reduce the burden on biologists.","doi":"10.24963/ijcai.2020/776","url":"https://ijcai.org/proceedings/2020/776","pdf":"https://ijcai.org/proceedings/2020/0776.pdf","markdown":"https://ijcai.org/proceedings/2020/776.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/776","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Demos","track":"Demos","number":776,"pages":{"first":5300,"last":5302},"keywords":["Machine Learning: general","Computer Vision: general"],"videos":[],"references":[{"position":1,"raw":"[Bertinetto et al., 2016] Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. Fully-convolutional siamese networks for object tracking. In European conference on computer vision, pages 850- 865. Springer, 2016.","title":"Fully-convolutional siamese networks for object tracking","authors":"Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, Philip Hs Torr","year":"2016","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen and Tsotsos, 2019] Baoxin Chen and John K Tsotsos. Fast visual object tracking with rotated bounding boxes. arXiv preprint arXiv:1907.03892, 2019.","title":"Fast visual object tracking with rotated bounding boxes","authors":"Chen, Tsotsos ; ] Baoxin Chen, John K Tsotsos","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dietrich et al., 2015] Marcelo O Dietrich, Marcelo R Zim- mer, Jeremy Bober, and Tamas L Horvath. Hypothalamic agrp neurons drive stereotypic behaviors beyond feeding. Cell, 160(6):1222-1232, 2015.","title":"Hypothalamic agrp neurons drive stereotypic behaviors beyond feeding","authors":"Marcelo O Dietrich, Marcelo R Zimmer, Jeremy Bober, Tamas L Horvath","year":"2015","venue":"Cell","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dugatkin, 2020] Lee Alan Dugatkin. Principles of animal behavior. University of Chicago Press, 2020.","title":"Principles of animal behavior","authors":"Lee Alan, Dugatkin","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.","title":"Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn","authors":"He","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jhuang et al., 2010] Hueihan Jhuang, Estibaliz Garrote, Xinlin Yu, Vinita Khilnani, Tomaso Poggio, Andrew D Steele, and Thomas Serre. Automated home-cage be- havioural phenotyping of mice. Nature communications, 1(1):1-10, 2010.","title":"Automated home-cage behavioural phenotyping of mice","authors":"Jhuang","year":"2010","venue":"Nature communications","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Li et al., 2018] Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu. High performance visual tracking with siamese region proposal network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 8971-8980, 2018.","title":"High performance visual tracking with siamese region proposal network","authors":"Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, Xiaolin Hu","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lin et al., 2017] Guosheng Lin, Anton Milan, Chunhua Shen, and Ian Reid. Refinenet: Multi-path refinement net- works for high-resolution semantic segmentation. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pages 1925-1934, 2017.","title":"Refinenet: Multi-path refinement networks for high-resolution semantic segmentation","authors":"Guosheng Lin, Anton Milan, Chunhua Shen, Ian Reid","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2020] Danqian Liu, Weifu Li, Chenyan Ma, Weitong Zheng, Yuanyuan Yao, Chak Foon Tso, Peng Zhong, Xi Chen, Jun Ho Song, Woochul Choi, et al. A common hub for sleep and motor control in the substantia nigra. Science, 367(6476):440-445, 2020.","title":"A common hub for sleep and motor control in the substantia nigra","authors":"Liu","year":"2020","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Mathis et al., 2018] Alexander Mathis, Pranav Mamidanna, Kevin M Cury, Taiga Abe, Venkatesh N Murthy, Macken- zie Weygandt Mathis, and Matthias Bethge. Deeplabcut: markerless pose estimation of user-defined body parts with deep learning. Nature neuroscience, 21(9):1281, 2018.","title":"Deeplabcut: markerless pose estimation of user-defined body parts with deep learning","authors":"Alexander Mathis, Pranav Mamidanna, Kevin M Cury, Taiga Abe, N Venkatesh, Mackenzie Weygandt Murthy, Matthias Mathis, Bethge","year":"2018","venue":"Nature neuroscience","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Perazzi et al., 2016] Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 724-732, 2016.","title":"A benchmark dataset and evaluation methodology for video object segmentation","authors":"Federico Perazzi, Jordi Pont-Tuset, Brian Mcwilliams, Luc Van Gool, Markus Gross, Alexander Sorkine-Hornung","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Rao et al., 2019] Srinivasa R Rao, Sam WZ Olechnowicz, Patrick Krätschmer, James EC Jepson, Claire M Edwards, and James R Edwards. Small animal video tracking for activity and path analysis using a novel open-source multi- platform application (animapp). Scientific reports, 9(1):1- 9, 2019.","title":"Small animal video tracking for activity and path analysis using a novel open-source multiplatform application (animapp)","authors":"Sam Wz Srinivasa R Rao, Patrick Olechnowicz, James Ec Krätschmer, Claire M Jepson, James R Edwards, Edwards","year":"2019","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time ob- ject detection with region proposal networks. In Advances in neural information processing systems, pages 91-99, 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pages 234-241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical image computing and computerassisted intervention","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sauer et al., 2019] Axel Sauer, Elie Aljalbout, and Sami Haddadin. Tracking holistic object representations. arXiv preprint arXiv:1907.12920, 2019.","title":"Tracking holistic object representations","authors":"Sauer","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Singh et al., 2019] Surjeet Singh, Edgar Bermudez- Contreras, Mojtaba Nazari, Robert J Sutherland, and Majid H Mohajerani. Low-cost solution for rodent home-cage behaviour monitoring. PloS one, 14(8), 2019.","title":"Low-cost solution for rodent home-cage behaviour monitoring","authors":"Surjeet Singh, Edgar Bermudez-Contreras, Mojtaba Nazari, Robert J Sutherland, H Majid, Mohajerani","year":"2019","venue":"PloS one","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2019] Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, and Philip HS Torr. Fast online object track- ing and segmentation: A unifying approach. In Proceed- ings of the IEEE conference on computer vision and pat- tern recognition, pages 1328-1338, 2019.","title":"Fast online object tracking and segmentation: A unifying approach","authors":"Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, Philip Hs Torr","year":"2019","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yu et al., 2017] Yao-Qing Yu, Devin M Barry, Yan Hao, Xue-Ting Liu, and Zhou-Feng Chen. Molecular and neu- ral basis of contagious itch behavior in mice. Science, 355(6329):1072-1076, 2017.","title":"Molecular and neural basis of contagious itch behavior in mice","authors":"Yao-Qing Yu, Devin M Barry, Yan Hao, Xue-Ting Liu, Zhou-Feng Chen","year":"2017","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhang et al., 2017] Qiao Zhang, Zhipeng Cui, Xiaoguang Niu, Shijie Geng, and Yu Qiao. Image segmentation with pyramid dilated convolution based on resnet and u-net. In International Conference on Neural Information Process- ing, pages 364-372. Springer, 2017.","title":"Image segmentation with pyramid dilated convolution based on resnet and u-net","authors":"Qiao Zhang, Zhipeng Cui, Xiaoguang Niu, Shijie Geng, Yu Qiao","year":"2017","venue":"International Conference on Neural Information Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang et al., 2019] Lichao Zhang, Abel Gonzalez-Garcia, Joost van de Weijer, Martin Danelljan, and Fahad Shah- baz Khan. Learning the model update for siamese track- ers. In Proceedings of the IEEE International Conference on Computer Vision, pages 4010-4019, 2019.","title":"Learning the model update for siamese trackers","authors":"Zhang","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}