{"type":"ScholarlyArticle","title":"Benchmarking Fish Dataset and Evaluation Metric in Keypoint Detection - Towards Precise Fish Morphological Assessment in Aquaculture Breeding","authors":[{"name":"Weizhen Liu","slug":"weizhen-liu","given":"Weizhen","family":"Liu","affiliation":"School of Computer Science and Artificial Intelligence , Wuhan University of Technology"},{"name":"Jiayu Tan","slug":"jiayu-tan","given":"Jiayu","family":"Tan","affiliation":"School of Computer Science and Artificial Intelligence , Wuhan University of Technology"},{"name":"Guangyu Lan","slug":"guangyu-lan","given":"Guangyu","family":"Lan","affiliation":"School of Computer Science and Artificial Intelligence , Wuhan University of Technology"},{"name":"Ao Li","slug":"ao-li","given":"Ao","family":"Li","affiliation":"School of Computer Science and Artificial Intelligence , Wuhan University of Technology"},{"name":"Dongye Li","slug":"dongye-li","given":"Dongye","family":"Li","affiliation":"Sanya Boruiyuan Technology Co. Ltd"},{"name":"Le Zhao","slug":"le-zhao","given":"Le","family":"Zhao","affiliation":"School of Computer Science and Artificial Intelligence , Wuhan University of Technology"},{"name":"Xiaohui Yuan","slug":"xiaohui-yuan","given":"Xiaohui","family":"Yuan","affiliation":"School of Computer Science and Artificial Intelligence , Wuhan University of Technology; Yazhouwan National Laboratory"},{"name":"Nanqing Dong","slug":"nanqing-dong","given":"Nanqing","family":"Dong","affiliation":"Shanghai Artificial Intelligence Laboratory"}],"abstract":"Accurate phenotypic analysis in aquaculture breeding necessitates the quantification of subtle morphological phenotypes. Existing datasets suffer from limitations such as small scale, limited species coverage, and inadequate annotation of keypoints for measuring refined and complex morphological phenotypes of fish body parts. To address this gap, we introduce FishPhenoKey, a comprehensive dataset comprising 23,331 high-resolution images spanning six fish species. Notably, FishPhenoKey includes 22 phenotype-oriented annotations, enabling the capture of intricate morphological phenotypes. Motivated by the nuanced evaluation of these subtle morphologies, we also propose a new evaluation metric, Percentage of Measured Phenotypes (PMP). It is designed to assess the accuracy of individual keypoint positions and is highly sensitive to the phenotype measured using the corresponding keypoints. To enhance keypoint detection accuracy, we further propose a novel loss, Anatomically-Calibrated Regularization (ACR), that can be integrated into keypoint detection models, leveraging biological insights to refine keypoint localization. Our contributions set a new benchmark in fish phenotype analysis, addressing the challenges of precise morphological quantification and opening new avenues for research in sustainable aquaculture and genetic studies. Our dataset and code are available at https://github.com/WeizhenLiuBioinform/FishPhenotype-Detect.","doi":"10.24963/ijcai.2024/816","url":"https://ijcai.org/proceedings/2024/816","pdf":"https://ijcai.org/proceedings/2024/0816.pdf","markdown":"https://ijcai.org/proceedings/2024/816.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/816","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"AI for Good","track":"AI for Good","number":816,"pages":{"first":7376,"last":7384},"keywords":["Multidisciplinary Topics and Applications: General","Computer Vision: General","Machine Learning: General"],"videos":[],"references":[{"position":1,"raw":"[Andriluka et al., 2014] Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2d human pose estimation: New benchmark and state of the art anal- ysis. In Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, pages 3686-3693, 2014.","title":"2d human pose estimation: New benchmark and state of the art analysis","authors":"Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, Bernt Schiele","year":"2014","venue":"Proceedings of the IEEE Conference on computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Castrillo et al., 2021] Pedro A Castrillo, Catuxa Varela- Dopico, Roberto Bermúdez, Paz Ondina, and María Isabel Quiroga. Morphopathology and gill recovery of atlantic salmon during the parasitic detachment of margaritifera margaritifera. Journal of Fish Diseases, 44(8):1101-1115, 2021.","title":"Morphopathology and gill recovery of atlantic salmon during the parasitic detachment of margaritifera margaritifera","authors":"Pedro A Castrillo, Catuxa Varela-Dopico, Roberto Bermúdez, Paz Ondina, María Isabel, Quiroga","year":"2021","venue":"Journal of Fish Diseases","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning, pages 794-803. PMLR, 2018.","title":"Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks","authors":"Vijay Zhao Chen, Chen-Yu Badrinarayanan, Andrew Lee, Rabinovich","year":"2018","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Devine et al., 2020] Jay Devine, Jose D Aponte, David C Katz, Wei Liu, Lucas D Lo Vercio, Nils D Forkert, Ralph Marcucio, Christopher J Percival, and Benedikt Hallgrímsson. A registration and deep learning approach to automated landmark detection for geometric morpho- metrics. Evolutionary Biology, 47(3):246-259, 2020.","title":"A registration and deep learning approach to automated landmark detection for geometric morphometrics","authors":"Jay Devine, Jose D Aponte, David C Katz, Wei Liu, Lucas D Lo Vercio, Nils D Forkert, Ralph Marcucio, Christopher J Percival, Benedikt Hallgrímsson","year":"2020","venue":"Evolutionary Biology","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2022] Nanqing Dong, Michael Kampffmeyer, Xiaodan Liang, Min Xu, Irina Voiculescu, and Eric Xing. Towards robust partially supervised multi-structure medi- cal image segmentation on small-scale data. Applied Soft Computing, 114:108074, 2022.","title":"Towards robust partially supervised multi-structure medical image segmentation on small-scale data","authors":"Dong","year":"2022","venue":"Applied Soft Computing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dong et al., 2023] Junyu Dong, Xinyu Shangguan, Kaim- ing Zhou, Yanhai Gan, Hao Fan, and Long Chen. A detection-regression based framework for fish keypoints detection. Intelligent Marine Technology and Systems, 1(1):9, 2023.","title":"A detection-regression based framework for fish keypoints detection","authors":"Dong","year":"2023","venue":"Intelligent Marine Technology and Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[FAO, 2022] FAO. Record fisheries and aquaculture production makes critical contribution to global food security. https://www.fao.org/newsroom/detail/record- fisheries-aquaculture-production-contributes-food- security-290622/, 2022.","title":"FAO. Record fisheries and aquaculture production makes critical contribution to global food security","authors":null,"year":"2022","venue":null,"doi":null,"url":"https://www.fao.org/newsroom/detail/record-fisheries-aquaculture-production-contributes-food-security-290622/,2022","internal_url":null},{"position":8,"raw":"[Figueroa et al., 2018] Rosa I Figueroa, Alfredo de Bustos, and Ángeles Cuadrado. A novel fish technique for labeling the chromosomes of dinoflagellates in suspension. Plos one, 13(10):e0204382, 2018.","title":"A novel fish technique for labeling the chromosomes of dinoflagellates in suspension","authors":"Alfredo Rosa I Figueroa, Ángeles De Bustos, Cuadrado","year":"2018","venue":"Plos one","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guesdon et al., 2021] Romain Guesdon, Carlos Crispim- Junior, and Laure Tougne. Dripe: A dataset for human pose estimation in real-world driving settings. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 2865-2874, 2021.","title":"Dripe: A dataset for human pose estimation in real-world driving settings","authors":"Romain Guesdon, Carlos Crispim-Junior, Laure Tougne","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jerry and Cairns, 1998] Dean R. Jerry and Stuart C. Cairns. Morphological variation in the catadromous aus- tralian bass, from seven geographically distinct riverine drainages. Journal of Fish Biology, 52:829-843, 1998.","title":"Morphological variation in the catadromous australian bass, from seven geographically distinct riverine drainages","authors":"Dean R Jerry, Stuart C Cairns","year":"1998","venue":"Journal of Fish Biology","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Johnson and Everingham, 2010] Sam Johnson and Mark Everingham. Clustered pose and nonlinear appearance models for human pose estimation. In British Machine Vision Conference, volume 2, page 5. Aberystwyth, UK, 2010.","title":"Clustered pose and nonlinear appearance models for human pose estimation","authors":"Johnson, Everingham, Sam Johnson, Mark Everingham","year":"2010","venue":"British Machine Vision Conference","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kumar et al., 2022] Navdeep Kumar, Claudia Di Biagio, Zachary Dellacqua, Ratish Raman, Arianna Martini, Clara Boglione, Marc Muller, Pierre Geurts, and Raphaël Marée. Empirical evaluation of deep learning approaches for land- mark detection in fish bioimages. In European Conference on Computer Vision, pages 470-486. Springer, 2022.","title":"Empirical evaluation of deep learning approaches for landmark detection in fish bioimages","authors":"Navdeep Kumar, Claudia Di Biagio, Zachary Dellacqua, Ratish Raman, Arianna Martini, Clara Boglione, Marc Muller, Pierre Geurts, Raphaël Marée","year":"2022","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liao et al., 2021] Yu-Hang Liao, Chao-Wei Zhou, Wei- Zhen Liu, Jing-Yi Jin, Dong-Ye Li, Fei Liu, Ding-Ding Fan, Yu Zou, Zen-Bo Mu, Jian Shen, et al. 3dphenofish: Application for two-and three-dimensional fish morpho- logical phenotype extraction from point cloud analysis. Zoological Research, 42(4):492, 2021.","title":"3dphenofish: Application for two-and three-dimensional fish morphological phenotype extraction from point cloud analysis","authors":"Yu-Hang Liao, Chao-Wei Zhou, Wei-Zhen Liu, Jing-Yi Jin, Dong-Ye Li, Fei Liu, Ding-Ding Fan, Yu Zou, Zen-Bo Mu, Jian Shen","year":"2021","venue":"Zoological Research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, Septem- ber 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"Computer Vision-ECCV 2014: 13th European Conference","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mitteroecker and Schaefer, 2022] Philipp Mitteroecker and Katrin Schaefer. Thirty years of geometric morphomet- rics: Achievements, challenges, and the ongoing quest for biological meaningfulness. American Journal of Biologi- cal Anthropology, 178:181-210, 2022.","title":"Philipp Mitteroecker and Katrin Schaefer. Thirty years of geometric morphometrics: Achievements, challenges, and the ongoing quest for biological meaningfulness","authors":"Schaefer Mitteroecker","year":"2022","venue":"American Journal of Biological Anthropology","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Moccetti et al., 2023] Paolo Moccetti, Jessica R Rodger, Jonathan D Bolland, Phoebe Kaiser-Wilks, Rowan Smith, Andy D Nunn, Colin E Adams, Jen A Bright, Hannele M Honkanen, Angus J Lothian, et al. Is shape in the eye of the beholder? assessing landmarking error in geomet- ric morphometric analyses on live fish. PeerJ, 11:e15545, 2023.","title":"Is shape in the eye of the beholder? assessing landmarking error in geometric morphometric analyses on live fish","authors":"Paolo Moccetti, Jessica R Rodger, Jonathan D Bolland, Phoebe Kaiser-Wilks, Rowan Smith, Andy D Nunn, Colin E Adams, Jen A Bright, M Hannele, Angus J Honkanen, Lothian","year":"2023","venue":"PeerJ","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Newell et al., 2016] Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hourglass networks for human pose esti- mation. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14, pages 483-499. Springer, 2016.","title":"Stacked hourglass networks for human pose estimation","authors":"Alejandro Newell, Kaiyu Yang, Jia Deng","year":"2016","venue":"Computer Vision-ECCV 2016: 14th European Conference","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural in- formation processing systems, 32, 2019.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pisov et al., 2020] Maxim Pisov, Vladimir Kondratenko, Alexey Zakharov, Alexey Petraikin, Victor Gombolevskiy, Sergey Morozov, and Mikhail Belyaev. Keypoints local- ization for joint vertebra detection and fracture severity quantification. In Medical Image Computing and Com- puter Assisted Intervention-MICCAI 2020: 23rd Interna- tional Conference, Lima, Peru, October 4-8, 2020, Pro- ceedings, Part VI 23, pages 723-732. Springer, 2020.","title":"Keypoints localization for joint vertebra detection and fracture severity quantification","authors":"Pisov","year":"2020","venue":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2020: 23rd International Conference","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Saleh et al., 2023] Alzayat Saleh, David Jones, Dean Jerry, and Mostafa Rahimi Azghadi. Mfld-net: a lightweight deep learning network for fish morphometry using land- mark detection. Aquatic Ecology, 57(4):913-931, 2023.","title":"Mfld-net: a lightweight deep learning network for fish morphometry using landmark detection","authors":"Saleh","year":"2023","venue":"Aquatic Ecology","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Stern et al., 2011] Olivier Stern, Raphaël Marée, Jessica Aceto, Nathalie Jeanray, Marc Muller, Louis Wehenkel, and Pierre Geurts. Automatic localization of interest points in zebrafish images with tree-based methods. In Pattern Recognition in Bioinformatics: 6th IAPR Inter- national Conference, PRIB 2011, Delft, The Netherlands, November 2-4, 2011. Proceedings 6, pages 179-190. Springer, 2011.","title":"Automatic localization of interest points in zebrafish images with tree-based methods","authors":"Olivier Stern, Raphaël Marée, Jessica Aceto, Nathalie Jeanray, Marc Muller, Louis Wehenkel, Pierre Geurts","year":"2011","venue":"Pattern Recognition in Bioinformatics: 6th IAPR International Conference, PRIB 2011","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Suo et al., 2020] Feiyang Suo, Kangwei Huang, Gui Ling, Yanjun Li, and Ji Xiang. Fish keypoints detection for ecol- ogy monitoring based on underwater visual intelligence. In 2020 16th International Conference on Control, Au- tomation, Robotics and Vision (ICARCV), pages 542-547. IEEE, 2020.","title":"Fish keypoints detection for ecology monitoring based on underwater visual intelligence","authors":"Suo","year":"2020","venue":"16th International Conference on Control, Automation, Robotics and Vision (ICARCV)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tang et al., 2023] Zhixin Tang, Zhuo Chen, Yuan Gao, Ruxian Xue, Zedong Geng, Qingyun Bu, Yanyan Wang, Xiaoqian Chen, Yuqiang Jiang, Fan Chen, et al. A strategy for the acquisition and analysis of image-based phenome in rice during the whole growth period. Plant Phenomics, 5:0058, 2023.","title":"A strategy for the acquisition and analysis of image-based phenome in rice during the whole growth period","authors":"Zhixin Tang, Zhuo Chen, Yuan Gao, Ruxian Xue, Zedong Geng, Qingyun Bu, Yanyan Wang, Xiaoqian Chen, Yuqiang Jiang, Fan Chen","year":"2023","venue":"Plant Phenomics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Toshev and Szegedy, 2014] Alexander Toshev and Chris- tian Szegedy. Deeppose: Human pose estimation via deep neural networks. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 1653-1660, 2014.","title":"Deeppose: Human pose estimation via deep neural networks","authors":"Alexander Toshev, Christian Szegedy","year":"2014","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[United Nations, 2023] United Nations. 17 sustainable de- velopment goals. https://sdgs.un.org/goals, 2023.","title":"United Nations. 17 sustainable development goals","authors":null,"year":"2023","venue":null,"doi":null,"url":"https://sdgs.un.org/goals,2023","internal_url":null},{"position":27,"raw":"[Verhaegen et al., 2007] Yves Verhaegen, Dominique Adri- aens, Tania De Wolf, Philippe Dhert, and Patrick Sorgeloos. Deformities in larval gilthead sea bream (sparus aurata): A qualitative and quantitative analysis using geometric morphometrics. Aquaculture, 268(1- 4):156-168, 2007.","title":"Deformities in larval gilthead sea bream (sparus aurata): A qualitative and quantitative analysis using geometric morphometrics","authors":"Verhaegen","year":"2007","venue":"Aquaculture","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2020] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al. Deep high-resolution representation learning for visual recogni- tion. IEEE transactions on pattern analysis and machine intelligence, 43(10):3349-3364, 2020.","title":"Deep high-resolution representation learning for visual recognition","authors":"Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang","year":"2020","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang and Ramanan, 2012] Yi Yang and Deva Ramanan. Ar- ticulated human detection with flexible mixtures of parts. IEEE transactions on pattern analysis and machine intel- ligence, 35(12):2878-2890, 2012.","title":"Articulated human detection with flexible mixtures of parts","authors":"Yang, Ramanan, Yi Yang, Deva Ramanan","year":"2012","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yu et al., 2023] Yaozhen Yu, Hao Zhang, and Fei Yuan. Key point detection method for fish size measurement based on deep learning. IET Image Processing, 17(14):4142-4158, 2023.","title":"Key point detection method for fish size measurement based on deep learning","authors":"Yu","year":"2023","venue":"IET Image Processing","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2019] Feng Zhang, Xiatian Zhu, and Mao Ye. Fast human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2019.","title":"Fast human pose estimation","authors":"Feng Zhang, Xiatian Zhu, Mao Ye","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhu et al., 2021] Chuancheng Zhu, Yusong Hu, Hude Mao, Shumin Li, Fangfang Li, Congyuan Zhao, Lin Luo, Weizhen Liu, and Xiaohui Yuan. A deep learning-based method for automatic assessment of stomatal index in wheat microscopic images of leaf epidermis. Frontiers in Plant Science, 12:716784, 2021.","title":"A deep learning-based method for automatic assessment of stomatal index in wheat microscopic images of leaf epidermis","authors":"Chuancheng Zhu, Yusong Hu, Hude Mao, Shumin Li, Fangfang Li, Congyuan Zhao, Lin Luo, Weizhen Liu, Xiaohui Yuan","year":"2021","venue":"Frontiers in Plant Science","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}