{"type":"ScholarlyArticle","title":"Fluid Dynamics-Inspired Network for Infrared Small Target Detection","authors":[{"name":"Tianxiang Chen","slug":"tianxiang-chen","given":"Tianxiang","family":"Chen","affiliation":"School of Cyber Science and Technology , University of Science and Technology of China , China"},{"name":"Qi Chu","slug":"qi-chu","given":"Qi","family":"Chu","affiliation":"School of Cyber Science and Technology , University of Science and Technology of China , China"},{"name":"Bin Liu","slug":"bin-liu","given":"Bin","family":"Liu","affiliation":""},{"name":"Nenghai Yu","slug":"nenghai-yu","given":"Nenghai","family":"Yu","affiliation":"School of Cyber Science and Technology , University of Science and Technology of China , China"}],"abstract":"Most infrared small target detection (ISTD) networks focus on building effective neural blocks or feature fusion modules but none describes the ISTD process from the image evolution perspective. The directional evolution of image pixels influenced by convolution, pooling and surrounding pixels is analogous to the movement of fluid elements constrained by surrounding variables ang particles. Inspired by this, we explore a novel research routine by abstracting the movement of pixels in the ISTD process as the flow of fluid in fluid dynamics (FD). Specifically, a new Fluid Dynamics-Inspired Network (FDI-Net) is devised for ISTD. Based on Taylor Central Difference (TCD) method, the TCD feature extraction block is designed, where convolution and Transformer structures are combined for local and global information. The pixel motion equation during the ISTD process is derived from the Navier–Stokes (N-S) equation, constructing a N-S Refinement Module that refines extracted features with edge details. Thus, the TCD feature extraction block determines the primary movement direction of pixels during detection, while the N-S Refinement Module corrects some skewed directions of the pixel stream to supplement the edge details. Experiments on IRSTD-1k and SIRST demonstrate that our method achieves SOTA performance in terms of evaluation metrics.","doi":"10.24963/ijcai.2023/66","url":"https://ijcai.org/proceedings/2023/66","pdf":"https://ijcai.org/proceedings/2023/0066.pdf","markdown":"https://ijcai.org/proceedings/2023/66.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/66","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":66,"pages":{"first":590,"last":598},"keywords":["Computer Vision: CV: Segmentation","Computer Vision: CV: Recognition (object detection, categorization)"],"videos":[],"references":[{"position":1,"raw":"[Anderson and Wendt, 1995] John David Anderson and John Wendt. Computational fluid dynamics, volume 206. Springer, 1995.","title":"Computational fluid dynamics","authors":"Anderson, Wendt, John David, Anderson, John Wendt","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Belbute-Peres et al., 2020] Filipe De Avila Belbute-Peres, Thomas Economon, and Zico Kolter. Combining differ- entiable pde solvers and graph neural networks for fluid flow prediction. In international conference on machine learning, pages 2402-2411. PMLR, 2020.","title":"Combining differentiable pde solvers and graph neural networks for fluid flow prediction","authors":"Filipe De, Avila Belbute-Peres, Thomas Economon, Zico Kolter","year":"2020","venue":"international conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2023] Tianxiang Chen, Qi Chu, Zhentao Tan, Bin Liu, and Nenghai Yu. Bauenet: Boundary-aware uncertainty enhanced network for infrared small target detection. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.","title":"Bauenet: Boundary-aware uncertainty enhanced network for infrared small target detection","authors":"Tianxiang Chen, Qi Chu, Zhentao Tan, Bin Liu, Nenghai Yu","year":"2023","venue":"ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dai et al., 2021a] Yimian Dai, Yiquan Wu, Fei Zhou, and Kobus Barnard. Asymmetric contextual modulation for infrared small target detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 950-959, 2021.","title":"Asymmetric contextual modulation for infrared small target detection","authors":"Dai","year":"2021","venue":"Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dai et al., 2021b] Yimian Dai, Yiquan Wu, Fei Zhou, and Kobus Barnard. Attentional local contrast networks for in- frared small target detection. IEEE Transactions on Geo- science and Remote Sensing, 59(11):9813-9824, 2021.","title":"Attentional local contrast networks for infrared small target detection","authors":"Dai","year":"2021","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng et al., 2019] Zhiwen Deng, Chuangxin He, Yingzheng Liu, and Kyung Chun Kim. Super-resolution reconstruction of turbulent velocity fields using a gen- erative adversarial network-based artificial intelligence framework. Physics of Fluids, 31(12):125111, 2019.","title":"Super-resolution reconstruction of turbulent velocity fields using a generative adversarial network-based artificial intelligence framework","authors":"Zhiwen Deng, Chuangxin He, Yingzheng Liu, Kyung Chun Kim","year":"2019","venue":"Physics of Fluids","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Han et al., 2019] Jinhui Han, Sibang Liu, Gang Qin, Qian Zhao, Honghui Zhang, and Nana Li. A local contrast method combined with adaptive background estimation for infrared small target detection. IEEE Geoscience and Remote Sensing Letters, 16(9):1442-1446, 2019.","title":"A local contrast method combined with adaptive background estimation for infrared small target detection","authors":"Jinhui Han, Sibang Liu, Gang Qin, Qian Zhao, Honghui Zhang, Nana Li","year":"2019","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2019] Xiangyu He, Zitao Mo, Peisong Wang, Yang Liu, Mingyuan Yang, and Jian Cheng. Ode-inspired network design for single image super-resolution. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1732-1741, 2019.","title":"Ode-inspired network design for single image super-resolution","authors":"Xiangyu He, Zitao Mo, Peisong Wang, Yang Liu, Mingyuan Yang, Jian Cheng","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2021] Bei Li, Quan Du, Tao Zhou, Shuhan Zhou, Xin Zeng, Tong Xiao, and Jingbo Zhu. Ode transformer: An ordinary differential equation-inspired model for neu- ral machine translation. arXiv preprint arXiv:2104.02308, 2021.","title":"Ode transformer: An ordinary differential equation-inspired model for neural machine translation","authors":"Bei Li, Quan Du, Tao Zhou, Shuhan Zhou, Xin Zeng, Tong Xiao, Jingbo Zhu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2022a] Boyang Li, Chao Xiao, Longguang Wang, Yingqian Wang, Zaiping Lin, Miao Li, Wei An, and Yulan Guo. Dense nested attention network for infrared small target detection. IEEE Transactions on Image Processing, 2022.","title":"Dense nested attention network for infrared small target detection","authors":"Li","year":"2022","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2022b] Kunchang Li, Yali Wang, Junhao Zhang, Peng Gao, Guanglu Song, Yu Liu, Hongsheng Li, and Yu Qiao. Uniformer: Unifying convolution and self-attention for visual recognition. arXiv preprint arXiv:2201.09450, 2022.","title":"Uniformer: Unifying convolution and self-attention for visual recognition","authors":"Li","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2021] Fangcen Liu, Chenqiang Gao, Fang Chen, Deyu Meng, Wangmeng Zuo, and Xinbo Gao. Infrared small-dim target detection with transformer under com- plex backgrounds. arXiv preprint arXiv:2109.14379, 2021.","title":"Infrared small-dim target detection with transformer under complex backgrounds","authors":"Liu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lu et al., 2019] Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang, and Tie-Yan Liu. Understanding and improving transformer from a multi- particle dynamic system point of view. arXiv preprint arXiv:1906.02762, 2019.","title":"Understanding and improving transformer from a multiparticle dynamic system point of view","authors":"Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang, Tie-Yan Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Marvasti et al., 2018] Fereshteh Seyed Marvasti, Moham- mad Reza Mosavi, and Mahdi Nasiri. Flying small target detection in ir images based on adaptive toggle operator. IET Computer Vision, 12(4):527-534, 2018.","title":"Flying small target detection in ir images based on adaptive toggle operator","authors":"Seyed Fereshteh, Mohammad Marvasti, Reza Mosavi, Mahdi Nasiri","year":"2018","venue":"IET Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Qi et al., 2022] Meibin Qi, Liu Liu, Shuo Zhuang, Yimin Liu, Kunyuan Li, Yanfang Yang, and Xiaohong Li. Ftc- net: Fusion of transformer and cnn features for infrared small target detection. IEEE Journal of Selected Top- ics in Applied Earth Observations and Remote Sensing, 15:8613-8623, 2022.","title":"Ftcnet: Fusion of transformer and cnn features for infrared small target detection","authors":"Meibin Qi, Liu Liu, Shuo Zhuang, Yimin Liu, Kunyuan Li, Yanfang Yang, Xiaohong Li","year":"2022","venue":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rivest and Fortin, 1996] Jean-Francois Rivest and Roger Fortin. Detection of dim targets in digital infrared imagery by morphological image processing. Optical Engineering, 35(7):1886-1893, 1996.","title":"Detection of dim targets in digital infrared imagery by morphological image processing","authors":"Fortin Rivest, Jean-Francois Rivest, Roger Fortin","year":"1996","venue":"Optical Engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schenck and Fox, 2018] Connor Schenck and Dieter Fox. Spnets: Differentiable fluid dynamics for deep neural net- works. In Conference on Robot Learning, pages 317-335.","title":"Spnets: Differentiable fluid dynamics for deep neural networks","authors":"Connor Schenck, Dieter Fox","year":null,"venue":"Conference on Robot Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"PMLR, 2018.","title":null,"authors":"Pmlr","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sudre et al., 2017] Carole H Sudre, Wenqi Li, Tom Ver- cauteren, Sebastien Ourselin, and M Jorge Cardoso. Gen- eralised dice overlap as a deep learning loss function for highly unbalanced segmentations. In Deep learning in medical image analysis and multimodal learning for clin- ical decision support, pages 240-248. Springer, 2017.","title":"Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations","authors":"Carole H Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, Jorge Cardoso","year":"2017","venue":"Deep learning in medical image analysis and multimodal learning for clinical decision support","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2020] Yang Sun, Jungang Yang, and Wei An. In- frared dim and small target detection via multiple sub- space learning and spatial-temporal patch-tensor model. IEEE Transactions on Geoscience and Remote Sensing, 59(5):3737-3752, 2020.","title":"Infrared dim and small target detection via multiple subspace learning and spatial-temporal patch-tensor model","authors":"Yang Sun, Jungang Yang, Wei An","year":"2020","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2019] Huan Wang, Luping Zhou, and Lei Wang. Miss detection vs. false alarm: Adversarial learn- ing for small object segmentation in infrared images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8509-8518, 2019.","title":"Miss detection vs. false alarm: Adversarial learning for small object segmentation in infrared images","authors":"Huan Wang, Luping Zhou, Lei Wang","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2022] Kewei Wang, Shuaiyuan Du, Chengxin Liu, and Zhiguo Cao. Interior attention-aware network for infrared small target detection. IEEE Transactions on Geo- science and Remote Sensing, 60:1-13, 2022.","title":"Interior attention-aware network for infrared small target detection","authors":"Kewei Wang, Shuaiyuan Du, Chengxin Liu, Zhiguo Cao","year":"2022","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Weinan, 2017] E Weinan. A proposal on machine learning via dynamical systems. Communications in Mathematics and Statistics, 1(5):1-11, 2017.","title":"A proposal on machine learning via dynamical systems","authors":"Weinan","year":"2017","venue":"Communications in Mathematics and Statistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2022] Xin Wu, Danfeng Hong, and Jocelyn Chanussot. Uiu-net: U-net in u-net for infrared small ob- ject detection. IEEE Transactions on Image Processing, 2022.","title":"Uiu-net: U-net in u-net for infrared small object detection","authors":"Xin Wu, Danfeng Hong, Jocelyn Chanussot","year":"2022","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang and Peng, 2019] Landan Zhang and Zhenming Peng. Infrared small target detection based on partial sum of the tensor nuclear norm. Remote Sensing, 11(4):382, 2019.","title":"Infrared small target detection based on partial sum of the tensor nuclear norm","authors":"Peng ; ] Landan Zhang, Zhenming Zhang, Peng","year":"2019","venue":"Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2021] Tianfang Zhang, Siying Cao, Tian Pu, and Zhenming Peng. Agpcnet: Attention-guided pyramid context networks for infrared small target detection. arXiv preprint arXiv:2111.03580, 2021.","title":"Agpcnet: Attention-guided pyramid context networks for infrared small target detection","authors":"Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2022a] Mingjin Zhang, Haichen Bai, Jing Zhang, Rui Zhang, Chaoyue Wang, Jie Guo, and Xinbo Gao. Rkformer: Runge-kutta transformer with random- connection attention for infrared small target detection. In Proceedings of the 30th ACM International Conference on Multimedia, pages 1730-1738, 2022.","title":"Rkformer: Runge-kutta transformer with randomconnection attention for infrared small target detection","authors":"Zhang","year":"2022","venue":"Proceedings of the 30th ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2022b] Mingjin Zhang, Qianqian Wu, Jing Zhang, Xinbo Gao, Jie Guo, and Dacheng Tao. Fluid mi- celle network for image super-resolution reconstruction. IEEE Transactions on Cybernetics, 2022.","title":"Fluid micelle network for image super-resolution reconstruction","authors":"Zhang","year":"2022","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2022c] Mingjin Zhang, Rui Zhang, Yuxiang Yang, Haichen Bai, Jing Zhang, and Jie Guo. Isnet: Shape matters for infrared small target detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 877-886, 2022.","title":"Isnet: Shape matters for infrared small target detection","authors":"Zhang","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}