{"type":"ScholarlyArticle","title":"Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction","authors":[{"name":"Hong Wang","slug":"hong-wang","given":"Hong","family":"Wang","affiliation":"Tencent Jarvis Lab , Shenzhen , China; Xi'an Jiaotong University , Xi'an , China"},{"name":"Yuexiang Li","slug":"yuexiang-li","given":"Yuexiang","family":"Li","affiliation":"Tencent Jarvis Lab , Shenzhen , China"},{"name":"Deyu Meng","slug":"deyu-meng","given":"Deyu","family":"Meng","affiliation":"Xi'an Jiaotong University , Xi'an , China; Peng Cheng Laboratory , Shenzhen , China"},{"name":"Yefeng Zheng","slug":"yefeng-zheng","given":"Yefeng","family":"Zheng","affiliation":"Tencent Jarvis Lab , Shenzhen , China"}],"abstract":"Inspired by the great success of deep neural networks, learning-based methods have gained promising performances for metal artifact reduction (MAR) in computed tomography (CT) images. However, most of the existing approaches put less emphasis on modelling and embedding the intrinsic prior knowledge underlying this specific MAR task into their network designs. Against this issue, we propose an adaptive convolutional dictionary network (ACDNet), which leverages both model-based and learning-based methods. Specifically, we explore the prior structures of metal artifacts, e.g., non-local repetitive streaking patterns, and encode them as an explicit weighted convolutional dictionary model. Then, a simple-yet-effective algorithm is carefully designed to solve the model. By unfolding every iterative substep of the proposed algorithm into a network module, we explicitly embed the prior structure into a deep network , i.e., a clear interpretability for the MAR task. Furthermore, our ACDNet can automatically learn the prior for artifact-free CT images via training data and adaptively adjust the representation kernels for each input CT image based on its content. Hence, our method inherits the clear interpretability of model-based methods and maintains the powerful representation ability of learning-based methods. Comprehensive experiments executed on synthetic and clinical datasets show the superiority of our ACDNet in terms of effectiveness and model generalization. Code and supplementary material are available at https://github.com/hongwang01/ACDNet.","doi":"10.24963/ijcai.2022/195","url":"https://ijcai.org/proceedings/2022/195","pdf":"https://ijcai.org/proceedings/2022/0195.pdf","markdown":"https://ijcai.org/proceedings/2022/195.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/195","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":195,"pages":{"first":1401,"last":1407},"keywords":["Computer Vision: Biomedical Image Analysis"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986832","length_sec":65,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984975","length_sec":358,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Beck and Teboulle, 2009] Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183-202, 2009.","title":"A fast iterative shrinkage-thresholding algorithm for linear inverse problems","authors":"Amir Beck, Marc Teboulle","year":"2009","venue":"SIAM Journal on Imaging Sciences","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Donoho, 1995] David L Donoho. De-noising by soft- thresholding. IEEE Transactions on Information Theory, 41(3):613-627, 1995.","title":"De-noising by softthresholding","authors":"L David, Donoho","year":"1995","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ghani and Karl, 2019] Muhammad Usman Ghani and W Clem Karl. Fast enhanced CT metal artifact reduction using data domain deep learning. IEEE Transactions on Computational Imaging, 6:181-193, 2019.","title":"Fast enhanced CT metal artifact reduction using data domain deep learning","authors":"Karl ; Ghani, Usman Muhammad, W Ghani, Karl Clem","year":"2019","venue":"IEEE Transactions on Computational Imaging","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Huang et al., 2018] Xia Huang, Jian Wang, Fan Tang, Tao Zhong, and Yu Zhang. Metal artifact reduction on cervical CT images by deep residual learning. Biomedical Engi- neering Online, 17(1):1-15, 2018.","title":"Metal artifact reduction on cervical CT images by deep residual learning","authors":"Xia Huang, Jian Wang, Fan Tang, Tao Zhong, Yu Zhang","year":"2018","venue":"Biomedical Engineering Online","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kalender et al., 1987] Willi A Kalender, Robert Hebel, and Johannes Ebersberger. Reduction of CT artifacts caused by metallic implants. Radiology, 164(2):576-577, 1987.","title":"Reduction of CT artifacts caused by metallic implants","authors":"Willi A Kalender, Robert Hebel, Johannes Ebersberger","year":"1987","venue":"Radiology","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Lemmens et al., 2008] Catherine Lemmens, David Faul, and Johan Nuyts. Suppression of metal artifacts in CT using a reconstruction procedure that combines MAP and projection completion. IEEE Transactions on Medical Imaging, 28(2):250-260, 2008.","title":"Suppression of metal artifacts in CT using a reconstruction procedure that combines MAP and projection completion","authors":"Catherine Lemmens, David Faul, Johan Nuyts","year":"2008","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Lin et al., 2019] Wei-An Lin, Haofu Liao, Cheng Peng, Xi- aohang Sun, Jingdan Zhang, Jiebo Luo, Rama Chellappa, and Shaohua Kevin Zhou. DuDoNet: Dual domain net- work for CT metal artifact reduction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 10512-10521, 2019.","title":"DuDoNet: Dual domain network for CT metal artifact reduction","authors":"Wei-An Lin, Haofu Liao, Cheng Peng, Xiaohang Sun, Jingdan Zhang, Jiebo Luo, Rama Chellappa, Kevin Shaohua, Zhou","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2021] Pengbo Liu, Hu Han, Yuanqi Du, Heqin Zhu, Yinhao Li, Feng Gu, Honghu Xiao, Jun Li, Chun- peng Zhao, Li Xiao, et al. Deep learning to segment pelvic bones: large-scale CT datasets and baseline models. In- ternational Journal of Computer Assisted Radiology and Surgery, 16(5):749-756, 2021.","title":"Deep learning to segment pelvic bones: large-scale CT datasets and baseline models","authors":"Pengbo Liu, Hu Han, Yuanqi Du, Heqin Zhu, Yinhao Li, Feng Gu, Honghu Xiao, Jun Li, Chunpeng Zhao, Li Xiao","year":"2021","venue":"International Journal of Computer Assisted Radiology and Surgery","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2022] Xinyi Liu, Qi Xie, Qian Zhao, Hong Wang, and Deyu Meng. Low-light image enhancement by retinex based algorithm unrolling and adjustment. arXiv preprint arXiv:2202.05972, 2022.","title":"Low-light image enhancement by retinex based algorithm unrolling and adjustment","authors":"Xinyi Liu, Qi Xie, Qian Zhao, Hong Wang, Deyu Meng","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lyu et al., 2020] Yuanyuan Lyu, Wei-An Lin, Haofu Liao, Jingjing Lu, and S Kevin Zhou. Encoding metal mask pro- jection for metal artifact reduction in computed tomogra- phy. In International Conference on Medical Image Com- puting and Computer Assisted Intervention, pages 147- 157, 2020.","title":"Encoding metal mask projection for metal artifact reduction in computed tomography","authors":"Yuanyuan Lyu, Wei-An Lin, Haofu Liao, Jingjing Lu, Kevin Zhou","year":"2020","venue":"International Conference on Medical Image Com-puting and Computer Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Meyer et al., 2010] Esther Meyer, Rainer Raupach, Michael Lell, Bernhard Schmidt, and Marc Kachelrieß. Normal- ized metal artifact reduction (NMAR) in computed tomog- raphy. Medical Physics, 37(10):5482-5493, 2010.","title":"Normalized metal artifact reduction (NMAR) in computed tomography","authors":"Esther Meyer, Rainer Raupach, Michael Lell, Bernhard Schmidt, Marc Kachelrieß","year":"2010","venue":"Medical Physics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Wang et al., 2018] Jianing Wang, Yiyuan Zhao, Jack H No- ble, and Benoit M Dawant. Conditional generative adver- sarial networks for metal artifact reduction in CT images of the ear. In International Conference on Medical Image Computing and Computer Assisted Intervention, pages 3- 11, 2018.","title":"Conditional generative adversarial networks for metal artifact reduction in CT images of the ear","authors":"Jianing Wang, Yiyuan Zhao, Jack H Noble, Benoit M Dawant","year":"2018","venue":"International Conference on Medical Image Computing and Computer Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Wang et al., 2021a] Hong Wang, Yuexiang Li, Nanjun He, Kai Ma, Deyu Meng, and Yefeng Zheng. DICDNet: Deep interpretable convolutional dictionary network for metal artifact reduction in CT images. IEEE Transactions on Medical Imaging, 2021.","title":"DICDNet: Deep interpretable convolutional dictionary network for metal artifact reduction in CT images","authors":"Wang","year":"2021","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Wang et al., 2021b] Hong Wang, Yuexiang Li, Haimiao Zhang, Jiawei Chen, Kai Ma, Deyu Meng, and Yefeng Zheng. InDuDoNet: An interpretable dual domain net- work for CT metal artifact reduction. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pages 107-118, 2021.","title":"InDuDoNet: An interpretable dual domain network for CT metal artifact reduction","authors":"Wang","year":"2021","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wang et al., 2021c] Hong Wang, Yuexiang Li, Haimiao Zhang, Deyu Meng, and Yefeng Zheng. InDuDoNet+: A model-driven interpretable dual domain network for metal artifact reduction in CT images. arXiv preprint arXiv:2112.12660, 2021.","title":"InDuDoNet+: A model-driven interpretable dual domain network for metal artifact reduction in CT images","authors":"Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2021d] Hong Wang, Qi Xie, Qian Zhao, Yong Liang, and Deyu Meng. RCDNet: An interpretable rain convolutional dictionary network for single image derain- ing. arXiv preprint arXiv:2107.06808, 2021.","title":"RCDNet: An interpretable rain convolutional dictionary network for single image deraining","authors":"Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Xie et al., 2019] Qi Xie, Minghao Zhou, Qian Zhao, Deyu Meng, Wangmeng Zuo, and Zongben Xu. Multispectral and hyperspectral image fusion by MS/HS fusion net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1585-1594, 2019.","title":"Multispectral and hyperspectral image fusion by MS/HS fusion net","authors":"Qi Xie, Minghao Zhou, Qian Zhao, Deyu Meng, Wangmeng Zuo, Zongben Xu","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yan et al., 2018] Ke Yan, Xiaosong Wang, Le Lu, Ling Zhang, Adam P Harrison, Mohammadhadi Bagheri, and Ronald M Summers. Deep lesion graphs in the wild: Rela- tionship learning and organization of significant radiology image findings in a diverse large-scale lesion database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9261-9270, 2018.","title":"Deep lesion graphs in the wild: Relationship learning and organization of significant radiology image findings in a diverse large-scale lesion database","authors":"Ke Yan, Xiaosong Wang, Le Lu, Ling Zhang, Adam P Harrison, Mohammadhadi Bagheri, Ronald M Summers","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yu et al., 2020] Lequan Yu, Zhicheng Zhang, Xiaomeng Li, and Lei Xing. Deep sinogram completion with image prior for metal artifact reduction in CT images. IEEE Transac- tions on Medical Imaging, 40(1):228-238, 2020.","title":"Deep sinogram completion with image prior for metal artifact reduction in CT images","authors":"Lequan Yu, Zhicheng Zhang, Xiaomeng Li, Lei Xing","year":"2020","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang and Yu, 2018] Yanbo Zhang and Hengyong Yu. Con- volutional neural network based metal artifact reduction in X-ray computed tomography. IEEE Transactions on Med- ical Imaging, 37(6):1370-1381, 2018.","title":"Convolutional neural network based metal artifact reduction in X-ray computed tomography","authors":"Yu ; ] Yanbo Zhang, Hengyong Zhang, Yu","year":"2018","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2011] Xiaomeng Zhang, Jing Wang, and Lei Xing. Metal artifact reduction in X-ray computed tomog- raphy (CT) by constrained optimization. Medical Physics, 38(2):701-711, 2011.","title":"Metal artifact reduction in X-ray computed tomography (CT) by constrained optimization","authors":"Xiaomeng Zhang, Jing Wang, Lei Xing","year":"2011","venue":"Medical Physics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}