{"type":"ScholarlyArticle","title":"Bi-level Probabilistic Feature Learning for Deformable Image Registration","authors":[{"name":"Risheng Liu","slug":"risheng-liu","given":"Risheng","family":"Liu","affiliation":"International School of Information Science & Engineering , Dalian University of Technology; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province"},{"name":"Zi Li","slug":"zi-li","given":"Zi","family":"Li","affiliation":"International School of Information Science & Engineering , Dalian University of Technology; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province"},{"name":"Yuxi Zhang","slug":"yuxi-zhang","given":"Yuxi","family":"Zhang","affiliation":"International School of Information Science & Engineering , Dalian University of Technology; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province"},{"name":"Xin Fan","slug":"xin-fan","given":"Xin","family":"Fan","affiliation":"International School of Information Science & Engineering , Dalian University of Technology; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province"},{"name":"Zhongxuan Luo","slug":"zhongxuan-luo","given":"Zhongxuan","family":"Luo","affiliation":"Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province; School of Software Technology , Dalian University of Technology; Institute of Artificial Intelligence , Guilin University of Electronic Technology"}],"abstract":"We address the challenging issue of deformable registration that robustly and efficiently builds dense correspondences between images. Traditional approaches upon iterative energy optimization typically invoke expensive computational load. Recent learning-based methods are able to efficiently predict deformation maps by incorporating learnable deep networks. Unfortunately, these deep networks are designated to learn deterministic features for classification tasks, which are not necessarily optimal for registration. In this paper, we propose a novel bi-level optimization model that enables jointly learning deformation maps and features for image registration. The bi-level model takes the energy for deformation computation as the upper-level optimization while formulates the maximum \\emph{a posterior} (MAP) for features as the lower-level optimization. Further, we design learnable deep networks to simultaneously optimize the cooperative bi-level model, yielding robust and efficient registration. These deep networks derived from our bi-level optimization constitute an unsupervised end-to-end framework for learning both features and deformations. Extensive experiments of image-to-atlas and image-to-image deformable registration on 3D brain MR datasets demonstrate that we achieve state-of-the-art performance in terms of accuracy, efficiency, and robustness.","doi":"10.24963/ijcai.2020/101","url":"https://ijcai.org/proceedings/2020/101","pdf":"https://ijcai.org/proceedings/2020/0101.pdf","markdown":"https://ijcai.org/proceedings/2020/101.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/101","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":101,"pages":{"first":723,"last":730},"keywords":["Machine Learning: Unsupervised Learning","Computer Vision: Statistical Methods and Machine Learning","Machine Learning: Deep Learning: Convolutional networks","Computer Vision: Biomedical Image Understanding"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/26924/file_26924.m3u8","length_sec":408,"provider":"hls"}],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. Tensorflow: Large-scale machine learn- ing on heterogeneous distributed systems. CoRR, ab- s/1603.04467, 2016.","title":"Tensorflow: Large-scale machine learning on heterogeneous distributed systems","authors":"Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin","year":"2016","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ashburner, 2007] John Ashburner. A fast diffeomorphic im- age registration algorithm. NeuroImage, 38(1):95-113, 2007.","title":"A fast diffeomorphic image registration algorithm","authors":"John Ashburner","year":"2007","venue":"NeuroImage","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Avants et al., 2008] Brian B Avants, Charles L Epstein, Murray Grossman, and James C Gee. Symmetric diffeo- morphic image registration with cross-correlation: eval- uating automated eling of elderly and neurodegenerative brain. Medical Image Analysis, 12(1):26-41, 2008.","title":"Symmetric diffeomorphic image registration with cross-correlation: evaluating automated eling of elderly and neurodegenerative brain","authors":"Charles L Brian B Avants, Murray Epstein, James C Grossman, Gee","year":"2008","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Avants et al., 2011] Brian B Avants, Nicholas J Tustison, Gang Song, Philip A Cook, Arno Klein, and James C Gee. A reproducible evaluation of ants similarity met- ric performance in brain image registration. NeuroImage, 54(3):2033-2044, 2011.","title":"A reproducible evaluation of ants similarity metric performance in brain image registration","authors":"Nicholas J Brian B Avants, Gang Tustison, Philip A Song, Arno Cook, James C Klein, Gee","year":"2011","venue":"NeuroImage","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bajcsy and Kovacic, 1989] Ruzena Bajcsy and Stanislav Kovacic. Multiresolution elastic matching. Computer Vi- sion, Graphics, and Image Processing, 46(1):1-21, 1989.","title":"Multiresolution elastic matching","authors":"Kovacic Bajcsy, Ruzena Bajcsy, Stanislav Kovacic","year":"1989","venue":"Computer Vision, Graphics, and Image Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Balakrishnan et al., 2019] Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. Vox- elmorph: A learning framework for deformable medical image registration. IEEE Transactions on Medical Imag- ing, 38(8):1788-1800, 2019.","title":"Voxelmorph: A learning framework for deformable medical image registration","authors":"Guha Balakrishnan, Amy Zhao, R Mert, John Sabuncu, Adrian V Guttag, Dalca","year":"2019","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Banerjee et al., 2019] Sayan Banerjee, Avik Hati, Subhasis Chaudhuri, and Rajbabu Velmurugan. Cosegnet: Image co-segmentation using a conditional siamese convolution- al network. In IJCAI, pages 673-679, 2019.","title":"Cosegnet: Image co-segmentation using a conditional siamese convolutional network","authors":"Sayan Banerjee, Avik Hati, Subhasis Chaudhuri, Rajbabu Velmurugan","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/95"},{"position":8,"raw":"[Cheng et al., 2018] Gong Cheng, Decheng Gao, Yang Liu, and Junwei Han. Multi-scale and discriminative part de- tectors based features for multi-label image classification. In IJCAI, pages 649-655, 2018.","title":"Multi-scale and discriminative part detectors based features for multi-label image classification","authors":"Gong Cheng, Decheng Gao, Yang Liu, Junwei Han","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/90"},{"position":9,"raw":"[Colson et al., 2005] Benoît Colson, Patrice Marcotte, and Gilles Savard. Bilevel programming: A survey. A Quar- terly Journal of Operations Research, 3(2):87-107, 2005.","title":"Bilevel programming: A survey","authors":"Benoît Colson, Patrice Marcotte, Gilles Savard","year":"2005","venue":"A Quarterly Journal of Operations Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dalca et al., 2019] Adrian V Dalca, Guha Balakrishnan, John Guttag, and Mert R Sabuncu. Unsupervised learn- ing of probabilistic diffeomorphic registration for images and surfaces. Medical Image Analysis, 57:226-236, 2019.","title":"Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces","authors":"Adrian V Dalca, Guha Balakrishnan, John Guttag, Mert R Sabuncu","year":"2019","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[de Vos et al., 2019] Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Hessam Sokooti, Marius Staring, and Ivana Isgum. A deep learning framework for unsupervised affine and deformable image registration. Medical Image Analysis, 52:128-143, 2019.","title":"A deep learning framework for unsupervised affine and deformable image registration","authors":"D Bob, Floris F De Vos, Max A Berendsen, Hessam Viergever, Marius Sokooti, Ivana Staring, Isgum","year":"2019","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Di Martino et al., 2014] Adriana Di Martino, Chao-Gan Yan, Qingyang Li, Erin Denio, Francisco X Castellanos, Kaat Alaerts, Jeffrey S Anderson, Michal Assaf, Susan Y Bookheimer, Mirella Dapretto, et al. The Autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in Autism. Molecular psychiatry, 19(6):659-667, 2014.","title":"The Autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in Autism","authors":"Adriana Di, Martino, Chao-Gan Yan, Qingyang Li, Erin Denio, Kaat Francisco X Castellanos, Alaerts, Michal Jeffrey S Anderson, Susan Y Assaf, Mirella Bookheimer, Dapretto","year":"2014","venue":"Molecular psychiatry","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Dice, 1945] Lee Dice. Measures of the amount of ecolog- ic association between species. Ecology, 26(3):297-302, 1945.","title":"Measures of the amount of ecologic association between species","authors":"Lee Dice","year":"1945","venue":"Ecology","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Dosovitskiy et al., 2015] Alexey Dosovitskiy, Philipp Fis- cher, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with con- volutional networks. In IEEE International Conference on Computer Vision, pages 2758-2766, 2015.","title":"Flownet: Learning optical flow with convolutional networks","authors":"Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der, Daniel Smagt, Thomas Cremers, Brox","year":"2015","venue":"IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2019] Xiaojun Hu, Miao Kang, Weilin Huang, Matthew R Scott, Roland Wiest, and Mauricio Reyes. Dual-stream pyramid registration network. In MICCAI, pages 382-390, 2019.","title":"Dual-stream pyramid registration network","authors":"Xiaojun Hu, Miao Kang, Weilin Huang, Matthew R Scott, Roland Wiest, Mauricio Reyes","year":"2019","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jaderberg et al., 2015] Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. In NeurIPS, pages 2017-2025, 2015.","title":"Spatial transformer networks","authors":"Jaderberg","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. ADAM: A method for stochastic optimization. In ICLR, 2015.","title":"ADAM: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Klein et al., 2010] Stefan Klein, Marius Staring, Keelin Murphy, Max A Viergever, and Josien PW Pluim. elastix: A toolbox for intensity-based medical image registration. IEEE Transactions on Medical Imaging, 29(1):196-205, 2010.","title":"elastix: A toolbox for intensity-based medical image registration","authors":"Stefan Klein, Marius Staring, Keelin Murphy, Max A Viergever, Josien Pw Pluim","year":"2010","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2018a] Risheng Liu, Shichao Cheng, Yi He, Xin Fan, Zhouchen Lin, and Zhongxuan Luo. On the conver- gence of learning-based iterative methods for nonconvex inverse problems. CoRR, abs/1808.05331, 2018.","title":"On the convergence of learning-based iterative methods for nonconvex inverse problems","authors":"Liu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2018b] Risheng Liu, Shichao Cheng, Xiaokun Liu, Long Ma, Xin Fan, and Zhongxuan Luo. A bridging framework for model optimization and deep propagation. In NeurIPS, pages 4323-4332, 2018.","title":"A bridging framework for model optimization and deep propagation","authors":"Liu","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2019a] Risheng Liu, Long Ma, Yiyang Wang, and Lei Zhang. Learning converged propagations with deep prior ensemble for image enhancement. IEEE Trans- actions on Image Processing, 28(3):1528-1543, 2019.","title":"Learning converged propagations with deep prior ensemble for image enhancement","authors":"Liu","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2019b] Risheng Liu, Yuxi Zhang, Shichao Cheng, Xin Fan, and Zhongxuan Luo. A theoretically guaranteed deep optimization framework for robust com- pressive sensing MRI. In AAAI, pages 4368-4375, 2019.","title":"A theoretically guaranteed deep optimization framework for robust compressive sensing MRI","authors":"Liu","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu et al., 2020] Risheng Liu, Zi Li, Yuxi Zhang, Chenying Zhao, Hao Huang, Zhongxuan Luo, and Xin Fan. A multi- scale optimization learning framework for diffeomorphic deformable registration. CoRR, abs/2004.14557, 2020.","title":"A multiscale optimization learning framework for diffeomorphic deformable registration","authors":"Risheng Liu, Zi Li, Yuxi Zhang, Chenying Zhao, Hao Huang, Zhongxuan Luo, Xin Fan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Maintz and Viergever, 1998] J. B. Antoine Maintz and Max A. Viergever. A survey of medical image registra- tion. Medical Image Analysis, 2(1):1-36, 1998.","title":"A survey of medical image registration","authors":"J B, Antoine Maintz, Max A Viergever","year":"1998","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Marcus et al., 2007] Daniel S Marcus, Tracy H Wang, Jamie Parker, John G Csernansky, John C Morris, and Randy L Buckner. Open access series of imaging studies (oasis): Cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of Cog- nitive Neuroscience, 19(9):1498-1507, 2007.","title":"Open access series of imaging studies (oasis): Cross-sectional mri data in young, middle aged, nondemented, and demented older adults","authors":"Tracy H Daniel S Marcus, Jamie Wang, John G Parker, John C Csernansky, Randy L Morris, Buckner","year":"2007","venue":"Journal of Cognitive Neuroscience","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Marek et al., 2011] Kenneth Marek, Danna Jennings, Shirley Lasch, Andrew Siderowf, Caroline Tanner, Tanya Simuni, Chris Coffey, Karl Kieburtz, Emily Flagg, Sohini Chowdhury, et al. The parkinson progression marker initiative. Progress in Neurobiology, 95(4):629-635, 2011.","title":"The parkinson progression marker initiative","authors":"Kenneth Marek, Danna Jennings, Shirley Lasch, Andrew Siderowf, Caroline Tanner, Tanya Simuni, Chris Coffey, Karl Kieburtz, Emily Flagg, Sohini Chowdhury","year":"2011","venue":"Progress in Neurobiology","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Mueller et al., 2005] Susanne G Mueller, Michael W Wein- er, Leon J Thal, Ronald C Petersen, Clifford R Jack, William Jagust, John Q Trojanowski, Arthur W Toga, and Laurel Beckett. Ways toward an early diagnosis in Alzheimer's disease: The Alzheimer's disease neuroimag- ing initiative. Alzheimer's Dementia, 1(1):55-66, 2005.","title":"Ways toward an early diagnosis in Alzheimer's disease: The Alzheimer's disease neuroimaging initiative","authors":"Susanne G Mueller, Michael W Weiner, Leon J Thal, Ronald C Petersen, Clifford R Jack, William Jagust, John Q Trojanowski, Arthur W Toga, Laurel Beckett","year":"2005","venue":"Alzheimer's Dementia","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Shen et al., 2019] Zhengyang Shen, Xu Han, Zhenlin X- u, and Marc Niethammer. Networks for joint affine and non-parametric image registration. In IEEE CVPR, pages 4224-4233, 2019.","title":"Networks for joint affine and non-parametric image registration","authors":"Shen","year":"2019","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sun et al., 2014] Wei Sun, Wiro J Niessen, and Stefan K- lein. Free-form deformation using lower-order B-spline for nonrigid image registration. In MICCAI, pages 194- 201, 2014.","title":"Free-form deformation using lower-order B-spline for nonrigid image registration","authors":"Wei Sun, Stefan Wiro J Niessen, Klein","year":"2014","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sun et al., 2018] Deqing Sun, Xiaodong Yang, Ming-Yu Li- u, and Jan Kautz. Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In IEEE CVPR, pages 8934-8943, 2018.","title":"Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume","authors":"Deqing Sun, Xiaodong Yang, Ming-Yu Liu, Jan Kautz","year":"2018","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2017] Xiao Yang, Roland Kwitt, Martin Styner, and Marc Niethammer. Quicksilver: Fast predictive im- age registration -A deep learning approach. NeuroImage, 158:378-396, 2017.","title":"Quicksilver: Fast predictive image registration -A deep learning approach","authors":"Xiao Yang, Roland Kwitt, Martin Styner, Marc Niethammer","year":"2017","venue":"NeuroImage","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}