{"type":"ScholarlyArticle","title":"Dual-view Correlation Hybrid Attention Network for Robust Holistic Mammogram Classification","authors":[{"name":"Zhiwei Wang","slug":"zhiwei-wang","given":"Zhiwei","family":"Wang","affiliation":"Britton Chance Center for Biomedical Photonics , Wuhan National Laboratory for Optoelectronics , Huazhong University of Science and Technology; MoE Key Laboratory for Biomedical Photonics , Collaborative Innovation Center for Biomedical Engineering , School of Engineering Sciences , Huazhong University of Science and Technology"},{"name":"Junlin Xian","slug":"junlin-xian","given":"Junlin","family":"Xian","affiliation":"School of Electronic Information and Communications , Huazhong University of Science and Technology"},{"name":"Kangyi Liu","slug":"kangyi-liu","given":"Kangyi","family":"Liu","affiliation":"School of Electronic Information and Communications , Huazhong University of Science and Technology"},{"name":"Xin Li","slug":"xin-li","given":"Xin","family":"Li","affiliation":"Britton Chance Center for Biomedical Photonics , Wuhan National Laboratory for Optoelectronics , Huazhong University of Science and Technology; MoE Key Laboratory for Biomedical Photonics , Collaborative Innovation Center for Biomedical Engineering , School of Engineering Sciences , Huazhong University of Science and Technology"},{"name":"Qiang Li","slug":"qiang-li","given":"Qiang","family":"Li","affiliation":"Britton Chance Center for Biomedical Photonics , Wuhan National Laboratory for Optoelectronics , Huazhong University of Science and Technology; MoE Key Laboratory for Biomedical Photonics , Collaborative Innovation Center for Biomedical Engineering , School of Engineering Sciences , Huazhong University of Science and Technology"},{"name":"Xin Yang","slug":"xin-yang","given":"Xin","family":"Yang","affiliation":"School of Electronic Information and Communications , Huazhong University of Science and Technology"}],"abstract":"Mammogram image is important for breast cancer screening, and typically obtained in a dual-view form, i.e., cranio-caudal (CC) and mediolateral oblique (MLO), to provide complementary information for clinical decisions. However, previous methods mostly learn features from the two views independently, which violates the clinical knowledge and ignores the importance of dual-view correlation in the feature learning. In this paper, we propose a dual-view correlation hybrid attention network (DCHA-Net) for robust holistic mammogram classification. Specifically, DCHA-Net is carefully designed to extract and reinvent deep feature maps for the two views, and meanwhile to maximize the underlying correlations between them. A hybrid attention module, consisting of local relation and non-local attention blocks, is proposed to alleviate the spatial misalignment of the paired views in the correlation maximization. A dual-view correlation loss is introduced to maximize the feature similarity between corresponding strip-like regions with equal distance to the chest wall, motivated by the fact that their features represent the same breast tissues, and thus should be highly-correlated with each other. Experimental results on the two public datasets, i.e., INbreast and CBIS-DDSM, demonstrate that the DCHA-Net can well preserve and maximize feature correlations across views, and thus outperforms previous state-of-the-art methods for classifying a whole mammogram as malignant or not.","doi":"10.24963/ijcai.2023/168","url":"https://ijcai.org/proceedings/2023/168","pdf":"https://ijcai.org/proceedings/2023/0168.pdf","markdown":"https://ijcai.org/proceedings/2023/168.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/168","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":168,"pages":{"first":1515,"last":1523},"keywords":["Machine Learning: ML: Multi-view learning","Machine Learning: ML: Knowledge-aided learning","Computer Vision: CV: Biomedical image analysis"],"videos":[],"references":[{"position":1,"raw":"[AlGhamdi and Abdel-Mottaleb, 2021] Manal AlGhamdi and Mohamed Abdel-Mottaleb. Dv-dcnn: Dual-view deep convolutional neural network for matching de- tected masses in mammograms. Computer methods and programs in biomedicine, 207:106152, 2021.","title":"Manal AlGhamdi and Mohamed Abdel-Mottaleb. Dv-dcnn: Dual-view deep convolutional neural network for matching detected masses in mammograms","authors":"Abdel-Mottaleb Alghamdi","year":"2021","venue":"Computer methods and programs in biomedicine","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arevalo et al., 2016] John Arevalo, Fabio A González, Raúl Ramos-Pollán, Jose L Oliveira, and Miguel Angel Gue- vara Lopez. Representation learning for mammography mass lesion classification with convolutional neural net- works. Computer methods and programs in biomedicine, 127:248-257, 2016.","title":"Representation learning for mammography mass lesion classification with convolutional neural networks","authors":"John Arevalo, Fabio A González, Raúl Ramos-Pollán, Jose L Oliveira, Miguel Angel, Guevara Lopez","year":"2016","venue":"Computer methods and programs in biomedicine","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bae et al., 2014] Min Sun Bae, Woo Kyung Moon, Jung Min Chang, Hye Ryoung Koo, Won Hwa Kim, Nariya Cho, Ann Yi, Bo La Yun, Su Hyun Lee, Mi Young Kim, et al. Breast cancer detected with screening us: reasons for nondetection at mammography. Radiology, 270(2):369-377, 2014.","title":"Breast cancer detected with screening us: reasons for nondetection at mammography","authors":"Bae","year":"2014","venue":"Radiology","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bekker et al., 2015] Alan Joseph Bekker, Moran Shalhon, Hayit Greenspan, and Jacob Goldberger. Multi-view prob- abilistic classification of breast microcalcifications. IEEE Transactions on medical imaging, 35(2):645-653, 2015.","title":"Multi-view probabilistic classification of breast microcalcifications","authors":"Alan Joseph Bekker, Moran Shalhon, Hayit Greenspan, Jacob Goldberger","year":"2015","venue":"IEEE Transactions on medical imaging","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2021] Zhenjie Cao, Zhicheng Yang, Yuxing Tang, Yanbo Zhang, Mei Han, Jing Xiao, Jie Ma, and Peng Chang. Supervised contrastive pre-training formam- mographic triage screening models. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part VII 24, pages 129-139. Springer, 2021.","title":"Supervised contrastive pre-training formammographic triage screening models","authors":"Zhenjie Cao, Zhicheng Yang, Yuxing Tang, Yanbo Zhang, Mei Han, Jing Xiao, Jie Ma, Peng Chang","year":"2021","venue":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Carneiro et al., 2015] Gustavo Carneiro, Jacinto Nasci- mento, and Andrew P Bradley. Unregistered multiview mammogram analysis with pre-trained deep learning mod- els. In International Conference on Medical Image Com- puting and Computer-Assisted Intervention, pages 652- 660. Springer, 2015.","title":"Unregistered multiview mammogram analysis with pre-trained deep learning models","authors":"Gustavo Carneiro, Jacinto Nascimento, Andrew P Bradley","year":"2015","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Carneiro et al., 2017] Gustavo Carneiro, Jacinto Nasci- mento, and Andrew P Bradley. Automated analysis of unregistered multi-view mammograms with deep learn- ing. IEEE transactions on medical imaging, 36(11):2355- 2365, 2017.","title":"Automated analysis of unregistered multi-view mammograms with deep learning","authors":"Gustavo Carneiro, Jacinto Nascimento, Andrew P Bradley","year":"2017","venue":"IEEE transactions on medical imaging","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dhungel et al., 2016] Neeraj Dhungel, Gustavo Carneiro, and Andrew P Bradley. The automated learning of deep features for breast mass classification from mammograms. In International Conference on Medical Image Comput- ing and Computer-Assisted Intervention, pages 106-114. Springer, 2016.","title":"The automated learning of deep features for breast mass classification from mammograms","authors":"Neeraj Dhungel, Gustavo Carneiro, Andrew P Bradley","year":"2016","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Domingues et al., 2012] I Domingues, E Sales, J Cardoso, and W Pereira. Inbreast-database masses characterization. XXIII CBEB, 2012.","title":"Inbreast-database masses characterization","authors":"Domingues","year":"2012","venue":"XXIII CBEB","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[El-Naqa et al., 2002] Issam El-Naqa, Yongyi Yang, Miles N Wernick, Nikolas P Galatsanos, and Robert M Nishikawa. A support vector machine approach for detection of mi- crocalcifications. IEEE transactions on medical imaging, 21(12):1552-1563, 2002.","title":"A support vector machine approach for detection of microcalcifications","authors":"Issam El-Naqa, Yongyi Yang, Nikolas P Miles N Wernick, Robert M Galatsanos, Nishikawa","year":"2002","venue":"IEEE transactions on medical imaging","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"He","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Helgason and Helgason, 1980] Sigurdur Helgason and S Helgason. The radon transform, volume 2. Springer, 1980.","title":"The radon transform","authors":"Sigurdur Helgason, Helgason","year":"1980","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hosseini et al., 2016] Hedayatollah Hosseini, Milan MS Obradović, Martin Hoffmann, Kathryn L Harper, Maria Soledad Sosa, Melanie Werner-Klein, Lahiri Kanth Nanduri, Christian Werno, Carolin Ehrl, Matthias Ma- neck, et al. Early dissemination seeds metastasis in breast cancer. Nature, 540(7634):552-558, 2016.","title":"Early dissemination seeds metastasis in breast cancer","authors":"Hosseini","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hu et al., 2019] Han Hu, Zheng Zhang, Zhenda Xie, and Stephen Lin. Local relation networks for image recogni- tion. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 3464-3473, 2019.","title":"Local relation networks for image recognition","authors":"Han Hu, Zheng Zhang, Zhenda Xie, Stephen Lin","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lee et al., 2017] Rebecca Sawyer Lee, Francisco Gimenez, Assaf Hoogi, Kanae Kawai Miyake, Mia Gorovoy, and Daniel L Rubin. A curated mammography data set for use in computer-aided detection and diagnosis research. Sci- entific data, 4(1):1-9, 2017.","title":"A curated mammography data set for use in computer-aided detection and diagnosis research","authors":"Rebecca Sawyer, Lee, Francisco Gimenez, Assaf Hoogi, Kanae Kawai Miyake, Mia Gorovoy, Daniel L Rubin","year":"2017","venue":"Scientific data","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2021] Dong Li, Lituan Wang, Ting Hu, Lei Zhang, and Qing Lv. Deep multiinstance mammogram classifi- cation with region label assignment strategy and metric- based optimization. IEEE Transactions on Cognitive and Developmental Systems, 14(4):1717-1728, 2021.","title":"Deep multiinstance mammogram classification with region label assignment strategy and metricbased optimization","authors":"Dong Li, Lituan Wang, Ting Hu, Lei Zhang, Qing Lv","year":"2021","venue":"IEEE Transactions on Cognitive and Developmental Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liberman and Menell, 2002] Laura Liberman and Jen- nifer H Menell. Breast imaging reporting and data system (bi-rads). Radiologic Clinics, 40(3):409-430, 2002.","title":"Breast imaging reporting and data system (bi-rads)","authors":"Laura Liberman, Jennifer H Menell","year":"2002","venue":"Radiologic Clinics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lu et al., 2018] Chia-Feng Lu, Fei-Ting Hsu, Kevin Li- Chun Hsieh, Yu-Chieh Jill Kao, Sho-Jen Cheng, Justin Bo-Kai Hsu, Ping-Huei Tsai, Ray-Jade Chen, Chao-Ching Huang, Yun Yen, et al. Machine learning-based radiomics for molecular subtyping of gliomas. Clinical Cancer Re- search, 24(18):4429-4436, 2018.","title":"Machine learning-based radiomics for molecular subtyping of gliomas","authors":"Chia-Feng Lu, Fei-Ting Hsu, Kevin Li-Chun, Yu-Chieh Jill Hsieh, Sho-Jen Kao, Justin Cheng, Bo-Kai Hsu, Ping-Huei Tsai, Ray-Jade Chen, Chao-Ching Huang, Yun Yen","year":"2018","venue":"Clinical Cancer Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ma et al., 2021] Jiechao Ma, Xiang Li, Hongwei Li, Ruix- uan Wang, Bjoern Menze, and Wei-Shi Zheng. Cross- view relation networks for mammogram mass detection. In 2020 25th International Conference on Pattern Recog- nition (ICPR), pages 8632-8638. IEEE, 2021.","title":"Crossview relation networks for mammogram mass detection","authors":"Jiechao Ma, Xiang Li, Hongwei Li, Ruixuan Wang, Bjoern Menze, Wei-Shi Zheng","year":"2020","venue":"25th International Conference on Pattern Recognition (ICPR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Moreira et al., 2012] Inês C Moreira, Igor Amaral, Inês Domingues, António Cardoso, Maria Joao Cardoso, and Jaime S Cardoso. Inbreast: toward a full-field digital mam- mographic database. Academic radiology, 19(2):236-248, 2012.","title":"Inbreast: toward a full-field digital mammographic database","authors":"Igor Inês C Moreira, Inês Amaral, António Domingues, Maria Joao Cardoso, Jaime S Cardoso, Cardoso","year":"2012","venue":"Academic radiology","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Moss et al., 2012] SM Moss, Lennarth Nyström, Hakan Jonsson, E Paci, E Lynge, S Njor, and M Broeders. The impact of mammographic screening on breast cancer mor- tality in europe: a review of trend studies. Journal of med- ical screening, 19(1 suppl):26-32, 2012.","title":"The impact of mammographic screening on breast cancer mortality in europe: a review of trend studies","authors":"Moss","year":"2012","venue":"Journal of medical screening","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Petrini et al., 2022] Daniel GP Petrini, Carlos Shimizu, Rosimeire A Roela, Gabriel Vansuita Valente, Maria Aparecida Azevedo Koike Folgueira, and Hae Yong Kim. Breast cancer diagnosis in two-view mammography using end-to-end trained efficientnet-based convolutional net- work. Ieee Access, 10:77723-77731, 2022.","title":"Breast cancer diagnosis in two-view mammography using end-to-end trained efficientnet-based convolutional network","authors":"Carlos Daniel Gp Petrini, Rosimeire A Shimizu, Gabriel Vansuita Roela, Maria Valente, Hae Yong Aparecida Azevedo Koike Folgueira, Kim","year":"2022","venue":"Ieee Access","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Redmon and Farhadi, 2018] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018.","title":"Yolov3: An incremental improvement","authors":"Joseph Redmon, Ali Farhadi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Selvaraju et al., 2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on com- puter vision, pages 618-626, 2017.","title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","authors":"Michael Ramprasaath R Selvaraju, Abhishek Cogswell, Ramakrishna Das, Devi Vedantam, Dhruv Parikh, Batra","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shams et al., 2018] Shayan Shams, Richard Platania, Jian Zhang, Joohyun Kim, Kisung Lee, and Seung-Jong Park. Deep generative breast cancer screening and diagnosis. In International Conference on Medical Image Comput- ing and Computer-Assisted Intervention, pages 859-867. Springer, 2018.","title":"Deep generative breast cancer screening and diagnosis","authors":"Shayan Shams, Richard Platania, Jian Zhang, Joohyun Kim, Kisung Lee, Seung-Jong Park","year":"2018","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shu et al., 2020] Xin Shu, Lei Zhang, Zizhou Wang, Qing Lv, and Zhang Yi. Deep neural networks with region- based pooling structures for mammographic image clas- sification. IEEE Transactions on Medical Imaging, 39(6):2246-2255, 2020.","title":"Deep neural networks with regionbased pooling structures for mammographic image classification","authors":"Xin Shu, Lei Zhang, Zizhou Wang, Qing Lv, Zhang Yi","year":"2020","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2018] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7794-7803, 2018.","title":"Non-local neural networks","authors":"Xiaolong Wang, Ross Girshick, Abhinav Gupta, Kaiming He","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xian et al., 2021] Junlin Xian, Zhiwei Wang, Kwang-Ting Cheng, and Xin Yang. Towards robust dual-view transfor- mation via densifying sparse supervision for mammogra- phy lesion matching. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th In- ternational Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part V 24, pages 355- 365. Springer, 2021.","title":"Towards robust dual-view transformation via densifying sparse supervision for mammography lesion matching","authors":"Xian","year":"2021","venue":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yan et al., 2021a] Yutong Yan, Pierre-Henri Conze, Math- ieu Lamard, Gwenolé Quellec, Béatrice Cochener, and Gouenou Coatrieux. Towards improved breast mass de- tection using dual-view mammogram matching. Medical image analysis, 71:102083, 2021.","title":"Towards improved breast mass detection using dual-view mammogram matching","authors":"Yan","year":"2021","venue":"Medical image analysis","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yan et al., 2021b] Yutong Yan, Pierre-Henri Conze, Math- ieu Lamard, Heng Zhang, Gwenolé Quellec, Béatrice Cochener, and Gouenou Coatrieux. Deep active learning for dual-view mammogram analysis. In Machine Learn- ing in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 12, pages 180-189. Springer, 2021.","title":"Deep active learning for dual-view mammogram analysis","authors":"Yan","year":"2021","venue":"Machine Learning in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yao et al., 2017] Jiawen Yao, Xinliang Zhu, Feiyun Zhu, and Junzhou Huang. Deep correlational learning for sur- vival prediction from multi-modality data. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pages 406-414. Springer, 2017.","title":"Deep correlational learning for survival prediction from multi-modality data","authors":"Jiawen Yao, Xinliang Zhu, Feiyun Zhu, Junzhou Huang","year":"2017","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhou et al., 2019] Yuyin Zhou, Yingwei Li, Zhishuai Zhang, Yan Wang, Angtian Wang, Elliot K Fishman, Alan L Yuille, and Seyoun Park. Hyper-pairing net- work for multi-phase pancreatic ductal adenocarcinoma segmentation. In International conference on medical im- age computing and computer-assisted intervention, pages 155-163. Springer, 2019.","title":"Hyper-pairing network for multi-phase pancreatic ductal adenocarcinoma segmentation","authors":"Zhou","year":"2019","venue":"International conference on medical image computing and computer-assisted intervention","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhu et al., 2017] Wentao Zhu, Qi Lou, Yeeleng Scott Vang, and Xiaohui Xie. Deep multi-instance networks with sparse label assignment for whole mammogram classifica- tion. In International Conference on Medical Image Com- puting and Computer-Assisted Intervention, pages 603- 611. Springer, 2017.","title":"Deep multi-instance networks with sparse label assignment for whole mammogram classification","authors":"Wentao Zhu, Qi Lou, Yeeleng Scott Vang, Xiaohui Xie","year":"2017","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}