{"type":"ScholarlyArticle","title":"Improving Paratope and Epitope Prediction by Multi-Modal Contrastive Learning and Interaction Informativeness Estimation","authors":[{"name":"Zhiwei Wang","slug":"zhiwei-wang","given":"Zhiwei","family":"Wang","affiliation":"College of Informatics , Huazhong Agricultural University , Wuhan 430070 , China"},{"name":"Yongkang Wang","slug":"yongkang-wang","given":"Yongkang","family":"Wang","affiliation":"College of Informatics , Huazhong Agricultural University , Wuhan 430070 , China"},{"name":"Wen Zhang","slug":"wen-zhang","given":"Wen","family":"Zhang","affiliation":"College of Informatics , Huazhong Agricultural University , Wuhan 430070 , China; Hubei Key Laboratory of Agricultural Bioinformatics , Huazhong Agricultural University , Wuhan 430070 , China; Engineering Research Center of Intelligent Technology for Agriculture , Ministry of Education , Wuhan 430070 , China"}],"abstract":"Accurately predicting antibody-antigen binding residues, i.e., paratopes and epitopes, is crucial in antibody design. However, existing methods solely focus on uni-modal data (either sequence or structure), disregarding the complementary information present in multi-modal data, and most methods predict paratopes and epitopes separately, overlooking their specific spatial interactions. In this paper, we propose a novel Multi-modal contrastive learning and Interaction informativeness estimation-based method for Paratope and Epitope prediction, named MIPE, by using both sequence and structure data of antibodies and antigens. MIPE implements a multi-modal contrastive learning strategy, which maximizes representations of binding and non-binding residues within each modality and meanwhile aligns uni-modal representations towards effective modal representations. To exploit the spatial interaction information, MIPE also incorporates an interaction informativeness estimation that computes the estimated interaction matrices between antibodies and antigens, thereby approximating them to the actual ones. Extensive experiments demonstrate the superiority of our method in predicting paratopes and epitopes compared to baselines. Additionally, the ablation studies and visualizations demonstrate the superiority of MIPE owing to the better representations acquired through multi-modal contrastive learning and the interaction patterns comprehended by the interaction informativeness estimation.","doi":"10.24963/ijcai.2024/669","url":"https://ijcai.org/proceedings/2024/669","pdf":"https://ijcai.org/proceedings/2024/0669.pdf","markdown":"https://ijcai.org/proceedings/2024/669.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/669","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":669,"pages":{"first":6053,"last":6061},"keywords":["Multidisciplinary Topics and Applications: MTA: Health and medicine","Multidisciplinary Topics and Applications: MTA: Bioinformatics"],"videos":[],"references":[{"position":1,"raw":"[Akbar et al., 2021] Rahmad Akbar, Philippe A Robert, Milena Pavlović, Jeliazko R Jeliazkov, Igor Snapkov, Andrei Slabodkin, Cédric R Weber, Lonneke Scheffer, Enkelejda Miho, Ingrid Hobæk Haff, et al. A compact vocabulary of paratope-epitope interactions enables pre- dictability of antibody-antigen binding. Cell Reports, 34(11), 2021.","title":"A compact vocabulary of paratope-epitope interactions enables predictability of antibody-antigen binding","authors":"Rahmad Akbar, Philippe A Robert, Milena Pavlović, R Jeliazko, Igor Jeliazkov, Andrei Snapkov, Slabodkin, R Cédric, Lonneke Weber, Enkelejda Scheffer, Ingrid Miho, Hobæk Haff","year":"2021","venue":"Cell Reports","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bax and Grzesiek, 1993] A. Bax and S. Grzesiek. Method- ological advances in protein nmr. Accounts of Chemical Research, 26:131-138, 1993.","title":"Methodological advances in protein nmr","authors":"Grzesiek Bax, A Bax, S Grzesiek","year":"1993","venue":"Accounts of Chemical Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Campbell et al., 2021] Sharon M Campbell, Joseph DeBar- tolo, James R Apgar, Lydia Mosyak, Virginie McManus, Sonia Beyer, Eric M Bennett, Matthew Lambert, and Orla Cunningham. Combining random mutagenesis, structure- guided design and next-generation sequencing to mitigate polyreactivity of an anti-il-21r antibody. In MAbs, vol- ume 13, page 1883239.","title":"Combining random mutagenesis, structureguided design and next-generation sequencing to mitigate polyreactivity of an anti-il-21r antibody","authors":"Sharon M Campbell, Joseph Debartolo, Lydia James R Apgar, Virginie Mosyak, Sonia Mcmanus, Eric M Beyer, Matthew Bennett, Orla Lambert, Cunningham","year":null,"venue":"MAbs","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Taylor & Francis, 2021.","title":null,"authors":"Francis Taylor","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In In- ternational conference on machine learning, pages 1597- 1607. PMLR, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chinery et al., 2022] Lewis Chinery, Newton Wahome, Iain Moal, and Charlotte M Deane. Paragraphâ C\"antibody paratope prediction using graph neural networks with min- imal feature vectors. Bioinformatics, 39(1):btac732, 11 2022.","title":"Paragraphâ C\"antibody paratope prediction using graph neural networks with minimal feature vectors","authors":"Lewis Chinery, Newton Wahome, Iain Moal, Charlotte M Deane","year":"2022","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Clifford et al., 2022] Joakim Nøddeskov Clifford, Mag- nus Haraldson Høie, Sebastian Deleuran, Bjoern Peters, Morten Nielsen, and Paolo Marcatili. Bepipred-3.0: Im- proved b-cell epitope prediction using protein language models. Protein Science, 31(12):e4497, 11 2022.","title":"Bepipred-3.0: Improved b-cell epitope prediction using protein language models","authors":"Clifford","year":"2022","venue":"Protein Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Daberdaku and Ferrari, 2018] Sebastian Daberdaku and Carlo Ferrari. Antibody interface prediction with 3D Zernike descriptors and SVM. Bioinformatics, 35(11):1870-1876, 11 2018.","title":"Sebastian Daberdaku and Carlo Ferrari. Antibody interface prediction with 3D Zernike descriptors and SVM","authors":"Ferrari Daberdaku","year":"2018","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deac et al., 2019] Andreea Deac, Petar Veli Čković, and Pietro Sormanni. Attentive cross-modal paratope predic- tion. Journal of Computational Biology, 26(6):536-545, 2019.","title":"Attentive cross-modal paratope prediction","authors":"Andreea Deac, Petar Veli Čković, Pietro Sormanni","year":"2019","venue":"Journal of Computational Biology","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Del Vecchio et al., 2021] Alice Del Vecchio, Andreea Deac, Pietro Liò, and Petar Veličković. Neural message passing for joint paratope-epitope prediction. In The 2021 ICML Workshop on Computational Biology, 2021.","title":"Neural message passing for joint paratope-epitope prediction","authors":"Del Vecchio","year":"2021","venue":"The 2021 ICML Workshop on Computational Biology","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dransfield et al., 2010] Daniel T Dransfield, Edward H Co- hen, Qing Chang, Lindsay G Sparrow, John D Bent- ley, Olan Dolezal, Xiaowen Xiao, Thomas S Peat, Janet Newman, Patricia A Pilling, et al. A human mono- clonal antibody against insulin-like growth factor-ii blocks the growth of human hepatocellular carcinoma cell lines in vitro and in vivo. Molecular cancer therapeutics, 9(6):1809-1819, 2010.","title":"A human monoclonal antibody against insulin-like growth factor-ii blocks the growth of human hepatocellular carcinoma cell lines in vitro and in vivo","authors":"Edward H Daniel T Dransfield, Qing Cohen, Lindsay G Chang, John D Sparrow, Olan Bentley, Xiaowen Dolezal, Thomas S Xiao, Janet Peat, Patricia A Newman, Pilling","year":"2010","venue":"Molecular cancer therapeutics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dutta et al., 2015] Sanjib Dutta, Amy E Keating, et al. Sortcery-a high-throughput method to affinity rank pep- tide ligands. Journal of molecular biology, 427(11):2135- 2150, 2015.","title":"Sortcery-a high-throughput method to affinity rank peptide ligands","authors":"Sanjib Dutta, Amy E Keating","year":"2015","venue":"Journal of molecular biology","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[García-Sánchez et al., 2023] Miguel García-Sánchez, Mario Castro, and José Faro. B cell receptors and free antibodies have different antigen-binding kinet- ics. Proceedings of the National Academy of Sciences, 120(35):e2220669120, 2023.","title":"Miguel García-Sánchez, Mario Castro, and José Faro. B cell receptors and free antibodies have different antigen-binding kinetics","authors":"García-Sánchez","year":"2023","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Graves et al., 2020] Jordan Graves, Jacob Byerly, Eduardo Priego, Naren Makkapati, S. Vince Parish, Brenda Medellin, and Monica Berrondo. A review of deep learn- ing methods for antibodies. Antibodies, 9(2), 2020.","title":"A review of deep learning methods for antibodies","authors":"Graves","year":"2020","venue":"Antibodies","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Honda et al., 2020] Shion Honda, Kyohei Koyama, and Kamiya Kotaro. Cross attentive antibody-antigen inter- action prediction with multi-task learning. In The 2020 ICML Workshop on Computational Biology, 2020.","title":"Cross attentive antibody-antigen interaction prediction with multi-task learning","authors":"Honda","year":"2020","venue":"The 2020 ICML Workshop on Computational Biology","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huang et al., 2015] Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991, 2015.","title":"Bidirectional lstm-crf models for sequence tagging","authors":"Zhiheng Huang, Wei Xu, Kai Yu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Krapp et al., 2023] Lucien F Krapp, Luciano A Abriata, Fabio Cortés Rodriguez, and Matteo Dal Peraro. Pesto: parameter-free geometric deep learning for accurate pre- diction of protein binding interfaces. Nature Communica- tions, 14(1):2175, 2023.","title":"Pesto: parameter-free geometric deep learning for accurate prediction of protein binding interfaces","authors":"Lucien F Krapp, Luciano A Abriata, Fabio Cortés Rodriguez, Matteo Dal Peraro","year":"2023","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li and Godzik, 2006] Weizhong Li and Adam Godzik. Cd- hit: a fast program for clustering and comparing large sets of protein or nucleotide sequences. Bioinformatics, 22(13):1658-1659, 05 2006.","title":"Cdhit: a fast program for clustering and comparing large sets of protein or nucleotide sequences","authors":"Weizhong Li, Adam Godzik","year":"2006","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liberis et al., 2018] Edgar Liberis, Petar Veličković, Pietro Sormanni, Michele Vendruscolo, and Pietro Liò. Parapred: antibody paratope prediction using convolutional and re- current neural networks. Bioinformatics, 34(17):2944- 2950, 04 2018.","title":"Parapred: antibody paratope prediction using convolutional and recurrent neural networks","authors":"Edgar Liberis, Petar Veličković, Pietro Sormanni, Michele Vendruscolo, Pietro Liò","year":"2018","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2023] Shengchao Liu, Yutao Zhu, Jiarui Lu, Zhao Xu, Weili Nie, Anthony Gitter, Chaowei Xiao, Jian Tang, Hongyu Guo, and Anima Anandkumar. A Text-guided Protein Design Framework, February 2023.","title":"A Text-guided Protein Design Framework","authors":"Shengchao Liu, Yutao Zhu, Jiarui Lu, Zhao Xu, Weili Nie, Anthony Gitter, Chaowei Xiao, Jian Tang, Hongyu Guo, Anima Anandkumar","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Olimpieri et al., 2013] Pier Paolo Olimpieri, Anna Chai- lyan, Anna Tramontano, and Paolo Marcatili. Predic- tion of site-specific interactions in antibody-antigen com- plexes: the proABC method and server. Bioinformatics, 29(18):2285-2291, 06 2013.","title":"Prediction of site-specific interactions in antibody-antigen complexes: the proABC method and server","authors":"Olimpieri","year":"2013","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Olsen et al., 2022] Tobias H Olsen, Iain H Moal, and Char- lotte M Deane. Ablang: an antibody language model for completing antibody sequences. Bioinformatics Advances, 2(1):vbac046, 2022.","title":"Ablang: an antibody language model for completing antibody sequences","authors":"Iain H Tobias H Olsen, Charlotte M Moal, Deane","year":"2022","venue":"Bioinformatics Advances","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pittala and Bailey-Kellogg, 2020] Srivamshi Pittala and Chris Bailey-Kellogg. Learning context-aware structural representations to predict antigen and antibody binding interfaces. Bioinformatics, 36(13):3996-4003, 04 2020.","title":"Srivamshi Pittala and Chris Bailey-Kellogg. Learning context-aware structural representations to predict antigen and antibody binding interfaces","authors":"Bailey-Kellogg Pittala","year":"2020","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rahate et al., 2022] Anil Rahate, Rahee Walambe, Sheela Ramanna, and Ketan Kotecha. Multimodal co-learning: Challenges, applications with datasets, recent advances and future directions. Inf. Fusion, 81(C):203-239, may 2022.","title":"Multimodal co-learning: Challenges, applications with datasets, recent advances and future directions","authors":"Anil Rahate, Rahee Walambe, Sheela Ramanna, Ketan Kotecha","year":"2022","venue":"Inf. Fusion","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rangel et al., 2022] Mauricio Aguilar Rangel, Alice Bed- well, Elisa Costanzi, Ross J. Taylor, Rosaria Russo, Gon Ȃ §alo J. L. Bernardes, Stefano Ricagno, Ju- dith Frydman, Michele Vendruscolo, and Pietro Sor- manni. Fragment-based computational design of anti- bodies targeting structured epitopes. Science Advances, 8(45):eabp9540, 2022.","title":"Fragment-based computational design of antibodies targeting structured epitopes","authors":"Mauricio Aguilar Rangel, Alice Bedwell, Elisa Costanzi, Ross J Taylor, Rosaria Russo, Ȃ Gon, J L Bernardes, Stefano Ricagno, Judith Frydman, Michele Vendruscolo, Pietro Sormanni","year":"2022","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rives et al., 2021] Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scal- ing unsupervised learning to 250 million protein se- quences. Proceedings of the National Academy of Sci- ences, 118(15):e2016239118, 2021.","title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","authors":"Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma","year":"2021","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schotte et al., 2003] Friedrich Schotte, Manho Lim, Timo- thy A. Jackson, Aleksandr V. Smirnov, Jayashree Soman, John S. Olson, George N. Phillips, Michael Wulff, and Philip A. Anfinrud. Watching a protein as it functions with 150-ps time-resolved x-ray crystallography. Science, 300(5627):1944-1947, 2003.","title":"Watching a protein as it functions with 150-ps time-resolved x-ray crystallography","authors":"Friedrich Schotte, Manho Lim, Timothy A Jackson, Aleksandr V Smirnov, Jayashree Soman, John S Olson, George N Phillips, Michael Wulff, Philip A Anfinrud","year":"2003","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2024] Yongkang Wang, Xuan Liu, Feng Huang, Zhankun Xiong, and Wen Zhang. A multi-modal contrastive diffusion model for therapeutic peptide gener- ation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(1):3-11, Mar. 2024.","title":"A multi-modal contrastive diffusion model for therapeutic peptide generation","authors":"Wang","year":"2024","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xia et al., 2023] Xiaoqiong Xia, Chaoyu Zhu, Fan Zhong, and Lei Liu. MDTips: a multimodal-data-based drug-target interaction prediction system fusing knowl- edge, gene expression profile, and structural data. Bioin- formatics, 39(7):btad411, 06 2023.","title":"MDTips: a multimodal-data-based drug-target interaction prediction system fusing knowledge, gene expression profile, and structural data","authors":"Xia","year":"2023","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu and Zhao, 2022] Haodong Xu and Zhongming Zhao. Netbce: an interpretable deep neural network for accurate prediction of linear b-cell epitopes. Genomics, Proteomics & Bioinformatics, 20(5):1002-1012, 10 2022.","title":"Netbce: an interpretable deep neural network for accurate prediction of linear b-cell epitopes","authors":"Zhao Xu, Haodong Xu, Zhongming Zhao","year":"2022","venue":"Genomics, Proteomics & Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yu and Koltun, 2016] Fisher Yu and Vladlen Koltun. Multi- scale context aggregation by dilated convolutions. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Pro- ceedings, 2016.","title":"Multiscale context aggregation by dilated convolutions","authors":"Koltun ; Fisher Yu, Vladlen Yu, Koltun","year":"2016","venue":"4th International Conference on Learning Representations, ICLR 2016","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang and Kabuka, 2019] Da Zhang and Mansur Kabuka. Multimodal deep representation learning for protein in- teraction identification and protein family classification. BMC bioinformatics, 20(16):1-14, 2019.","title":"Multimodal deep representation learning for protein interaction identification and protein family classification","authors":"Da Zhang, Mansur Kabuka","year":"2019","venue":"BMC bioinformatics","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2022] Zuobai Zhang, Minghao Xu, Ar- ian Rokkum Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. In First Work- shop on Pre-training: Perspectives, Pitfalls, and Paths Forward at ICML 2022, 2022.","title":"Protein representation learning by geometric structure pretraining","authors":"Zhang","year":"2022","venue":"First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward at ICML 2022","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhou, 2008] Z Hong Zhou. Towards atomic resolution structural determination by single-particle cryo-electron microscopy. Current opinion in structural biology, 18(2):218-228, 2008.","title":"Towards atomic resolution structural determination by single-particle cryo-electron microscopy","authors":"Hong Zhou","year":"2008","venue":"Current opinion in structural biology","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhu et al., 2023] Ye Zhu, Yu Wu, Kyle Olszewski, Jian Ren, Sergey Tulyakov, and Yan Yan. Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation, February 2023.","title":"Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation","authors":"Ye Zhu, Yu Wu, Kyle Olszewski, Jian Ren, Sergey Tulyakov, Yan Yan","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zou et al., 2023] Heqing Zou, Meng Shen, Chen Chen, Yuchen Hu, Deepu Rajan, and Eng Siong Chng. Unis-mmc: Multimodal classification via unimodality- supervised multimodal contrastive learning. arXiv preprint arXiv:2305.09299, 2023.","title":"Unis-mmc: Multimodal classification via unimodalitysupervised multimodal contrastive learning","authors":"Zou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}