{"type":"ScholarlyArticle","title":"VF-Detector: Making Multi-Granularity Code Changes on Vulnerability Fix Detector Robust to Mislabeled Changes","authors":[{"name":"Zhenkan Fu","slug":"zhenkan-fu","given":"Zhenkan","family":"Fu","affiliation":"School of Information Science and Technology , Dalian Maritime University , Dalian , China; Dalian Key Laboratory of Artificial Intelligence , DUT Artificial Intelligence Institute , Dalian , China"},{"name":"Shikai Guo","slug":"shikai-guo","given":"Shikai","family":"Guo","affiliation":"School of Information Science and Technology , Dalian Maritime University , Dalian , China; Dalian Key Laboratory of Artificial Intelligence , DUT Artificial Intelligence Institute , Dalian , China"},{"name":"Hui Li","slug":"hui-li","given":"Hui","family":"Li","affiliation":"School of Information Science and Technology , Dalian Maritime University , Dalian , China"},{"name":"Rong Chen","slug":"rong-chen","given":"Rong","family":"Chen","affiliation":"School of Information Science and Technology , Dalian Maritime University , Dalian , China"},{"name":"Xiaochen Li","slug":"xiaochen-li","given":"Xiaochen","family":"Li","affiliation":"School of Software , Dalian University of Technology , Dalian , China"},{"name":"He Jiang","slug":"he-jiang","given":"He","family":"Jiang","affiliation":"Dalian Key Laboratory of Artificial Intelligence , DUT Artificial Intelligence Institute , Dalian , China; School of Software , Dalian University of Technology , Dalian , China"}],"abstract":"As software development projects increasingly rely on open-source software, users face the risk of security vulnerabilities from third-party libraries. To address label and character noise in code changes, we present VF-Detector to automatically identifying bug-fix commits in actual noise development environment. VF-Detector consists of three componments: Data Pre-processing (DP), Vulnerability Confidence Computation (VCC) and Confidence Learning Denoising (CLD). The DP component is responsible for preprocessing code change data. The VCC component calculates code change confidence value for each bug-fix by extracting features at various granularity levels. The CLD component removes noise and enhances model robustness by pruning noisy data with confidence values and performing effort-aware adjustments. Experimental results demonstrate VF-Detector's superiority over state-of-the-art methods in EffortCost@L and Popt@L metrics on Java and Python datasets. The improvements were 6.5% and 5% for Java, and 23.4% and 17.8% for Python.","doi":"10.24963/ijcai.2024/643","url":"https://ijcai.org/proceedings/2024/643","pdf":"https://ijcai.org/proceedings/2024/0643.pdf","markdown":"https://ijcai.org/proceedings/2024/643.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/643","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":643,"pages":{"first":5817,"last":5825},"keywords":["Machine Learning: ML: Applications","Data Mining: DM: Applications","Multidisciplinary Topics and Applications: MTA: Software engineering","Agent-based and Multi-agent Systems: MAS: Trust and reputation"],"videos":[],"references":[{"position":1,"raw":"[Carvalho et al., 2014] Marco M. Carvalho, Jared DeMott, Richard Ford, and David A. Wheeler. Heartbleed 101. IEEE Secur. Priv., 12(4):63-67, 2014.","title":"Heartbleed 101","authors":"M Marco, Jared Carvalho, Richard Demott, David A Ford, Wheeler","year":"2014","venue":"IEEE Secur. Priv","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Feng et al., 2020] Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. Code- bert: A pre-trained model for programming and natural languages. In Trevor Cohn, Yulan He, and Yang Liu, ed- itors, Findings of the Association for Computational Lin- guistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Findings of ACL, pages 1536-1547. Association for Computational Linguistics, 2020.","title":"Codebert: A pre-trained model for programming and natural languages","authors":"Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, Ming Zhou","year":"2020","venue":"Findings of the Association for Computational Linguistics: EMNLP 2020","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Hoang et al., 2019] Thong Hoang, Hoa Khanh Dam, Yasu- taka Kamei, David Lo, and Naoyasu Ubayashi. Deepjit: an end-to-end deep learning framework for just-in-time de- fect prediction. In Margaret-Anne D. Storey, Bram Adams, and Sonia Haiduc, editors, Proceedings of the 16th In- ternational Conference on Mining Software Repositories, MSR 2019, 26-27 May 2019, Montreal, Canada, pages 34-45. IEEE / ACM, 2019.","title":"Deepjit: an end-to-end deep learning framework for just-in-time defect prediction","authors":"Thong Hoang, Hoa Khanh Dam, Yasutaka Kamei, David Lo, Naoyasu Ubayashi","year":"2019","venue":"Proceedings of the 16th International Conference on Mining Software Repositories","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Imtiaz et al., 2023] Nasif Imtiaz, Aniqa Khanom, and Lau- rie A. Williams. Open or sneaky? fast or slow? light or heavy?: Investigating security releases of open source packages. IEEE Trans. Software Eng., 49(4):1540-1560, 2023.","title":"Open or sneaky? fast or slow? light or heavy?: Investigating security releases of open source packages","authors":"Nasif Imtiaz, Aniqa Khanom, Laurie A Williams","year":"2023","venue":"IEEE Trans. Software Eng","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kang et al., 2022] Hong Jin Kang, Truong Giang Nguyen, Xuan-Bach Dinh Le, Corina S. Pasareanu, and David Lo. Test mimicry to assess the exploitability of library vulner- abilities. In Sukyoung Ryu and Yannis Smaragdakis, ed- itors, ISSTA '22: 31st ACM SIGSOFT International Sym- posium on Software Testing and Analysis, Virtual Event, South Korea, July 18 -22, 2022, pages 276-288. ACM, 2022.","title":"Test mimicry to assess the exploitability of library vulnerabilities","authors":"Kang","year":"2022","venue":"ISSTA '22: 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Con- ference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Pro- ceedings, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"3rd International Conference on Learning Representations, ICLR 2015","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Le et al., 2021] Triet Huynh Minh Le, David Hin, Roland Croft, and Muhammad Ali Babar. Deepcva: Automated commit-level vulnerability assessment with deep multi- task learning. In 36th IEEE/ACM International Confer- ence on Automated Software Engineering, ASE 2021, Mel- bourne, Australia, November 15-19, 2021, pages 717-729. IEEE, 2021.","title":"Deepcva: Automated commit-level vulnerability assessment with deep multitask learning","authors":"Triet Huynh, Minh Le, David Hin, Roland Croft, Muhammad Ali Babar","year":"2021","venue":"36th IEEE/ACM International Conference on Automated Software Engineering, ASE 2021","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Mashhadi and Hemmati, 2021] Ehsan Mashhadi and Hadi Hemmati. Applying codebert for automated program re- pair of java simple bugs. In 18th IEEE/ACM International Conference on Mining Software Repositories, MSR 2021, Madrid, Spain, May 17-19, 2021, pages 505-509. IEEE, 2021.","title":"Applying codebert for automated program repair of java simple bugs","authors":"Hemmati Mashhadi, Ehsan Mashhadi, Hadi Hemmati","year":"2021","venue":"18th IEEE/ACM International Conference on Mining Software Repositories, MSR 2021","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Nguyen et al., 2022] Truong Giang Nguyen, Thanh Le- Cong, Hong Jin Kang, Xuan-Bach Dinh Le, and David Lo. Vulcurator: a vulnerability-fixing commit detector. In Abhik Roychoudhury, Cristian Cadar, and Miryung Kim, editors, Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022, Singapore, Singapore, November 14-18, 2022, pages 1726-1730. ACM, 2022.","title":"Vulcurator: a vulnerability-fixing commit detector","authors":"Truong Giang Nguyen, Thanh Le-Cong, Hong, Jin Kang, Xuan-Bach Dinh Le, David Lo","year":"2022","venue":"Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Nguyen et al., 2023] Truong Giang Nguyen, Thanh Le- Cong, Hong Jin Kang, Ratnadira Widyasari, Chengran Yang, Zhipeng Zhao, Bowen Xu, Jiayuan Zhou, Xin Xia, Ahmed E. Hassan, Xuan-Bach Dinh Le, and David Lo. Multi-granularity detector for vulnerability fixes. IEEE Trans. Software Eng., 49(8):4035-4057, 2023.","title":"Multi-granularity detector for vulnerability fixes","authors":"Truong Giang Nguyen, Thanh Le-Cong, Hong, Jin Kang, Ratnadira Widyasari, Chengran Yang, Zhipeng Zhao, Bowen Xu, Jiayuan Zhou, Xin Xia, Ahmed E Hassan, Xuan-Bach Dinh Le, David Lo","year":"2023","venue":"IEEE Trans. Software Eng","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Nguyen-Truong et al., 2022] Giang Nguyen-Truong, Hong Jin Kang, David Lo, Abhishek Sharma, An- drew E. Santosa, Asankhaya Sharma, and Ming Yi Ang. HERMES: using commit-issue linking to detect vulnerability-fixing commits. In IEEE International Conference on Software Analysis, Evolution and Reengi- neering, SANER 2022, Honolulu, HI, USA, March 15-18, 2022, pages 51-62. IEEE, 2022.","title":"HERMES: using commit-issue linking to detect vulnerability-fixing commits","authors":"Giang Nguyen-Truong, Hong, Jin Kang, David Lo, Abhishek Sharma, Andrew E Santosa, Asankhaya Sharma, Ming Yi, Ang","year":"2022","venue":"IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Northcutt et al., 2021] Curtis G. Northcutt, Lu Jiang, and Isaac L. Chuang. Confident learning: Estimating uncer- tainty in dataset labels. J. Artif. Intell. Res., 70:1373-1411, 2021.","title":"Confident learning: Estimating uncertainty in dataset labels","authors":"Curtis G Northcutt, Lu Jiang, Isaac L Chuang","year":"2021","venue":"J. Artif. Intell. Res","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Perl et al., 2015] Henning Perl, Sergej Dechand, Matthew Smith, Daniel Arp, Fabian Yamaguchi, Konrad Rieck, Sascha Fahl, and Yasemin Acar. Vccfinder: Finding potential vulnerabilities in open-source projects to assist code audits. In Indrajit Ray, Ninghui Li, and Christopher Kruegel, editors, Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, CO, USA, October 12-16, 2015, pages 426-437. ACM, 2015.","title":"Vccfinder: Finding potential vulnerabilities in open-source projects to assist code audits","authors":"Sergej Henning Perl, Matthew Dechand, Daniel Smith, Fabian Arp, Konrad Yamaguchi, Sascha Rieck, Yasemin Fahl, Acar","year":"2015","venue":"Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ponta et al., 2019] Serena Elisa Ponta, Henrik Plate, An- tonino Sabetta, Michele Bezzi, and Cédric Dangremont. A manually-curated dataset of fixes to vulnerabilities of open-source software. In Margaret-Anne D. Storey, Bram Adams, and Sonia Haiduc, editors, Proceedings of the 16th International Conference on Mining Software Repos- itories, MSR 2019, 26-27 May 2019, Montreal, Canada, pages 383-387. IEEE / ACM, 2019.","title":"A manually-curated dataset of fixes to vulnerabilities of open-source software","authors":"Ponta","year":"2019","venue":"Proceedings of the 16th International Conference on Mining Software Repositories","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[VF-Detector, 2024] VF-Detector. Our replication package. https://github.com/buildcb/VF-Detector, 2024.","title":"VF-Detector. Our replication package","authors":null,"year":"2024","venue":null,"doi":null,"url":"https://github.com/buildcb/VF-Detector,2024","internal_url":null},{"position":16,"raw":"[Xia and Zhang, 2022] Chunqiu Steven Xia and Lingming Zhang. Less training, more repairing please: revisiting au- tomated program repair via zero-shot learning. In Abhik Roychoudhury, Cristian Cadar, and Miryung Kim, editors, Proceedings of the 30th ACM Joint European Software En- gineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022, Singapore, Sin- gapore, November 14-18, 2022, pages 959-971. ACM, 2022.","title":"Less training, more repairing please: revisiting automated program repair via zero-shot learning","authors":"Zhang ; Chunqiu Xia, Steven Xia, Lingming Zhang","year":"2022","venue":"Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Yang et al., 2022] Zhou Yang, Jieke Shi, Junda He, and David Lo. Natural attack for pre-trained models of code. In 44th IEEE/ACM 44th International Conference on Soft- ware Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022, pages 1482-1493. ACM, 2022.","title":"Natural attack for pre-trained models of code","authors":"Zhou Yang, Jieke Shi, Junda He, David Lo","year":"2022","venue":"44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zeng et al., 2021] Zhengran Zeng, Yuqun Zhang, Haotian Zhang, and Lingming Zhang. Deep just-in-time defect prediction: how far are we? In Cristian Cadar and Xi- angyu Zhang, editors, ISSTA '21: 30th ACM SIGSOFT In- ternational Symposium on Software Testing and Analysis, Virtual Event, Denmark, July 11-17, 2021, pages 427-438. ACM, 2021.","title":"Deep just-in-time defect prediction: how far are we? In Cristian Cadar and Xiangyu Zhang","authors":"Zeng","year":"2021","venue":"ISSTA '21: 30th ACM SIGSOFT International Symposium on Software Testing and Analysis","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhong et al., 2022] Hao Zhong, Xiaoyin Wang, and Hong Mei. Inferring bug signatures to detect real bugs. IEEE Trans. Software Eng., 48(2):571-584, 2022.","title":"Inferring bug signatures to detect real bugs","authors":"Hao Zhong, Xiaoyin Wang, Hong Mei","year":"2022","venue":"IEEE Trans. Software Eng","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhou and Sharma, 2017] Yaqin Zhou and Asankhaya Sharma. Automated identification of security issues from commit messages and bug reports. In Eric Bodden, Wilhelm Schäfer, Arie van Deursen, and Andrea Zisman, editors, Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, Paderborn, Germany, September 4-8, 2017, pages 914-919. ACM, 2017.","title":"Automated identification of security issues from commit messages and bug reports","authors":"Sharma Zhou, Yaqin Zhou, Asankhaya Sharma","year":"2017","venue":"Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhou et al., 2021] Jiayuan Zhou, Michael Pacheco, Zhiyuan Wan, Xin Xia, David Lo, Yuan Wang, and Ahmed E. Hassan. Finding A needle in a haystack: Automated mining of silent vulnerability fixes. In 36th IEEE/ACM International Conference on Automated Software Engineering, ASE 2021, Melbourne, Australia, November 15-19, 2021, pages 705-716. IEEE, 2021.","title":"Finding A needle in a haystack: Automated mining of silent vulnerability fixes","authors":"Zhou","year":"2021","venue":"36th IEEE/ACM International Conference on Automated Software Engineering, ASE 2021","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhou et al., 2022] Yaqin Zhou, Jing Kai Siow, Chenyu Wang, Shangqing Liu, and Yang Liu. SPI: automated iden- tification of security patches via commits. ACM Trans. Softw. Eng. Methodol., 31(1):13:1-13:27, 2022.","title":"SPI: automated identification of security patches via commits","authors":"Yaqin Zhou, Jing, Kai Siow, Chenyu Wang, Shangqing Liu, Yang Liu","year":"2022","venue":"ACM Trans. Softw. Eng. Methodol","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}