{"type":"ScholarlyArticle","title":"Augmenting Automated Spectrum Based Fault Localization for Multiple Faults","authors":[{"name":"Prantik Chatterjee","slug":"prantik-chatterjee","given":"Prantik","family":"Chatterjee","affiliation":"Indian Institute of Technology Kanpur , India"},{"name":"Jose Campos","slug":"jose-campos","given":"Jose","family":"Campos","affiliation":"Faculty of Engineering of University of Porto , Porto , Portugal; LASIGE , Faculdade de Ciencias , Universidade de Lisboa , Lisboa , Portugal"},{"name":"Rui Abreu","slug":"rui-abreu","given":"Rui","family":"Abreu","affiliation":"Faculty of Engineering of University of Porto , Porto , Portugal; INESC-ID , Porto , Portugal"},{"name":"Subhajit Roy","slug":"subhajit-roy","given":"Subhajit","family":"Roy","affiliation":"Indian Institute of Technology Kanpur , India"}],"abstract":"Spectrum-based Fault Localization (SBFL) uses the coverage of test cases and their outcome (pass/fail) to predict the \"suspiciousness'' of program components, e.g., lines of code. SBFL is, perhaps, the most successful fault localization technique due to its simplicity and scalability. However, SBFL heuristics do not perform well in scenarios where a program may have multiple faulty components. In this work, we propose a new algorithm that \"augments'' previously proposed SBFL heuristics to produce a ranked list where faulty components ranked low by base SBFL metrics are ranked significantly higher. We implement our ideas in a tool, ARTEMIS, that attempts to \"bubble up'' faulty components which are ranked lower by base SBFL metrics. We compare our technique to the most popular SBFL metrics and demonstrate statistically significant improvement in the developer effort for fault localization with respect to the basic strategies.","doi":"10.24963/ijcai.2023/350","url":"https://ijcai.org/proceedings/2023/350","pdf":"https://ijcai.org/proceedings/2023/0350.pdf","markdown":"https://ijcai.org/proceedings/2023/350.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/350","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":350,"pages":{"first":3140,"last":3148},"keywords":["Multidisciplinary Topics and Applications: MDA: Software engineering","Knowledge Representation and Reasoning: KRR: Diagnosis and abductive reasoning"],"videos":[],"references":[{"position":1,"raw":"[Abreu et al., 2009a] Rui Abreu, Peter Zoeteweij, Rob Gol- steijn, and Arjan JC Van Gemund. A practical evaluation of spectrum-based fault localization. Journal of Systems and Software, 82(11):1780-1792, 2009.","title":"A practical evaluation of spectrum-based fault localization","authors":"Abreu","year":"2009","venue":"Journal of Systems and Software","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Abreu et al., 2009b] Rui Abreu, Peter Zoeteweij, and Ar- jan JC Van Gemund. Localizing software faults simultane- ously. In 2009 Ninth International Conference on Quality Software, pages 367-376. IEEE, 2009.","title":"Localizing software faults simultaneously","authors":"Abreu","year":"2009","venue":"Ninth International Conference on Quality Software","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Abreu et al., 2009c] Rui Abreu, Peter Zoeteweij, and Ar- jan J.C. van Gemund. Spectrum-Based Multiple Fault Lo- calization. In 2009 IEEE/ACM International Conference on Automated Software Engineering, pages 88-99, 2009.","title":"Spectrum-Based Multiple Fault Localization","authors":"Abreu","year":"2009","venue":"IEEE/ACM International Conference on Automated Software Engineering","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Abreu et al., 2011] Rui Abreu, Peter Zoeteweij, and Ar- jan JC Van Gemund. Simultaneous debugging of software faults. Journal of Systems and Software, 84(4):573-586, 2011.","title":"Simultaneous debugging of software faults","authors":"Rui Abreu, Peter Zoeteweij, Arjan Jc Van Gemund","year":"2011","venue":"Journal of Systems and Software","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[B. Le et al., 2016] Tien-Duy B. Le, David Lo, Claire Le Goues, and Lars Grunske. A Learning-to-Rank Based Fault Localization Approach Using Likely Invariants. In Proceedings of the 25th International Symposium on Soft- ware Testing and Analysis, ISSTA 2016, page 177-188, New York, NY, USA, 2016. Association for Computing Machinery.","title":"A Learning-to-Rank Based Fault Localization Approach Using Likely Invariants","authors":"B Tien-Duy, David Le, Claire Le Lo, Lars Goues, Grunske","year":"2016","venue":"Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bandyopadhyay and Ghosh, 2012] Aritra Bandyopadhyay and Sudipto Ghosh. Tester feedback driven fault local- ization. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pages 41- 50, 2012.","title":"Tester feedback driven fault localization","authors":"Aritra Bandyopadhyay, Sudipto Ghosh","year":"2012","venue":"IEEE Fifth International Conference on Software Testing, Verification and Validation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chatterjee et al., 2020] Prantik Chatterjee, Abhijit Chatter- jee, José Campos, Rui Abreu, and Subhajit Roy. Diagnos- ing software faults using multiverse analysis. In Christian Bessiere, editor, Proceedings of the Twenty-Ninth Interna- tional Joint Conference on Artificial Intelligence, IJCAI- 20, pages 1629-1635. International Joint Conferences on Artificial Intelligence Organization, 7 2020. Main track.","title":"Diagnosing software faults using multiverse analysis","authors":"Chatterjee","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20","doi":null,"url":null,"internal_url":"/proceedings/2020/226"},{"position":8,"raw":"[Choi et al., 2010] Seung-Seok Choi, Sung-Hyuk Cha, and Charles C Tappert. A survey of binary similarity and dis- tance measures. Journal of systemics, cybernetics and in- formatics, 8(1):43-48, 2010.","title":"A survey of binary similarity and distance measures","authors":"Seung-Seok Choi, Sung-Hyuk Cha, Charles C Tappert","year":"2010","venue":"Journal of systemics, cybernetics and informatics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Do et al., 2005] Hyunsook Do, Sebastian Elbaum, and Gregg Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering, 10(4):405-435, 2005.","title":"Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact","authors":"Do","year":"2005","venue":"Empirical Software Engineering","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fraser and Arcuri, 2011] Gordon Fraser and Andrea Arcuri. EvoSuite: Automatic Test Suite Generation for Object- Oriented Software. In Proc. of the 19th ACM ESEC/FSE, page 416-419. ACM, 2011.","title":"EvoSuite: Automatic Test Suite Generation for Object-Oriented Software","authors":"Arcuri Fraser, Gordon Fraser, Andrea Arcuri","year":"2011","venue":"Proc. of the 19th ACM ESEC/FSE","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hutchins et al., 1994] Monica Hutchins, Herb Foster, Tarak Goradia, and Thomas Ostrand. Experiments on the effec- tiveness of dataflow-and control-flow-based test adequacy criteria. In Proceedings of 16th International conference on Software engineering, pages 191-200. IEEE, 1994.","title":"Experiments on the effectiveness of dataflow-and control-flow-based test adequacy criteria","authors":"Monica Hutchins, Herb Foster, Tarak Goradia, Thomas Ostrand","year":"1994","venue":"Proceedings of 16th International conference on Software engineering","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jeffrey et al., 2009] Dennis Jeffrey, Neelam Gupta, and Ra- jiv Gupta. Effective and efficient localization of multiple faults using value replacement. In 2009 IEEE Interna- tional Conference on Software Maintenance, pages 221- 230. IEEE, 2009.","title":"Effective and efficient localization of multiple faults using value replacement","authors":"Dennis Jeffrey, Neelam Gupta, Rajiv Gupta","year":"2009","venue":"IEEE International Conference on Software Maintenance","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jones and Harrold, 2005] James A Jones and Mary Jean Harrold. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated soft- ware engineering, pages 273-282, 2005.","title":"Empirical evaluation of the tarantula automatic fault-localization technique","authors":"A James, Mary Jones, Harrold Jean","year":"2005","venue":"Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jones et al., 2002] James A Jones, Mary Jean Harrold, and John Stasko. Visualization of test information to assist fault localization. In Proceedings of the 24th International Conference on Software Engineering. ICSE 2002, pages 467-477. IEEE, 2002.","title":"Visualization of test information to assist fault localization","authors":"James A Jones, Mary Jean Harrold, John Stasko","year":"2002","venue":"Proceedings of the 24th International Conference on Software Engineering. ICSE 2002","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Just et al., 2014] René Just, Darioush Jalali, and Michael D. Ernst. Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proc. of the 23rd ISSTA, page 437-440. ACM, 2014.","title":"Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs","authors":"Just","year":"2014","venue":"Proc. of the 23rd ISSTA","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li and Zhang, 2017] Xia Li and Lingming Zhang. Trans- forming Programs and Tests in Tandem for Fault Localiza- tion. Proc. ACM Program. Lang., 1(OOPSLA), oct 2017.","title":"Transforming Programs and Tests in Tandem for Fault Localization","authors":"Xia Li, Lingming Zhang","year":"2017","venue":"Proc. ACM Program. Lang","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Majd et al., 2022] Amirabbas Majd, Mojtaba Vahidi-Asl, Alireza Khalilian, and Babak Bagheri. Consilientsfl: using preferential voting system to generate combinatorial rank- ing metrics for spectrum-based fault localization. Applied Intelligence, pages 1-21, 2022.","title":"Consilientsfl: using preferential voting system to generate combinatorial ranking metrics for spectrum-based fault localization","authors":"Majd","year":"2022","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Naish et al., 2011] Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. A model for spectra-based software di- agnosis. ACM Transactions on software engineering and methodology (TOSEM), 20(3):1-32, 2011.","title":"A model for spectra-based software diagnosis","authors":"Lee Naish, Hua Jie Lee, Kotagiri Ramamohanarao","year":"2011","venue":"ACM Transactions on software engineering and methodology","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pearson et al., 2017] Spencer Pearson, José Campos, René Just, Gordon Fraser, Rui Abreu, Michael D Ernst, Deric Pang, and Benjamin Keller. Evaluating and improving fault localization. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), pages 609- 620. IEEE, 2017.","title":"Evaluating and improving fault localization","authors":"Spencer Pearson, José Campos, René Just, Gordon Fraser, Rui Abreu, D Michael, Deric Ernst, Benjamin Pang, Keller","year":"2017","venue":"IEEE/ACM 39th International Conference on Software Engineering (ICSE)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Perez et al., 2017] Alexandre Perez, Rui Abreu, and Arie van Deursen. A test-suite diagnosability metric for spectrum-based fault localization approaches. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), pages 654-664. IEEE, 2017.","title":"A test-suite diagnosability metric for spectrum-based fault localization approaches","authors":"Alexandre Perez, Rui Abreu, Arie Van Deursen","year":"2017","venue":"IEEE/ACM 39th International Conference on Software Engineering (ICSE)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rosner et al., 2006] Bernard Rosner, Robert J Glynn, and Mei-Ling T Lee. The wilcoxon signed rank test for paired comparisons of clustered data. Biometrics, 62(1):185-192, 2006.","title":"The wilcoxon signed rank test for paired comparisons of clustered data","authors":"Bernard Rosner, Robert J Glynn, Mei-Ling T Lee","year":"2006","venue":"Biometrics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Serra et al., 2019] Domenico Serra, Giovanni Grano, Fabio Palomba, Filomena Ferrucci, Harald C Gall, and Alberto Bacchelli. On the effectiveness of manual and automatic unit test generation: ten years later. In 2019 IEEE/ACM 16th International Conference on Mining Software Repos- itories (MSR), pages 121-125. IEEE, 2019.","title":"On the effectiveness of manual and automatic unit test generation: ten years later","authors":"Domenico Serra, Giovanni Grano, Fabio Palomba, Filomena Ferrucci, Harald C Gall, Alberto Bacchelli","year":"2019","venue":"IEEE/ACM 16th International Conference on Mining Software Repositories (MSR)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Steimann and Frenkel, 2012] Friedrich Steimann and Mar- cus Frenkel. Improving coverage-based localization of multiple faults using algorithms from integer linear pro- gramming. In 2012 IEEE 23rd International Sympo- sium on Software Reliability Engineering, pages 121-130. IEEE, 2012.","title":"Friedrich Steimann and Marcus Frenkel. Improving coverage-based localization of multiple faults using algorithms from integer linear programming","authors":"Frenkel Steimann","year":"2012","venue":"IEEE 23rd International Symposium on Software Reliability Engineering","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wei and Han, 2013] Zheng Wei and Bai Han. Multiple-bug oriented fault localization: A parameter-based combina- tion approach. In 2013 IEEE Seventh International Con- ference on Software Security and Reliability Companion, pages 125-130. IEEE, 2013.","title":"Multiple-bug oriented fault localization: A parameter-based combination approach","authors":"Zheng Wei, Bai Han","year":"2013","venue":"IEEE Seventh International Conference on Software Security and Reliability Companion","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wong et al., 2013] W Eric Wong, Vidroha Debroy, Ruizhi Gao, and Yihao Li. The dstar method for effective soft- ware fault localization. IEEE Transactions on Reliability, 63(1):290-308, 2013.","title":"The dstar method for effective software fault localization","authors":"Eric Wong, Vidroha Debroy, Ruizhi Gao, Yihao Li","year":"2013","venue":"IEEE Transactions on Reliability","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wong et al., 2016] W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. A survey on software fault localization. IEEE Transactions on Software Engineering, 42(8):707-740, 2016.","title":"A survey on software fault localization","authors":"Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, Franz Wotawa","year":"2016","venue":"IEEE Transactions on Software Engineering","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2022] Guyu Zhao, Hongdou He, and Yifang Huang. Fault centrality: boosting spectrum-based fault lo- calization via local influence calculation. Applied Intelli- gence, 52(7):7113-7135, 2022.","title":"Fault centrality: boosting spectrum-based fault localization via local influence calculation","authors":"Guyu Zhao, Hongdou He, Yifang Huang","year":"2022","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zou et al., 2021] Daming Zou, Jingjing Liang, Yingfei Xiong, Michael D. Ernst, and Lu Zhang. An Empiri- cal Study of Fault Localization Families and Their Com- binations. IEEE Transactions on Software Engineering, 47(2):332-347, 2021.","title":"An Empirical Study of Fault Localization Families and Their Combinations","authors":"Daming Zou, Jingjing Liang, Yingfei Xiong, Michael D Ernst, Lu Zhang","year":"2021","venue":"IEEE Transactions on Software Engineering","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}