{"type":"ScholarlyArticle","title":"Lazy Testing of Machine-Learning Models","authors":[{"name":"Anastasia Isychev","slug":"anastasia-isychev","given":"Anastasia","family":"Isychev","affiliation":""},{"name":"Valentin Wüstholz","slug":"valentin-wustholz","given":"Valentin","family":"Wüstholz","affiliation":""},{"name":"Maria Christakis","slug":"maria-christakis","given":"Maria","family":"Christakis","affiliation":""}],"abstract":"Checking the reliability of machine-learning models is a crucial, but challenging task. Nomos is an existing, automated framework for testing general, user-provided functional properties of models, including so-called hyperproperties expressed over more than one model execution. Nomos aims to find model inputs that expose ``bugs'', that is, property violations. However, performing thousands of model invocations during testing is costly both in terms of time and money (for metered APIs, such as OpenAI's).\r\n\r\nWe present LaZ (pronounced ``lazy''), an extension of Nomos that automatically minimizes the number of model invocations to boost the test throughput and thereby find bugs more efficiently. During test execution, LaZ automatically identifies redundant invocations---invocations where the model output does not affect the final test outcome---and skips them, much like lazy evaluation in certain programming languages. This optimization enables a second one that dynamically reorders model invocations to skip the more expensive ones. As a result, LaZ finds the same number of bugs as Nomos, but does so median 33% and up to 60% faster.","doi":"10.24963/ijcai.2025/826","url":"https://ijcai.org/proceedings/2025/826","pdf":"https://ijcai.org/proceedings/2025/0826.pdf","markdown":"https://ijcai.org/proceedings/2025/826.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/826","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":826,"pages":{"first":7428,"last":7436},"keywords":["AI Ethics, Trust, Fairness: ETF: Safety and robustness","Multidisciplinary Topics and Applications: MTA: Software engineering","Agent-based and Multi-agent Systems: MAS: Engineering methods, platforms, languages and tools"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}