{"type":"ScholarlyArticle","title":"ResearchEnvBench: Benchmarking Agents on Environment Synthesis for Research Code Execution","authors":[{"name":"Yubang Wang","slug":"yubang-wang","given":"Yubang","family":"Wang","affiliation":""},{"name":"Chenxi Zhang","slug":"chenxi-zhang","given":"Chenxi","family":"Zhang","affiliation":""},{"name":"Bowen Chen","slug":"bowen-chen","given":"Bowen","family":"Chen","affiliation":""},{"name":"Zezheng Huai","slug":"zezheng-huai","given":"Zezheng","family":"Huai","affiliation":""},{"name":"Zihao Dai","slug":"zihao-dai","given":"Zihao","family":"Dai","affiliation":""},{"name":"Xinchi Chen","slug":"xinchi-chen","given":"Xinchi","family":"Chen","affiliation":""},{"name":"Yuxin Wang","slug":"yuxin-wang","given":"Yuxin","family":"Wang","affiliation":""},{"name":"Yining Zheng","slug":"yining-zheng","given":"Yining","family":"Zheng","affiliation":""},{"name":"Jingjing Gong","slug":"jingjing-gong","given":"Jingjing","family":"Gong","affiliation":""},{"name":"Xipeng Qiu","slug":"xipeng-qiu","given":"Xipeng","family":"Qiu","affiliation":""}],"abstract":"Autonomous agents are increasingly expected to support scientific research, and recent benchmarks report progress in code repair and autonomous experimentation. However, these evaluations typically assume a pre-configured execution environment, which requires resolving complex software dependencies, aligning hardware and framework versions, and configuring distributed execution, yet this capability remains largely unbenchmarked. We introduce ResearchEnvBench, a benchmark for environment synthesis in research code execution. Given a research repository, documentation, and a target execution setting, agents must construct an environment that successfully executes at runtime. Evaluations on diverse research repositories reveal a substantial gap in current SOTA agents, with failures dominated by incomplete dependency resolution and brittle version coupling. ResearchEnvBench provides a realistic testbed for advancing autonomous agents toward reproducible scientific research.","doi":"10.24963/ijcai.2026/48","url":"https://ijcai.org/proceedings/2026/48","pdf":"https://ijcai.org/proceedings/2026/0048.pdf","markdown":"https://ijcai.org/proceedings/2026/48.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/48","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":48,"pages":{"first":419,"last":427},"keywords":["Agent-based and Multi-agent Systems: Applications"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}