{"type":"ScholarlyArticle","title":"PENTESTLLMAGENT: A Task Dependency Graph Planning-Based Multi-Agent Framework for Automated Penetration Testing","authors":[{"name":"Shuo Sheng","slug":"shuo-sheng","given":"Shuo","family":"Sheng","affiliation":""},{"name":"Jixin Zhang","slug":"jixin-zhang","given":"Jixin","family":"Zhang","affiliation":""},{"name":"Jia Yang","slug":"jia-yang","given":"Jia","family":"Yang","affiliation":""},{"name":"Ke Cheng","slug":"ke-cheng","given":"Ke","family":"Cheng","affiliation":""},{"name":"Zheng Qin","slug":"zheng-qin","given":"Zheng","family":"Qin","affiliation":""}],"abstract":"Fully autonomous IP-to-Root penetration testing remains challenging for LLM agents. We conduct an exploratory study on 10 LLMs and introduce AutoPentest-Bench, an end-to-end benchmark with 13 VulnHub targets and 93 sub-tasks. From 130 interaction logs, we identify three challenges: Rigid Strategy, Contextual Forgetting, and Command Generation Hallucination. To address them, we propose PentestLLMAgent, which integrates a Task Dependency Graph (TDG) for dynamic planning and backtracking; a Hierarchical Multi-Agent Architecture (HMA) with function-calling-based tool invocation, output filtering, and semantic compression, and Executable Knowledge-Guided Command Generation (EKG-CG) for retrieving and executing pre-validated, environment-compatible commands. Evaluations demonstrate strong effectiveness: on end-to-end AutoPentest-Bench, PentestLLMAgent achieves a 77% success rate; on AutoPT’s web exploitation benchmark, it attains a 95% overall pass rate; and on the privilege escalation benchmark, it achieves a 100% success rate. In realistic end-to-end runs, it averages 10.9 minutes, 36 interaction rounds, and 69.6K tokens per target. The code, benchmark, and executable knowledge base are publicly available at https://github.com/sanbai123/PentestLLMAgent_code-and-videos.","doi":"10.24963/ijcai.2026/725","url":"https://ijcai.org/proceedings/2026/725","pdf":"https://ijcai.org/proceedings/2026/0725.pdf","markdown":"https://ijcai.org/proceedings/2026/725.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/725","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"AI4Tech: AI Enabling Technologies","track":"AI4Tech: AI Enabling Technologies","number":725,"pages":{"first":6518,"last":6527},"keywords":["Advanced AI4Tech: Generative and LLMs-driven AI4Tech","Domain-specific AI4Tech: AI4Security"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}