{"type":"ScholarlyArticle","title":"AgentRetro: Agent-Enhanced Molecular Spectral Domain Generalization Framework for Retrosynthesis Prediction Under Mixed OOD Shifts","authors":[{"name":"Jiahang Shen","slug":"jiahang-shen","given":"Jiahang","family":"Shen","affiliation":""},{"name":"Hongxin Xiang","slug":"hongxin-xiang","given":"Hongxin","family":"Xiang","affiliation":""},{"name":"Yiping Liu","slug":"yiping-liu","given":"Yiping","family":"Liu","affiliation":""}],"abstract":"Retrosynthesis prediction is a cornerstone of drug discovery, enabling the synthesis of novel therapeutic candidates. However, current deep learning models falter when navigating the unexplored chemical space essential for innovation. In these realistic scenarios, models face a challenging mixed out-of-distribution (OOD) shift: they must handle both novel molecular scaffolds (covariate shift) and unseen reaction templates (label shift), leading to severe performance degradation. To address this, we propose AgentRetro, a framework integrating spectral domain generalization with Large Language Model (LLM) agents. It features a Spectral Domain Enhancer (SDE) that partitions chemical space via spectral clustering to capture diverse domain structures, and a Multi-Knowledge Agent (MKA) that enriches data with LLM-generated OOD reactions and textual rationales. These components are unified by a dual training strategy: meta-learning for template adaptability and adversarial learning for structural invariance. Crucially, our analysis reveals that explicitly decoupling structural invariance from mechanistic adaptability is essential for overcoming the interference between conflicting distribution shifts. Experiments show AgentRetro consistently improves state-of-the-art baselines, notably boosting EditRetro’s OOD Top-10 accuracy from 41.7% to 50.1%. The source code is available at https://github.com/jiahangshen/IJCAI26-AgentRetro.","doi":"10.24963/ijcai.2026/39","url":"https://ijcai.org/proceedings/2026/39","pdf":"https://ijcai.org/proceedings/2026/0039.pdf","markdown":"https://ijcai.org/proceedings/2026/39.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/39","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":39,"pages":{"first":340,"last":347},"keywords":["Agent-based and Multi-agent Systems: Applications","Machine Learning: Meta-learning","Machine Learning: Open-World/Open-Set/OOD Learning","Machine Learning: Adversarial machine learning","Multidisciplinary Topics and Applications: Bioinformatics"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}