{"type":"ScholarlyArticle","title":"From End-to-end to Step-by-step: Learning to Abstract via Abductive Reinforcement Learning","authors":[{"name":"Zilong Wang","slug":"zilong-wang","given":"Zilong","family":"Wang","affiliation":""},{"name":"Jiongda Wang","slug":"jiongda-wang","given":"Jiongda","family":"Wang","affiliation":""},{"name":"Xiaoyong Chen","slug":"xiaoyong-chen","given":"Xiaoyong","family":"Chen","affiliation":""},{"name":"Meng Wang","slug":"meng-wang","given":"Meng","family":"Wang","affiliation":""},{"name":"Ming Ma","slug":"ming-ma","given":"Ming","family":"Ma","affiliation":""},{"name":"ZhiPeng Wang","slug":"zhipeng-wang","given":"Zhipeng","family":"Wang","affiliation":""},{"name":"Zhenyu Zhou","slug":"zhenyu-zhou","given":"Zhenyu","family":"Zhou","affiliation":""},{"name":"Tianming Yang","slug":"tianming-yang","given":"Tianming","family":"Yang","affiliation":""},{"name":"Wang-Zhou Dai","slug":"wang-zhou-dai","given":"Wang-Zhou","family":"Dai","affiliation":""}],"abstract":"Abstraction is a critical technique in general problem-solving, allowing complex tasks to be decomposed into smaller, manageable sub-tasks. While traditional symbolic planning relies on predefined primitive symbols to construct structured abstractions, its reliance on formal representations limits applicability to real-world tasks. On the other hand, reinforcement learning excels at learning end-to-end policies directly from sensory inputs in unstructured environments but struggles with compositional generalization in complex tasks with delayed rewards. In this paper, we propose Abductive Abstract Reinforcement Learning (A2RL), a novel neuro-symbolic RL framework bridging the two paradigms based on Abductive Learning (ABL), enabling RL agents to learn abstractions directly from raw sensory inputs without predefined symbols.\r\nA2RL induces a finite state machine to represent high-level, step-by-step procedures, where each abstract state corresponds to a sub-algebra of the original Markov Decision Process (MDP). This approach not only bridges the gap between symbolic abstraction and sub-symbolic learning but also provides a natural mechanism for the emergence of new symbols. Experiments show that A2RL can mitigate the delayed reward problem and improve the generalization capability compared to traditional end-to-end RL methods.","doi":"10.24963/ijcai.2025/725","url":"https://ijcai.org/proceedings/2025/725","pdf":"https://ijcai.org/proceedings/2025/0725.pdf","markdown":"https://ijcai.org/proceedings/2025/725.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/725","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":725,"pages":{"first":6515,"last":6523},"keywords":["Machine Learning: ML: Multiagent Reinforcement Learning","Machine Learning: ML: Neuro-symbolic methods/Abductive Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}