---
title: "From End-to-end to Step-by-step: Learning to Abstract via Abductive Reinforcement Learning"
authors:
  - "Zilong Wang"
  - "Jiongda Wang"
  - "Xiaoyong Chen"
  - "Meng Wang"
  - "Ming Ma"
  - "ZhiPeng Wang"
  - "Zhenyu Zhou"
  - "Tianming Yang"
  - "Wang-Zhou Dai"
doi: "10.24963/ijcai.2025/725"
url: "https://ijcai.org/proceedings/2025/725"
pdf: "https://ijcai.org/proceedings/2025/0725.pdf"
venue: "Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence"
year: 2025
pages: "6515-6523"
track: "Main Track"
keywords:
  - "Machine Learning: ML: Multiagent Reinforcement Learning"
  - "Machine Learning: ML: Neuro-symbolic methods/Abductive Learning"
---

# From End-to-end to Step-by-step: Learning to Abstract via Abductive Reinforcement Learning

Zilong Wang, Jiongda Wang, Xiaoyong Chen, Meng Wang, Ming Ma, ZhiPeng Wang, Zhenyu Zhou, Tianming Yang, Wang-Zhou Dai

*Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence*, pages 6515-6523. DOI: [10.24963/ijcai.2025/725](https://doi.org/10.24963/ijcai.2025/725)

## Abstract

Abstraction is a critical technique in general problem-solving, allowing complex tasks to be decomposed into smaller, manageable sub-tasks. While traditional symbolic planning relies on predefined primitive symbols to construct structured abstractions, its reliance on formal representations limits applicability to real-world tasks. On the other hand, reinforcement learning excels at learning end-to-end policies directly from sensory inputs in unstructured environments but struggles with compositional generalization in complex tasks with delayed rewards. In this paper, we propose Abductive Abstract Reinforcement Learning (A2RL), a novel neuro-symbolic RL framework bridging the two paradigms based on Abductive Learning (ABL), enabling RL agents to learn abstractions directly from raw sensory inputs without predefined symbols.
A2RL induces a finite state machine to represent high-level, step-by-step procedures, where each abstract state corresponds to a sub-algebra of the original Markov Decision Process (MDP). This approach not only bridges the gap between symbolic abstraction and sub-symbolic learning but also provides a natural mechanism for the emergence of new symbols. Experiments show that A2RL can mitigate the delayed reward problem and improve the generalization capability compared to traditional end-to-end RL methods.