Proceedings

Solving Continuous Control with Episodic Memory

Igor Kuznetsov, Andrey Filchenkov

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 2651-2657.

https://doi.org/10.24963/ijcai.2021/365

Keywords

  • Machine Learning: Reinforcement Learning
  • Machine Learning: Deep Reinforcement Learning

Abstract

Episodic memory lets reinforcement learning algorithms remember and exploit promising experience from the past to improve agent performance. Previous works on memory mechanisms show benefits of using episodic-based data structures for discrete action problems in terms of sample-efficiency. The application of episodic memory for continuous control with a large action space is not trivial. Our study aims to answer the question: can episodic memory be used to improve agent's performance in continuous control? Our proposed algorithm combines episodic memory with Actor-Critic architecture by modifying critic's objective. We further improve performance by introducing episodic-based replay buffer prioritization. We evaluate our algorithm on OpenAI gym domains and show greater sample-efficiency compared with the state-of-the art model-free off-policy algorithms.

Igor Kuznetsov, and Andrey Filchenkov. 2021. Solving Continuous Control with Episodic Memory. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 2651–2657. https://doi.org/10.24963/ijcai.2021/365
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards