---
title: "Solving Continuous Control with Episodic Memory"
authors:
  - "Igor Kuznetsov"
  - "Andrey Filchenkov"
doi: "10.24963/ijcai.2021/365"
url: "https://ijcai.org/proceedings/2021/365"
pdf: "https://ijcai.org/proceedings/2021/0365.pdf"
venue: "Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence"
year: 2021
pages: "2651-2657"
track: "Main Track"
keywords:
  - "Machine Learning: Reinforcement Learning"
  - "Machine Learning: Deep Reinforcement Learning"
---

# Solving Continuous Control with Episodic Memory

Igor Kuznetsov, Andrey Filchenkov

*Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence*, pages 2651-2657. DOI: [10.24963/ijcai.2021/365](https://doi.org/10.24963/ijcai.2021/365)

## Abstract

Episodic memory lets reinforcement learning algorithms remember and exploit promising experience from the past to improve agent performance. Previous works on memory mechanisms show benefits of using episodic-based data structures for discrete action problems in terms of sample-efficiency. The application of episodic memory for continuous control with a large action space is not trivial. Our study aims to answer the question: can episodic memory be used to improve agent's performance in continuous control? Our proposed algorithm combines episodic memory with Actor-Critic architecture by modifying critic's objective. We further improve performance by introducing episodic-based replay buffer prioritization. We evaluate our algorithm on OpenAI gym domains and show greater sample-efficiency compared with the state-of-the art model-free off-policy algorithms.