{"type":"ScholarlyArticle","title":"Solving Continuous Control with Episodic Memory","authors":[{"name":"Igor Kuznetsov","slug":"igor-kuznetsov","given":"Igor","family":"Kuznetsov","affiliation":""},{"name":"Andrey Filchenkov","slug":"andrey-filchenkov","given":"Andrey","family":"Filchenkov","affiliation":""}],"abstract":"Episodic memory lets reinforcement learning algorithms remember and exploit promising experience from the past to improve agent performance. Previous works on memory mechanisms show benefits of using episodic-based data structures for discrete action problems in terms of sample-efficiency. The application of episodic memory for continuous control with a large action space is not trivial. Our study aims to answer the question: can episodic memory be used to improve agent's performance in continuous control? Our proposed algorithm combines episodic memory with Actor-Critic architecture by modifying critic's objective. We further improve performance by introducing episodic-based replay buffer prioritization. We evaluate our algorithm on OpenAI gym domains and show greater sample-efficiency compared with the state-of-the art model-free off-policy algorithms.","doi":"10.24963/ijcai.2021/365","url":"https://ijcai.org/proceedings/2021/365","pdf":"https://ijcai.org/proceedings/2021/0365.pdf","markdown":"https://ijcai.org/proceedings/2021/365.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/365","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Machine Learning","track":"Main Track","number":365,"pages":{"first":2651,"last":2657},"keywords":["Machine Learning: Reinforcement Learning","Machine Learning: Deep Reinforcement Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}