{"type":"ScholarlyArticle","title":"Efficient Diversity-based Experience Replay for Deep Reinforcement Learning","authors":[{"name":"Kaiyan Zhao","slug":"kaiyan-zhao","given":"Kaiyan","family":"Zhao","affiliation":"School of Computer Science , Wuhan University , Wuhan , China"},{"name":"Yiming Wang","slug":"yiming-wang","given":"Yiming","family":"Wang","affiliation":"State Key Laboratory of Internet of Things for Smart City , University of Macau , Macao , China"},{"name":"Yuyang Chen","slug":"yuyang-chen","given":"Yuyang","family":"Chen","affiliation":"School of Professional Education , Northwestern University , USA"},{"name":"Yan Li","slug":"yan-li","given":"Yan","family":"Li","affiliation":"School of Artificial Intelligence , Shenzhen Polytechnic University , China"},{"name":"Leong Hou U","slug":"leong-hou-u","given":"Leong Hou","family":"U","affiliation":"State Key Laboratory of Internet of Things for Smart City , University of Macau , Macao , China"},{"name":"Xiaoguang Niu","slug":"xiaoguang-niu","given":"Xiaoguang","family":"Niu","affiliation":"School of Computer Science , Wuhan University , Wuhan , China"}],"abstract":"Experience replay is widely used to improve learning efficiency in reinforcement learning by leveraging past experiences. However, existing experience replay methods, whether based on uniform or prioritized sampling, often suffer from low efficiency, particularly in real-world scenarios with high-dimensional state spaces. To address this limitation, we propose a novel approach, Efficient Diversity-based Experience Replay (EDER). EDER employs a determinantal point process to model the diversity between samples and prioritizes replay based on the diversity between samples. To further enhance learning efficiency, we incorporate Cholesky decomposition for handling large state spaces in realistic environments. Additionally, rejection sampling is applied to select samples with higher diversity, thereby improving overall learning efficacy. Extensive experiments are conducted on robotic manipulation tasks in MuJoCo, Atari games, and realistic indoor environments in Habitat. The results demonstrate that our approach not only significantly improves learning efficiency but also achieves superior performance in high-dimensional, realistic environments.","doi":"10.24963/ijcai.2025/788","url":"https://ijcai.org/proceedings/2025/788","pdf":"https://ijcai.org/proceedings/2025/0788.pdf","markdown":"https://ijcai.org/proceedings/2025/788.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/788","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":788,"pages":{"first":7083,"last":7091},"keywords":["Machine Learning: ML: Reinforcement learning"],"videos":[],"references":[{"position":1,"raw":"[Andrychowicz et al., 2017] Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Joshua Tobin, Pieter Abbeel, and Wojciech Zaremba. Hindsight experience replay. In Neu- ral Information Processing Systems, 2017.","title":"Hindsight experience replay","authors":"Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob Mcgrew, Joshua Tobin, Pieter Abbeel, Wojciech Zaremba","year":"2017","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrychowicz et al., 2020] Marcin Andrychowicz, Bob Baker, Marcin Chociej, Rafal Jozefowicz, Brian McGrew, Jakub Pachocki, Andrew Petron, Matthias Plappert, Geof- frey Powell, Alex Ray, et al. Learning dexterous in-hand manipulation. International Journal of Robotics Research, 2020.","title":"Learning dexterous in-hand manipulation","authors":"Marcin Andrychowicz, Bob Baker, Marcin Chociej, Rafal Jozefowicz, Brian Mcgrew, Jakub Pachocki, Andrew Petron, Matthias Plappert, Geoffrey Powell, Alex Ray","year":"2020","venue":"International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Azadi et al., 2018] Samaneh Azadi, Catherine Olsson, Trevor Darrell, Ian Goodfellow, and Augustus Odena. Discriminator rejection sampling. arXiv preprint arXiv:1810.06758, 2018.","title":"Discriminator rejection sampling","authors":"Azadi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Espeholt et al., 2018] Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Volodymyr Mnih, Tom Ward, and Martin Riedmiller. Impala: Scalable distributed deep- rl with importance weighted actor-learner architectures. In Proceedings of the 35th International Conference on Ma- chine Learning, pages 1407-1416. PMLR, 2018.","title":"Impala: Scalable distributed deeprl with importance weighted actor-learner architectures","authors":"Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Volodymyr Mnih, Tom Ward, Martin Riedmiller","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fang et al., 2019] Minshu Fang, Tao Zhou, Yifeng Du, Lei Han, and Zhe Zhang. Curriculum-guided hindsight expe- rience replay. In Advances in Neural Information Process- ing Systems, volume 32, 2019.","title":"Curriculum-guided hindsight experience replay","authors":"Minshu Fang, Tao Zhou, Yifeng Du, Lei Han, Zhe Zhang","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Franc ¸ois-Lavet et al., 2018] Vincent Franc ¸ois-Lavet, Peter Henderson, Riashat Islam, Marc G Bellemare, Joelle Pineau, et al. An introduction to deep reinforcement learn- ing. Foundations and Trends® in Machine Learning, 11(3- 4):219-354, 2018.","title":"Vincent Franc ¸ois-Lavet, Peter Henderson, Riashat Islam","authors":"Franc ¸ois-Lavet","year":"2018","venue":"Foundations and Trends® in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hare, 2019] Joshua Hare. Dealing with sparse rewards in reinforcement learning. arXiv preprint arXiv:1910.09281, 2019.","title":"Dealing with sparse rewards in reinforcement learning","authors":"Joshua Hare, Hare","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hong et al., 2022] Zhang-Wei Hong, Tao Chen, Yen-Chen Lin, Joni Pajarinen, and Pulkit Agrawal. Topological ex- perience replay. arXiv preprint arXiv:2203.15845, 2022.","title":"Topological experience replay","authors":"Zhang-Wei Hong, Tao Chen, Yen-Chen Lin, Joni Pajarinen, Pulkit Agrawal","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Horgan et al., 2018] Dan Horgan, John Quan, David Bud- den, Gabriel Barth-Maron, Matteo Hessel, Hado Van Has- selt, and David Silver. Distributed prioritized experience replay. arXiv preprint arXiv:1803.00933, 2018.","title":"Distributed prioritized experience replay","authors":"Dan Horgan, John Quan, David Budden, Gabriel Barth-Maron, Matteo Hessel, Hado Van Hasselt, David Silver","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ibrahimi et al., 2012] Morteza Ibrahimi, Adel Javanmard, and Benjamin Roy. Efficient reinforcement learning for high dimensional linear quadratic systems. Advances in Neural Information Processing Systems, 25, 2012.","title":"Efficient reinforcement learning for high dimensional linear quadratic systems","authors":"Morteza Ibrahimi, Adel Javanmard, Benjamin Roy","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jiang et al., 2024] Yiding Jiang, J Zico Kolter, and Roberta Raileanu. On the importance of exploration for general- ization in reinforcement learning. In Advances in Neural Information Processing Systems 36, 2024.","title":"On the importance of exploration for generalization in reinforcement learning","authors":"Jiang","year":"2024","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krishnamoorthy and Menon, 2013] Aravindh Krish- namoorthy and Deepak Menon. Matrix inversion using cholesky decomposition, 2013.","title":"Aravindh Krishnamoorthy and Deepak Menon","authors":"Menon Krishnamoorthy","year":"2013","venue":"Matrix inversion using cholesky decomposition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kulesza et al., 2012] Alex Kulesza, Ben Taskar, et al. De- terminantal point processes for machine learning. Founda- tions and Trends® in Machine Learning, 2012.","title":"Determinantal point processes for machine learning","authors":"Alex Kulesza, Ben Taskar","year":"2012","venue":"Foundations and Trends® in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kunaver and Požrl, 2017] Matevž Kunaver and Tomaž Požrl. Diversity in recommender systems-a survey. Knowledge-based systems, 123:154-162, 2017.","title":"Matevž Kunaver and Tomaž Požrl. Diversity in recommender systems-a survey","authors":"Požrl Kunaver","year":"2017","venue":"Knowledge-based systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lahire et al., 2022] Thibault Lahire, Matthieu Geist, and Emmanuel Rachelson. Large batch experience replay. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Pro- ceedings of the 39th International Conference on Ma- chine Learning, volume 162 of Proceedings of Machine Learning Research, pages 11790-11813. PMLR, 17-23 Jul 2022.","title":"Large batch experience replay","authors":"Matthieu Thibault Lahire, Emmanuel Geist, Rachelson","year":"2022","venue":"Proceedings of the 39th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Levine et al., 2016] Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. End-to-end training of deep visuomotor policies. Journal of Machine Learning Re- search, 2016.","title":"End-to-end training of deep visuomotor policies","authors":"Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel","year":"2016","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lillicrap et al., 2019] Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yu- val Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning, 2019.","title":"Continuous control with deep reinforcement learning","authors":"Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lin, 1992] Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8(3-4):293-321, 1992.","title":"Self-improving reactive agents based on reinforcement learning, planning and teaching","authors":"Long-Ji Lin","year":"1992","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Luo et al., 2023] Yongle Luo, Yuxin Wang, Kun Dong, Qiang Zhang, Erkang Cheng, Zhiyong Sun, and Bo Song. Relay hindsight experience replay: Self-guided continual reinforcement learning for sequential object manipulation tasks with sparse rewards. Neurocomputing, page 126620, 2023.","title":"Relay hindsight experience replay: Self-guided continual reinforcement learning for sequential object manipulation tasks with sparse rewards","authors":"Yongle Luo, Yuxin Wang, Kun Dong, Qiang Zhang, Erkang Cheng, Zhiyong Sun, Bo Song","year":"2023","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mnih et al., 2013] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning, 2013.","title":"Playing atari with deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mnih, 2013] Volodymyr Mnih. Playing atari with deep re- inforcement learning. arXiv preprint arXiv:1312.5602, 2013.","title":"Volodymyr Mnih. Playing atari with deep reinforcement learning","authors":"Mnih","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Neal, 2003] Radford M Neal. Slice sampling. The annals of statistics, 31(3):705-767, 2003.","title":"Slice sampling","authors":"M Radford, Neal","year":"2003","venue":"The annals of statistics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Peng et al., 2019] Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine. Advantage-weighted regres- sion: Simple and scalable off-policy reinforcement learn- ing, 2019.","title":"Advantage-weighted regression: Simple and scalable off-policy reinforcement learning","authors":"Xue Bin Peng, Aviral Kumar, Grace Zhang, Sergey Levine","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ramakrishnan et al., 2021] Santhosh K. Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X. Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai, 2021.","title":"Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai","authors":"K Santhosh, Aaron Ramakrishnan, Erik Gokaslan, Oleksandr Wijmans, Alex Maksymets, John Clegg, Eric Turner, Wojciech Undersander, Andrew Galuba, Angel X Westbury, Manolis Chang, Yili Savva, Dhruv Zhao, Batra","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Saglam et al., 2022] Berkay Saglam, Fatih Burak Mutlu, Dilek Cetin Cicek, et al. Actor prioritized experience re- play. arXiv preprint arXiv:2209.00532, 2022.","title":"Actor prioritized experience replay","authors":"Berkay Saglam, Fatih Burak Mutlu, Dilek Cetin Cicek","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Savva et al., 2019] Manolis Savva, Abhishek Kadian, Olek- sandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV), 2019.","title":"Habitat: A platform for embodied ai research","authors":"Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, Dhruv Batra","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sayar et al., 2024] Erdi Sayar, Zhenshan Bing, Carlo D'Eramo, Ozgur S Oguz, and Alois Knoll. Contact en- ergy based hindsight experience prioritization. In 2024 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 5434-5440. IEEE, 2024.","title":"Contact energy based hindsight experience prioritization","authors":"Erdi Sayar, Zhenshan Bing, Carlo D' Eramo, Ozgur S Oguz, Alois Knoll","year":"2024","venue":"2024 IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Schaul et al., 2015] Tom Schaul, John Quan, Ioannis Antonoglou, et al. Prioritized experience replay. arXiv preprint arXiv:1511.05952, 2015.","title":"Prioritized experience replay","authors":"Schaul","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Schrittwieser et al., 2020] Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, and Thore Graepel. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 2020.","title":"Mastering atari, go, chess and shogi by planning with a learned model","authors":"Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel","year":"2020","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Silver et al., 2017] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. Nature, 2017.","title":"Mastering the game of go without human knowledge","authors":"David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton","year":"2017","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sujit et al., 2023] Shivakanth Sujit, Somjit Nath, Pedro Braga, and Samira Ebrahimi Kahou. Prioritizing samples in reinforcement learning with reducible loss. Advances in Neural Information Processing Systems, 36:23237-23258, 2023.","title":"Prioritizing samples in reinforcement learning with reducible loss","authors":"Sujit","year":"2023","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sun et al., 2020] Peiquan Sun, Wengang Zhou, and Houqiang Li. Attentive experience replay. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5900-5907, 2020.","title":"Attentive experience replay","authors":"Sun","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Todorov et al., 2012] Emanuel Todorov, Tom Erez, and Yu- val Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012.","title":"Mujoco: A physics engine for model-based control","authors":"Emanuel Todorov, Tom Erez, Yuval Tassa","year":"2012","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2024a] Yiming Wang, Ming Yang, Renzhi Dong, Binbin Sun, Furui Liu, et al. Efficient potential- based exploration in reinforcement learning using inverse dynamic bisimulation metric. Advances in Neural Infor- mation Processing Systems, 36, 2024.","title":"Efficient potentialbased exploration in reinforcement learning using inverse dynamic bisimulation metric","authors":"Wang","year":"2024","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2024b] Yiming Wang, Kaiyan Zhao, Furui Liu, et al. Rethinking exploration in reinforcement learning with effective metric-based exploration bonus. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.","title":"Rethinking exploration in reinforcement learning with effective metric-based exploration bonus","authors":"Wang","year":"2024","venue":"The Thirty-eighth Annual Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wei et al., 2021] Qing Wei, Hailan Ma, Chunlin Chen, and Daoyi Dong. Deep reinforcement learning with quantum- inspired experience replay. IEEE Transactions on Cyber- netics, 52(9):9326-9338, 2021.","title":"Deep reinforcement learning with quantuminspired experience replay","authors":"Qing Wei, Hailan Ma, Chunlin Chen, Daoyi Dong","year":"2021","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yang et al., 2023a] Ming Yang, Renzhi Dong, Yiming Wang, Furui Liu, Yali Du, Mingliang Zhou, and Leong Hou U. Tiecomm: Learning a hierarchical communica- tion topology based on tie theory. In International Con- ference on Database Systems for Advanced Applications, pages 604-613. Springer, 2023.","title":"Tiecomm: Learning a hierarchical communication topology based on tie theory","authors":"Yang","year":"2023","venue":"International Conference on Database Systems for Advanced Applications","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yang et al., 2023b] Ming Yang, Yiming Wang, Yang Yu, Mingliang Zhou, et al. Mixlight: Mixed-agent coopera- tive reinforcement learning for traffic light control. IEEE Transactions on Industrial Informatics, 2023.","title":"Mixlight: Mixed-agent cooperative reinforcement learning for traffic light control","authors":"Yang","year":"2023","venue":"IEEE Transactions on Industrial Informatics","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yang et al., 2024] Ming Yang, Kaiyan Zhao, Yiming Wang, Renzhi Dong, Yali Du, Furui Liu, Mingliang Zhou, and Leong Hou U. Team-wise effective communication in multi-agent reinforcement learning. Autonomous Agents and Multi-Agent Systems, 38(2):36, 2024.","title":"Team-wise effective communication in multi-agent reinforcement learning","authors":"Ming Yang, Kaiyan Zhao, Yiming Wang, Renzhi Dong, Yali Du, Furui Liu, Mingliang Zhou, Leong Hou, U","year":"2024","venue":"Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yarats et al., 2021] Denis Yarats, Amy Zhang, Ilya Kostrikov, Brandon Amos, Joelle Pineau, and Rob Fergus. Improving sample efficiency in model-free reinforce- ment learning from images. In Proceedings of the aaai conference on artificial intelligence, volume 35, pages 10674-10681, 2021.","title":"Improving sample efficiency in model-free reinforcement learning from images","authors":"Denis Yarats, Amy Zhang, Ilya Kostrikov, Brandon Amos, Joelle Pineau, Rob Fergus","year":"2021","venue":"Proceedings of the aaai conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yu et al., 2024] Jiayu Yu, Jingyao Li, Shuai Lü, and Shuai Han. Mixed experience sampling for off-policy rein- forcement learning. Expert Systems with Applications, 251:124017, 2024.","title":"Mixed experience sampling for off-policy reinforcement learning","authors":"Jiayu Yu, Jingyao Li, Shuai Lü, Shuai Han","year":"2024","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2017] Jianan Zhang, Jost Tobias Springen- berg, Joschka Boedecker, and Wolfram Burgard. Deep reinforcement learning with successor features for navi- gation across similar environments. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), pages 5276-5281. IEEE, 2017.","title":"Deep reinforcement learning with successor features for navigation across similar environments","authors":"Jianan Zhang, Jost Tobias Springenberg, Joschka Boedecker, Wolfram Burgard","year":"2017","venue":"Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS)","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhao and Tresp, 2018] Rui Zhao and Volker Tresp. Energy- based hindsight experience prioritization. In Conference on Robot Learning. PMLR, 2018.","title":"Energybased hindsight experience prioritization","authors":"Rui Zhao, Volker Tresp","year":"2018","venue":"Conference on Robot Learning","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhou et al., 2019] Hongyao Zhou, Peng Zhang, Zhanxing Qiu, Haifeng He, and Wei Zhang. Multi-goal rein- forcement learning: Learning to learn and exploring within a complex and sparse environment. arXiv preprint arXiv:1902.06899, 2019.","title":"Multi-goal reinforcement learning: Learning to learn and exploring within a complex and sparse environment","authors":"Hongyao Zhou, Peng Zhang, Zhanxing Qiu, Haifeng He, Wei Zhang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}