{"type":"ScholarlyArticle","title":"RLOP: A Framework for Reinforcement Learning, Optimization and Planning Algorithms","authors":[{"name":"Song Zhang","slug":"song-zhang","given":"Song","family":"Zhang","affiliation":"EII Inc. , Tokyo , Japan"}],"abstract":"Reinforcement learning, optimization, and planning/search are interconnected domains in artificial intelligence. Algorithms within these domains share many similarities. They complement each other in solving complex decision-making problems, and also offer opportunities for cross-disciplinary integration. However, conducting research on algorithms across these domains typically requires learning the specialized libraries. These libraries often couple algorithms with domain-specific problem classes, making it difficult to conduct cross-disciplinary researches. In order to solve this problem, we developed a generic and lightweight framework for reinforcement learning, optimization, and planning/search algorithms (RLOP). It implements only the core logic of algorithms, abstracting away domain-specific details by defining interface functions, which enables flexible customization and efficient integration across different domains. The framework has been open-sourced at https://github.com/songzhg/RLOP.","doi":"10.24963/ijcai.2024/1047","url":"https://ijcai.org/proceedings/2024/1047","pdf":"https://ijcai.org/proceedings/2024/1047.pdf","markdown":"https://ijcai.org/proceedings/2024/1047.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/1047","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Demo Track","track":"Demo Track","number":1047,"pages":{"first":8851,"last":8854},"keywords":["Planning and Scheduling: PS: Routing","Search: S: Combinatorial search and optimisation","Search: S: Local search","Search: S: Search and machine learning","Multidisciplinary Topics and Applications: MDA: Computer games","Constraint Satisfaction and Optimization: CSO: Solvers and tools","Planning and Scheduling: PS: Search in planning and scheduling","Planning and Scheduling: PS: Markov decisions processes","Search: S: Heuristic search","Planning and Scheduling: PS: Planning algorithms","Machine Learning: ML: Reinforcement learning","Search: S: Game playing"],"videos":[],"references":[{"position":1,"raw":"[Bello et al., 2016] Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. Neural combi- natorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940, 2016.","title":"Neural combinatorial optimization with reinforcement learning","authors":"Irwan Bello, Hieu Pham, V Quoc, Mohammad Le, Samy Norouzi, Bengio","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bertsimas and Tsitsiklis, 1993] Dimitris Bertsimas and John Tsitsiklis. Simulated annealing. Statistical science, 8(1):10-15, 1993.","title":"Simulated annealing","authors":"Dimitris Bertsimas, John Tsitsiklis","year":"1993","venue":"Statistical science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brockman et al., 2016] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016.","title":"Openai gym","authors":"Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dabas et al., 2022] Mayank Dabas, Nishthavan Dahiya, and Pratish Pushparaj. Solving connect 4 using artificial intel- ligence. In International Conference on Innovative Com- puting and Communications: Proceedings of ICICC 2021, Volume 1, pages 727-735. Springer, 2022.","title":"Solving connect 4 using artificial intelligence","authors":"Mayank Dabas, Nishthavan Dahiya, Pratish Pushparaj","year":"2022","venue":"International Conference on Innovative Computing and Communications: Proceedings of ICICC 2021","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Glover, 1990] Fred Glover. Tabu search: A tutorial. Inter- faces, 20(4):74-94, 1990.","title":"Fred Glover. Tabu search: A tutorial","authors":"Glover","year":"1990","venue":"Interfaces","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Haarnoja et al., 2018] Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905, 2018.","title":"Soft actor-critic algorithms and applications","authors":"Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jakob et al., 2017] Wenzel Jakob, Jason Rhinelander, and Dean Moldovan. pybind11-seamless operability be- tween c++ 11 and python. URL: https://github. com/pybind/pybind11, 2017.","title":"pybind11-seamless operability between c++ 11 and python","authors":"Jakob Wenzel, Jason Rhinelander, Dean Moldovan","year":null,"venue":null,"doi":null,"url":"https://github.com/pybind/pybind11,2017","internal_url":null},{"position":8,"raw":"[Knuth and Moore, 1975] Donald E Knuth and Ronald W Moore. An analysis of alpha-beta pruning. Artificial in- telligence, 6(4):293-326, 1975.","title":"An analysis of alpha-beta pruning","authors":"E Donald, Ronald W Knuth, Moore","year":"1975","venue":"Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lanctot et al., 2019] Marc Lanctot, Edward Lockhart, Jean- Baptiste Lespiau, Vinicius Zambaldi, Satyaki Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei, et al. Openspiel: A frame- work for reinforcement learning in games. arXiv preprint arXiv:1908.09453, 2019.","title":"Openspiel: A framework for reinforcement learning in games","authors":"Marc Lanctot, Edward Lockhart, Jean-Baptiste Lespiau, Vinicius Zambaldi, Satyaki Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Mnih et al., 2013] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.","title":"Playing atari with deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural in- formation processing systems, 32, 2019.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Raffin et al., 2019] Antonin Raffin, Ashley Hill, Maximil- ian Ernestus, Adam Gleave, Anssi Kanervisto, and Noah Dormann. Stable baselines3, 2019.","title":"Stable baselines3","authors":"Antonin Raffin, Ashley Hill, Maximilian Ernestus, Adam Gleave, Anssi Kanervisto, Noah Dormann","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ryan, 2003] Conor Ryan. Computer algorithms. In Robert A. Meyers, editor, Encyclopedia of Physical Sci- ence and Technology (Third Edition), pages 507-523.","title":"Computer algorithms","authors":"Conor Ryan","year":null,"venue":"Encyclopedia of Physical Science and Technology","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Academic Press, New York, third edition edition, 2003. Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":null,"year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Silver et al., 2017] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815, 2017.","title":"Mastering chess and shogi by self-play with a general reinforcement learning algorithm","authors":"David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Zhang et al., 2022] Tianyu Zhang, Amin Banitalebi- Dehkordi, and Yong Zhang. Deep reinforcement learning for exact combinatorial optimization: Learning to branch. In 2022 26th International Conference on Pattern Recognition (ICPR), pages 3105-3111. IEEE, 2022.","title":"Deep reinforcement learning for exact combinatorial optimization: Learning to branch","authors":"Tianyu Zhang, Amin Banitalebi-Dehkordi, Yong Zhang","year":"2022","venue":"26th International Conference on Pattern Recognition (ICPR)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}