{"type":"ScholarlyArticle","title":"Reinforced In-Context Black-Box Optimization","authors":[{"name":"Lei Song","slug":"lei-song","given":"Lei","family":"Song","affiliation":""},{"name":"Chen-Xiao Gao","slug":"chen-xiao-gao","given":"Chen-Xiao","family":"Gao","affiliation":""},{"name":"Ke Xue","slug":"ke-xue","given":"Ke","family":"Xue","affiliation":""},{"name":"Chenyang Wu","slug":"chenyang-wu","given":"Chenyang","family":"Wu","affiliation":""},{"name":"Dong Li","slug":"dong-li","given":"Dong","family":"Li","affiliation":""},{"name":"Jianye Hao","slug":"jianye-hao","given":"Jianye","family":"Hao","affiliation":""},{"name":"Zongzhang Zhang","slug":"zongzhang-zhang","given":"Zongzhang","family":"Zhang","affiliation":""},{"name":"Chao Qian","slug":"chao-qian","given":"Chao","family":"Qian","affiliation":""}],"abstract":"Black-Box Optimization (BBO) has found successful applications in many fields of science and engineering. Recently, there has been a growing interest in meta-learning particular components of BBO algorithms to speed up optimization and get rid of tedious hand-crafted heuristics. As an extension, learning the entire algorithm from data requires the least labor from experts and can provide the most flexibility. In this paper, we propose RIBBO, a method to reinforce-learn a BBO algorithm from offline data in an end-to-end fashion. RIBBO employs expressive sequence models to learn the optimization histories produced by multiple behavior algorithms and tasks, leveraging the in-context learning ability of large models to extract task information and make decisions accordingly. Central to our method is to augment the optimization histories with regret-to-go tokens, which are designed to represent the performance of an algorithm based on cumulative regret over the future part of the histories. The integration of regret-to-go tokens enables RIBBO to automatically generate sequences of query points that are positively correlated to the user-desired regret, verified by its universally good empirical performance on diverse problems, including BBO benchmark, hyper-parameter optimization, and robot control problems.","doi":"10.24963/ijcai.2025/994","url":"https://ijcai.org/proceedings/2025/994","pdf":"https://ijcai.org/proceedings/2025/0994.pdf","markdown":"https://ijcai.org/proceedings/2025/994.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/994","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Search","track":"Main Track","number":994,"pages":{"first":8939,"last":8947},"keywords":["Machine Learning: ML: Evolutionary learning","Search: S: Search and machine learning","Search: S: Evolutionary computation"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}