{"type":"ScholarlyArticle","title":"VMAgent: A Practical Virtual Machine Scheduling Platform","authors":[{"name":"Junjie Sheng","slug":"junjie-sheng","given":"Junjie","family":"Sheng","affiliation":"School of Computer Science and Technology , East China Normal University , Shanghai , China"},{"name":"Shengliang Cai","slug":"shengliang-cai","given":"Shengliang","family":"Cai","affiliation":"School of Computer Science and Technology , East China Normal University , Shanghai , China"},{"name":"Haochuan Cui","slug":"haochuan-cui","given":"Haochuan","family":"Cui","affiliation":"School of Computer Science and Technology , East China Normal University , Shanghai , China"},{"name":"Wenhao Li","slug":"wenhao-li","given":"Wenhao","family":"Li","affiliation":"School of Computer Science and Technology , East China Normal University , Shanghai , China"},{"name":"Yun Hua","slug":"yun-hua","given":"Yun","family":"Hua","affiliation":"School of Computer Science and Technology , East China Normal University , Shanghai , China"},{"name":"Bo Jin","slug":"bo-jin","given":"Bo","family":"Jin","affiliation":"School of Computer Science and Technology , East China Normal University , Shanghai , China"},{"name":"Wenli Zhou","slug":"wenli-zhou","given":"Wenli","family":"Zhou","affiliation":"Algorithm Innovation Lab , Cloud BU , Huawei Technologies Co"},{"name":"Yiqiu Hu","slug":"yiqiu-hu","given":"Yiqiu","family":"Hu","affiliation":"Algorithm Innovation Lab , Cloud BU , Huawei Technologies Co"},{"name":"Lei Zhu","slug":"lei-zhu","given":"Lei","family":"Zhu","affiliation":"Alkaid Lab , Cloud BU , Huawei Technologies Co"},{"name":"Qian Peng","slug":"qian-peng","given":"Qian","family":"Peng","affiliation":"Alkaid Lab , Cloud BU , Huawei Technologies Co"},{"name":"Hongyuan Zha","slug":"hongyuan-zha","given":"Hongyuan","family":"Zha","affiliation":"The Chinese University of Hong Kong (Shenzhen) , Shenzhen , China"},{"name":"Xiangfeng Wang","slug":"xiangfeng-wang","given":"Xiangfeng","family":"Wang","affiliation":"School of Computer Science and Technology , East China Normal University , Shanghai , China"}],"abstract":"Virtual machine (VM) scheduling is one of the critical tasks in cloud computing. Many works have attempted to incorporate machine learning, especially reinforcement learning, to empower VM scheduling procedures. Although improved results are shown in several demo simulators, the performances in real-world scenarios are still underexploited. In this paper, we design a practical VM scheduling platform, i.e., VMAgent, to assist researchers in developing their methods on the VM scheduling problem. VMAgent consists of three components: simulator, scheduler, and visualizer. The simulator abstracts three general realistic scheduling scenarios (fading, recovering, and expansion) based on Huawei Cloud’s scheduling data, which is the core of our platform. Flexible configurations are further provided to make the simulator compatible with practical cloud computing architecture (i.e., Multi Non-Uniform Memory Access) and scenarios. Researchers then need to instantiate the scheduler to interact with the simulator, which is also pre-built in various types (e.g., heuristic, machine learning, and operations research) of scheduling algorithms to speed up the algorithm design. The visualizer, as an auxiliary component of the simulator and scheduler, facilitates researchers to conduct an in-depth analysis of the scheduling procedure and comprehensively compare different scheduling algorithms. We believe that VMAgent would shed light on the AI for the VM scheduling community, and the demo video is presented in https://bit.ly/vmagent-demo-video.","doi":"10.24963/ijcai.2022/860","url":"https://ijcai.org/proceedings/2022/860","pdf":"https://ijcai.org/proceedings/2022/0860.pdf","markdown":"https://ijcai.org/proceedings/2022/860.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/860","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Demo Track","track":"Demo Track","number":860,"pages":{"first":5944,"last":5947},"keywords":["Planning and Scheduling: Scheduling","Planning and Scheduling: Learning in Planning and Scheduling","Machine Learning: Deep Reinforcement Learning","Uncertainty in AI: Sequential Decision Making"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987207","length_sec":78,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985350","length_sec":334,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bays, 1977] Carter Bays. A comparison of next-fit, first-fit, and best-fit. Communications of the ACM, 20(3):191-192, 1977.","title":"A comparison of next-fit, first-fit, and best-fit","authors":"Carter Bays","year":"1977","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bello et al., 2016] Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. Neural combina- torial optimization with reinforcement learning. In ICLR, 2016.","title":"Neural combinatorial optimization with reinforcement learning","authors":"Irwan Bello, Hieu Pham, V Quoc, Mohammad Le, Samy Norouzi, Bengio","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brockman et al., 2016] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016.","title":"Openai gym","authors":"Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deudon et al., 2018] Michel Deudon, Pierre Cournut, Alexandre Lacoste, Yossiri Adulyasak, and Louis-Martin Rousseau. Learning heuristics for the TSP by policy gradient. In CPAIOR, pages 170-181, 2018.","title":"Learning heuristics for the TSP by policy gradient","authors":"Pierre Michel Deudon, Alexandre Cournut, Yossiri Lacoste, Louis-Martin Adulyasak, Rousseau","year":"2018","venue":"CPAIOR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Duan et al., 2019] Lu Duan, Haoyuan Hu, Yu Qian, Yu Gong, Xiaodong Zhang, Yinghui Xu, and Jiangwen Wei. A multi-task selected learning approach for solving 3D flexible bin packing problem. In AAMAS, pages 1386- 1394, 2019.","title":"A multi-task selected learning approach for solving 3D flexible bin packing problem","authors":"Lu Duan, Haoyuan Hu, Yu Qian, Yu Gong, Xiaodong Zhang, Yinghui Xu, Jiangwen Wei","year":"2019","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hadary et al., 2020] Ori Hadary, Luke Marshall, Ishai Men- ache, Abhisek Pan, Esaias E Greeff, David Dion, Star Dorminey, Shailesh Joshi, Yang Chen, Mark Russinovich, et al. Protean: VM allocation service at scale. In OSDI, pages 845-861, 2020.","title":"Protean: VM allocation service at scale","authors":"Hadary","year":"2020","venue":"OSDI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hu et al., 2017] Haoyuan Hu, Xiaodong Zhang, Xiaowei Yan, Longfei Wang, and Yinghui Xu. Solving a new 3D bin packing problem with deep reinforcement learning method. arXiv preprint arXiv:1708.05930, 2017.","title":"Solving a new 3D bin packing problem with deep reinforcement learning method","authors":"Haoyuan Hu, Xiaodong Zhang, Xiaowei Yan, Longfei Wang, Yinghui Xu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lameter, 2013] Christoph Lameter. NUMA (Non-Uniform Memory Access): An overview. Queue, 11(7):40-51, 2013.","title":"NUMA (Non-Uniform Memory Access): An overview","authors":"Christoph Lameter","year":"2013","venue":"Queue","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei Rusu, Joel Veness, Marc Bellemare, Alex Graves, Martin Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human- level control through deep reinforcement learning. Nature, 518(7540):529-533, 2015.","title":"Humanlevel control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei Rusu, Joel Veness, Marc Bellemare, Alex Graves, Martin Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, Demis Hassabis","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Mnih et al., 2016] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asyn- chronous methods for deep reinforcement learning. In ICML, pages 1928-1937, 2016.","title":"Asynchronous methods for deep reinforcement learning","authors":"Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Nair et al., 2020] Vinod Nair, Sergey Bartunov, Felix Gi- meno, Ingrid von Glehn, Pawel Lichocki, Ivan Lobov, Brendan O'Donoghue, Nicolas Sonnerat, Christian Tjan- draatmadja, Pengming Wang, Ravichandra Addanki, Tharindi Hapuarachchi, Thomas Keck, James Keeling, Pushmeet Kohli, Ira Ktena, Yujia Li, Oriol Vinyals, and Yori Zwols. Solving mixed integer programs using neural networks. arXiv:2012.13349, 2020.","title":"Solving mixed integer programs using neural networks","authors":"Vinod Nair, Sergey Bartunov, Felix Gimeno, Pawel Ingrid Von Glehn, Ivan Lichocki, Lobov, O' Brendan, Nicolas Donoghue, Christian Sonnerat, Pengming Tjandraatmadja, Ravichandra Wang, Tharindi Addanki, Thomas Hapuarachchi, James Keck, Pushmeet Keeling, Ira Kohli, Yujia Ktena, Oriol Li, Yori Vinyals, Zwols","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Nazari et al., 2018] Mohammadreza Nazari, Afshin Oroo- jlooy, Lawrence V Snyder, and Martin Takac. Reinforce- ment learning for solving the vehicle routing problem. In NeurIPS, pages 9861-9871, 2018.","title":"Reinforcement learning for solving the vehicle routing problem","authors":"Mohammadreza Nazari, Afshin Oroojlooy, Lawrence V Snyder, Martin Takac","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Sheng et al., 2022] Junjie Sheng, Yiqiu Hu, Wenli Zhou, Lei Zhu, Bo Jin, Jun Wang, and Xiangfeng Wang. Learn- ing to schedule multi-NUMA virtual machines via rein- forcement learning. Pattern Recognition, 121:108254, 2022.","title":"Learning to schedule multi-NUMA virtual machines via reinforcement learning","authors":"Junjie Sheng, Yiqiu Hu, Wenli Zhou, Lei Zhu, Bo Jin, Jun Wang, Xiangfeng Wang","year":"2022","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Wolke et al., 2015] Andreas Wolke, Boldbaatar Tsend- Ayush, Carl Pfeiffer, and Martin Bichler. More than bin packing: Dynamic resource allocation strategies in cloud data centers. Information Systems, 52:83-95, 2015.","title":"More than bin packing: Dynamic resource allocation strategies in cloud data centers","authors":"Andreas Wolke, Boldbaatar Tsend-Ayush, Carl Pfeiffer, Martin Bichler","year":"2015","venue":"Information Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}