{"type":"ScholarlyArticle","title":"Fast and Fine-grained Autoscaler for Streaming Jobs with Reinforcement Learning","authors":[{"name":"Mingzhe Xing","slug":"mingzhe-xing","given":"Mingzhe","family":"Xing","affiliation":"School of Computer Science , Peking University"},{"name":"Hangyu Mao","slug":"hangyu-mao","given":"Hangyu","family":"Mao","affiliation":"Huawei Noah's Ark Lab"},{"name":"Zhen Xiao","slug":"zhen-xiao","given":"Zhen","family":"Xiao","affiliation":"School of Computer Science , Peking University"}],"abstract":"On computing clusters, the autoscaler is responsible for allocating resources for jobs or fine-grained tasks to ensure their Quality of Service. Due to a more precise resource management, fine-grained autoscaling can generally achieve better performance. However, the fine-grained autoscaling for streaming jobs needs intensive computation to model the complicated running states of tasks, and has not been adequately studied previously. In this paper, we propose a novel fine-grained autoscaler for streaming jobs based on reinforcement learning. We first organize the running states of streaming jobs as spatio-temporal graphs. To efficiently make autoscaling decisions, we propose a Neural Variational Subgraph Sampler to sample spatio-temporal subgraphs. Furthermore, we propose a mutual-information-based objective function to explicitly guide the sampler to extract more representative subgraphs. After that, the autoscaler makes decisions based on the learned subgraph representations. Experiments conducted on real-world datasets demonstrate the superiority of our method over six competitive baselines.","doi":"10.24963/ijcai.2022/80","url":"https://ijcai.org/proceedings/2022/80","pdf":"https://ijcai.org/proceedings/2022/0080.pdf","markdown":"https://ijcai.org/proceedings/2022/80.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/80","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":80,"pages":{"first":564,"last":570},"keywords":["Data Mining: Parallel, Distributed and Cloud-based High Performance Mining","Data Mining: Mining Spatial and/or Temporal Data","Planning and Scheduling: Scheduling","Agent-based and Multi-agent Systems: Resource Allocation","Machine Learning: Deep Reinforcement Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986617","length_sec":81,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984760","length_sec":338,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Anzai, 2012] Yuichiro Anzai. Pattern recognition and ma- chine learning. Elsevier, 2012.","title":"Pattern recognition and machine learning","authors":"Anzai","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Fahs et al., 2020] Ali J. Fahs, Guillaume Pierre, and Erik Elmroth. Voilà: Tail-latency-aware fog application repli- cas autoscaler. In MASCOTS, 2020.","title":"Voilà: Tail-latency-aware fog application replicas autoscaler","authors":"Ali J Fahs, Guillaume Pierre, Erik Elmroth","year":"2020","venue":"MASCOTS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Garí et al., 2021] Yisel Garí, David A Monge, Elina Pacini, Cristian Mateos, and Carlos García Garino. Reinforce- ment learning-based application autoscaling in the cloud: A survey. Eng. Appl. Artif. Intell., 2021.","title":"Reinforcement learning-based application autoscaling in the cloud: A survey","authors":"Garí","year":"2021","venue":"Eng. Appl. Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Grandl et al., 2016] Robert Grandl, Srikanth Kandula, Sri- ram Rao, Aditya Akella, and Janardhan Kulkarni. {GRAPHENE}: Packing and dependency-aware schedul- ing for data-parallel clusters. In OSDI, 2016.","title":"{GRAPHENE}: Packing and dependency-aware scheduling for data-parallel clusters","authors":"Robert Grandl, Srikanth Kandula, Sriram Rao, Aditya Akella, Janardhan Kulkarni","year":"2016","venue":"OSDI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guo et al., 2019] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proc. of AAAI, 2019.","title":"Attention based spatialtemporal graph convolutional networks for traffic flow forecasting","authors":"Guo","year":"2019","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hamilton et al., 2017] William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Proc. of NeuIPS, 2017.","title":"Inductive representation learning on large graphs","authors":"Rex William L Hamilton, Jure Ying, Leskovec","year":"2017","venue":"Proc. of NeuIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang and Zitnik, 2020] Kexin Huang and Marinka Zitnik. Graph meta learning via local subgraphs. NeuIPS, 2020.","title":"Graph meta learning via local subgraphs","authors":"Zitnik Huang, Kexin Huang, Marinka Zitnik","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hueske and Kalavri, 2019] Fabian Hueske and Vasiliki Kalavri. Stream processing with Apache Flink: fundamen- tals, implementation, and operation of streaming applica- tions. O'Reilly Media, 2019.","title":"Stream processing with Apache Flink: fundamentals, implementation, and operation of streaming applications","authors":"Fabian Hueske, Vasiliki Kalavri","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma et al., 2015] Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparame- terization trick. NeuIPS, 2015.","title":"Variational dropout and the local reparameterization trick","authors":"Tim Durk P Kingma, Max Salimans, Welling","year":"2015","venue":"NeuIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kraskov et al., 2004] Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical review E, 2004.","title":"Estimating mutual information","authors":"Alexander Kraskov, Harald Stögbauer, Peter Grassberger","year":"2004","venue":"Physical review E","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mao et al., 2016] Hongzi Mao, Mohammad Alizadeh, Ishai Menache, and Srikanth Kandula. Resource management with deep reinforcement learning. In HotNets, 2016.","title":"Resource management with deep reinforcement learning","authors":"Mao","year":"2016","venue":"HotNets","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mondal et al., 2021] Shanka Subhra Mondal, Nikhil Sheo- ran, and Subrata Mitra. Scheduling of time-varying work- loads using reinforcement learning. In Proc. of AAAI, 2021.","title":"Scheduling of time-varying workloads using reinforcement learning","authors":"Shanka Subhra Mondal, Nikhil Sheoran, Subrata Mitra","year":"2021","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nguyen et al., 2020] Thanh-Tung Nguyen, Yu-Jin Yeom, Taehong Kim, Dae-Heon Park, and Sehan Kim. Horizontal pod autoscaling in kubernetes for elastic container orches- tration. Sensors, 2020.","title":"Horizontal pod autoscaling in kubernetes for elastic container orchestration","authors":"Thanh-Tung Nguyen, Yu-Jin Yeom, Taehong Kim, Dae-Heon Park, Sehan Kim","year":"2020","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ni et al., 2020] Xiang Ni, Jing Li, Mo Yu, Wang Zhou, and Kun-Lung Wu. Generalizable resource allocation in stream processing via deep reinforcement learning. In Proc. of AAAI, 2020.","title":"Generalizable resource allocation in stream processing via deep reinforcement learning","authors":"Xiang Ni, Jing Li, Mo Yu, Wang Zhou, Kun-Lung Wu","year":"2020","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nowozin et al., 2016] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In Proc. of NeuIPS, 2016.","title":"f-gan: Training generative neural samplers using variational divergence minimization","authors":"Sebastian Nowozin, Botond Cseke, Ryota Tomioka","year":"2016","venue":"Proc. of NeuIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Qiu et al., 2020] Haoran Qiu, Subho S Banerjee, Saurabh Jha, Zbigniew T Kalbarczyk, and Ravishankar K Iyer. {FIRM}: An intelligent fine-grained resource manage- ment framework for slo-oriented microservices. In OSDI, 2020.","title":"{FIRM}: An intelligent fine-grained resource management framework for slo-oriented microservices","authors":"Haoran Qiu, S Subho, Saurabh Banerjee, Jha, T Zbigniew, Ravishankar K Kalbarczyk, Iyer","year":"2020","venue":"OSDI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2019] Guang Sun, YingJie Song, ZiQin Gong, Xiya Zhou, Xinyi Zhou, and YiLin Bi. Survey on stream- ing data computing system. In ACM TUR-C, 2019.","title":"Survey on streaming data computing system","authors":"Guang Sun, Yingjie Song, Ziqin Gong, Xiya Zhou, Xinyi Zhou, Yilin Bi","year":"2019","venue":"ACM TUR-C","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sun et al., 2020] Penghao Sun, Zehua Guo, Junchao Wang, Junfei Li, Julong Lan, and Yuxiang Hu. Deepweave: Ac- celerating job completion time with deep reinforcement learning-based coflow scheduling. In Proc. of IJCAI, 2020.","title":"Deepweave: Accelerating job completion time with deep reinforcement learning-based coflow scheduling","authors":"Sun","year":"2020","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/458"},{"position":19,"raw":"[Sun et al., 2021] Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Yuanxing Ning, Philip S Yu, and Lifang He. Sugar: Subgraph neural network with reinforcement pooling and self-supervised mutual information mechanism. In Proc. of WWW, 2021.","title":"Sugar: Subgraph neural network with reinforcement pooling and self-supervised mutual information mechanism","authors":"Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Yuanxing Ning, Philip S Yu, Lifang He","year":"2021","venue":"Proc. of WWW","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Taherizadeh and Grobelnik, 2020] Salman Taherizadeh and Marko Grobelnik. Key influencing factors of the kuber- netes auto-scaler for computing-intensive microservice- native cloud-based applications. Adv. Eng. Softw, 2020.","title":"Salman Taherizadeh and Marko Grobelnik. Key influencing factors of the kubernetes auto-scaler for computing-intensive microservicenative cloud-based applications","authors":"Grobelnik Taherizadeh","year":"2020","venue":"Adv. Eng. Softw","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Verma et al., 2015] Abhishek Verma, Luis Pedrosa, Mad- hukar Korupolu, David Oppenheimer, Eric Tune, and John Wilkes. Large-scale cluster management at google with borg. In Proc. of EuroSys, 2015.","title":"Large-scale cluster management at google with borg","authors":"Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune, John Wilkes","year":"2015","venue":"Proc. of EuroSys","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Williams, 1992] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 1992.","title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","authors":"Williams Ronald","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ye et al., 2021] Junchen Ye, Leilei Sun, Bowen Du, Yanjie Fu, and Hui Xiong. Coupled layer-wise graph convolution for transportation demand prediction. In Proc. of AAAI, 2021.","title":"Coupled layer-wise graph convolution for transportation demand prediction","authors":"Ye","year":"2021","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu et al., 2018] Chao Yu, Yuebin Bai, Hailong Yang, Kun Cheng, Yuhao Gu, Zhongzhi Luan, and Depei Qian. Sm- guard: A flexible and fine-grained resource management framework for gpus. IEEE Transactions on Parallel and Distributed Systems, 2018.","title":"Smguard: A flexible and fine-grained resource management framework for gpus","authors":"Chao Yu, Yuebin Bai, Hailong Yang, Kun Cheng, Yuhao Gu, Zhongzhi Luan, Depei Qian","year":"2018","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2020] Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He. Graph information bottleneck for subgraph recognition. In Proc. of ICLR, 2020.","title":"Graph information bottleneck for subgraph recognition","authors":"Yu","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhi et al., 2021] Yuan Zhi, Zhan Tong, Limin Wang, and Gangshan Wu. Mgsampler: An explainable sampling strategy for video action recognition. In Proc. of ICCV, 2021.","title":"Mgsampler: An explainable sampling strategy for video action recognition","authors":"Yuan Zhi, Zhan Tong, Limin Wang, Gangshan Wu","year":"2021","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}