{"type":"ScholarlyArticle","title":"DeepWeave: Accelerating Job Completion Time with Deep Reinforcement Learning-based Coflow Scheduling","authors":[{"name":"Penghao Sun","slug":"penghao-sun","given":"Penghao","family":"Sun","affiliation":"National Digital Switching System Engineering & Technological R&D Center"},{"name":"Zehua Guo","slug":"zehua-guo","given":"Zehua","family":"Guo","affiliation":"Beijing Institute of Technology"},{"name":"Junchao Wang","slug":"junchao-wang","given":"Junchao","family":"Wang","affiliation":"National Digital Switching System Engineering & Technological R&D Center"},{"name":"Junfei Li","slug":"junfei-li","given":"Junfei","family":"Li","affiliation":"National Digital Switching System Engineering & Technological R&D Center"},{"name":"Julong Lan","slug":"julong-lan","given":"Julong","family":"Lan","affiliation":"National Digital Switching System Engineering & Technological R&D Center"},{"name":"Yuxiang Hu","slug":"yuxiang-hu","given":"Yuxiang","family":"Hu","affiliation":"National Digital Switching System Engineering & Technological R&D Center"}],"abstract":"To improve the processing efficiency of jobs in distributed computing, the concept of coflow is proposed. A coflow is a collection of flows that are semantically correlated in a multi-stage computation task. A job consists of multiple coflows and can be usually formulated as a Directed-Acyclic Graph (DAG). A proper scheduling of coflows can significantly reduce the completion time of jobs in distributed computing. However, this scheduling problem is proved to be NP-hard. Different from existing schemes that use hand-crafted heuristic algorithms to solve this problem, in this paper, we propose a Deep Reinforcement Learning (DRL) framework named DeepWeave to generate coflow scheduling policies. To improve the inter-coflow scheduling ability in the job DAG, DeepWeave employs a Graph Neural Network (GNN) to process the DAG information. DeepWeave learns from the history workload trace to train the neural networks of the DRL agent and encodes the scheduling policy in the neural networks, which make coflow scheduling decisions without expert knowledge or a pre-assumed model. The proposed scheme is evaluated with a simulator using real-life traces. Simulation results show that DeepWeave completes jobs at least 1.7X faster than the state-of-the-art solutions.","doi":"10.24963/ijcai.2020/458","url":"https://ijcai.org/proceedings/2020/458","pdf":"https://ijcai.org/proceedings/2020/0458.pdf","markdown":"https://ijcai.org/proceedings/2020/458.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/458","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning Applications","track":"Main track","number":458,"pages":{"first":3314,"last":3320},"keywords":["Machine Learning Applications: Applications of Reinforcement Learning","Machine Learning Applications: Networks"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24308/file_24308.m3u8","length_sec":299,"provider":"hls"}],"references":[{"position":1,"raw":"[Agarwal et al., 2018] Saksham Agarwal, Shijin Rajakrish- nan, Akshay Narayan, Rachit Agarwal, David Shmoys, and Amin Vahdat. Sincronia: near-optimal network de- sign for coflows. In ACM SIGCOMM, pages 16-29, 2018.","title":"Sincronia: near-optimal network design for coflows","authors":"Saksham Agarwal, Shijin Rajakrishnan, Akshay Narayan, Rachit Agarwal, David Shmoys, Amin Vahdat","year":"2018","venue":"ACM SIGCOMM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2016] Li Chen, Wei Cui, Baochun Li, and Bo Li. Optimizing coflow completion times with utility max-min fairness. In IEEE INFOCOM, pages 1-9, 2016.","title":"Optimizing coflow completion times with utility max-min fairness","authors":"Li Chen, Wei Cui, Baochun Li, Bo Li","year":"2016","venue":"IEEE INFOCOM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chowdhury and Stoica, 2012] Mosharaf Chowdhury and Ion Stoica. Coflow: a networking abstraction for cluster applications. In ACM HotNets, pages 31-36, 2012.","title":"Mosharaf Chowdhury and Ion Stoica. Coflow: a networking abstraction for cluster applications","authors":"Stoica Chowdhury","year":"2012","venue":"ACM HotNets","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chowdhury and Stoica, 2015] Mosharaf Chowdhury and Ion Stoica. Efficient coflow scheduling without prior knowledge. ACM SIGCOMM CCR, 45(4):393-406, 2015.","title":"Mosharaf Chowdhury and Ion Stoica. Efficient coflow scheduling without prior knowledge","authors":"Stoica Chowdhury","year":"2015","venue":"ACM SIGCOMM CCR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chowdhury et al., 2011] Mosharaf Chowdhury, Matei Za- haria, Justin Ma, Michael I Jordan, and Ion Stoica. Man- aging data transfers in computer clusters with orchestra. ACM SIGCOMM CCR, 41(4):98-109, 2011.","title":"Managing data transfers in computer clusters with orchestra","authors":"Chowdhury","year":"2011","venue":"ACM SIGCOMM CCR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chowdhury et al., 2014] Mosharaf Chowdhury, Yuan Zhong, and Ion Stoica. Efficient coflow scheduling with varys. ACM SIGCOMM CCR, 44(4):443-454, 2014.","title":"Efficient coflow scheduling with varys","authors":"Chowdhury","year":"2014","venue":"ACM SIGCOMM CCR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cof, 2014] Coflowsim. https://github.com/coflow/ coflowsim, 2014.","title":"Coflowsim","authors":"Cof","year":"2014","venue":null,"doi":null,"url":"https://github.com/coflow/coflowsim","internal_url":null},{"position":8,"raw":"[Cordeiro et al., 2010] Daniel Cordeiro, Grégory Mounié, Swann Perarnau, Denis Trystram, Jean-Marc Vincent, and Frédéric Wagner. Random graph generation for scheduling simulations. In SIMUTOOLS 2010, page 60, 2010.","title":"Random graph generation for scheduling simulations","authors":"Daniel Cordeiro, Grégory Mounié, Swann Perarnau, Denis Trystram, Jean-Marc Vincent, Frédéric Wagner","year":"2010","venue":"SIMUTOOLS 2010","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dogar et al., 2014] Fahad R Dogar, Thomas Karagiannis, Hitesh Ballani, and Antony Rowstron. Decentralized task- aware scheduling for data center networks. ACM SIG- COMM CCR, 44(4):431-442, 2014.","title":"Decentralized taskaware scheduling for data center networks","authors":"Thomas Fahad R Dogar, Hitesh Karagiannis, Antony Ballani, Rowstron","year":"2014","venue":"ACM SIG-COMM CCR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gao et al., 2016] Yuanxiang Gao, Hongfang Yu, Shouxi Luo, and Shui Yu. Information-agnostic coflow schedul- ing with optimal demotion thresholds. In IEEE ICC, pages 1-6, 2016.","title":"Information-agnostic coflow scheduling with optimal demotion thresholds","authors":"Gao","year":"2016","venue":"IEEE ICC","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Greensmith et al., 2004] Evan Greensmith, Peter L Bartlett, and Jonathan Baxter. Variance reduction techniques for gradient estimates in reinforcement learning. JMLR, 5:1471-1530, 2004.","title":"Variance reduction techniques for gradient estimates in reinforcement learning","authors":"Evan Greensmith, Peter L Bartlett, Jonathan Baxter","year":"2004","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2015] Zhe Huang, Bharath Balasubramanian, Michael Wang, Tian Lan, Mung Chiang, and Danny HK Tsang. Need for speed: Cora scheduler for optimizing completion-times in the cloud. In IEEE INFOCOM, pages 891-899, 2015.","title":"Need for speed: Cora scheduler for optimizing completion-times in the cloud","authors":"Zhe Huang, Bharath Balasubramanian, Michael Wang, Tian Lan, Mung Chiang, Danny Hk Tsang","year":"2015","venue":"IEEE INFOCOM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2016] Xin Sunny Huang, Xiaoye Steven Sun, and T Sunflow Ng. Sunflow: Efficient optical circuit scheduling for coflows. In ACM CONEXT, pages 297- 311, 2016.","title":"Sunflow: Efficient optical circuit scheduling for coflows","authors":"Sunny Xin, Xiaoye Huang, T Sunflow Steven Sun, Ng","year":"2016","venue":"ACM CONEXT","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kelley Jr, 1961] James E Kelley Jr. Critical-path planning and scheduling: Mathematical basis. Operations research, 9(3):296-320, 1961.","title":"Critical-path planning and scheduling: Mathematical basis","authors":"James E, Kelley Jr","year":"1961","venue":"Operations research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2016] Ziyang Li, Yiming Zhang, Dongsheng Li, Kai Chen, and Yuxing Peng. Optas: Decentralized flow monitoring and scheduling for tiny tasks. In IEEE INFO- COM, pages 1-9, 2016.","title":"Optas: Decentralized flow monitoring and scheduling for tiny tasks","authors":"Ziyang Li, Yiming Zhang, Dongsheng Li, Kai Chen, Yuxing Peng","year":"2016","venue":"IEEE INFO-COM","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mao et al., 2019] Hongzi Mao, Malte Schwarzkopf, Shaileshh Bojja Venkatakrishnan, Zili Meng, and Moham- mad Alizadeh. Learning scheduling algorithms for data processing clusters. In ACM SIGCOMM, pages 270-288, 2019.","title":"Learning scheduling algorithms for data processing clusters","authors":"Mao","year":"2019","venue":"ACM SIGCOMM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qiu et al., 2015] Zhen Qiu, Cliff Stein, and Yuan Zhong. Minimizing the total weighted completion time of coflows in datacenter networks. In ACM SPAA, pages 294-303, 2015.","title":"Minimizing the total weighted completion time of coflows in datacenter networks","authors":"Zhen Qiu, Cliff Stein, Yuan Zhong","year":"2015","venue":"ACM SPAA","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rusek et al., 2019] Krzysztof Rusek, José Suárez-Varela, Albert Mestres, Pere Barlet-Ros, and Albert Cabellos- Aparicio. Unveiling the potential of graph neural networks for network modeling and optimization in sdn. In ACM SOSR, pages 140-151, 2019.","title":"Unveiling the potential of graph neural networks for network modeling and optimization in sdn","authors":"Krzysztof Rusek, José Suárez-Varela, Albert Mestres, Pere Barlet-Ros, Albert Cabellos-Aparicio","year":"2019","venue":"ACM SOSR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shafiee and Ghaderi, 2018] Mehrnoosh Shafiee and Javad Ghaderi. An improved bound for minimizing the to- tal weighted completion time of coflows in datacenters. IEEE/ACM TON, 26(4):1674-1687, 2018.","title":"An improved bound for minimizing the total weighted completion time of coflows in datacenters","authors":"Ghaderi Shafiee, Mehrnoosh Shafiee, Javad Ghaderi","year":"2018","venue":"IEEE/ACM TON","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sutton et al., 2000] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approx- imation. In NIPS, pages 1057-1063, 2000.","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"David A Richard S Sutton, Mcallester, P Satinder, Yishay Singh, Mansour","year":"2000","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tan et al., 2019] Haisheng Tan, Shaofeng H-C Jiang, Yu- peng Li, Xiang-Yang Li, Chenzi Zhang, Zhenhua Han, and Francis Chi Moon Lau. Joint online coflow routing and scheduling in data center networks. IEEE/ACM ToN, 27(5):1771-1786, 2019.","title":"Joint online coflow routing and scheduling in data center networks","authors":"Haisheng Tan, H-C Shaofeng, Yupeng Jiang, Xiang-Yang Li, Chenzi Li, Zhenhua Zhang, Francis Han, Moon Chi, Lau","year":"2019","venue":"IEEE/ACM ToN","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[tpc, 2019] TPC Benchmark DS (TPC-DS). http://www.tpc. org/tpcds, 2019.","title":"TPC Benchmark DS (TPC-DS)","authors":null,"year":"2019","venue":null,"doi":null,"url":"http://www.tpc.org/tpcds,2019","internal_url":null},{"position":24,"raw":"[Wang et al., 2018a] Shuo Wang, Jiao Zhang, , Tao Huang, Tian Pan, Jiang Liu, and Yunjie Liu. Multi- attributes-based coflow scheduling without prior knowl- edge. IEEE/ACM ToN, 26(4):1962-1975, 2018.","title":"Multiattributes-based coflow scheduling without prior knowledge","authors":"Wang","year":"2018","venue":"IEEE/ACM ToN","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2018b] Shuo Wang, Jiao Zhang, Tao Huang, Jiang Liu, Tian Pan, and Yunjie Liu. A survey of coflow scheduling schemes for data center networks. IEEE Com- munications Magazine, 56(6):179-185, 2018.","title":"A survey of coflow scheduling schemes for data center networks","authors":"Wang","year":"2018","venue":"IEEE Communications Magazine","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2019] Zhiliang Wang, Han Zhang, Xingang Shi, Xia Yin, Yahui Li, Haijun Geng, Qianhong Wu, and Jianwei Liu. Efficient scheduling of weighted coflows in data centers. IEEE TPDS, 30(9):2003-2017, 2019.","title":"Efficient scheduling of weighted coflows in data centers","authors":"Zhiliang Wang, Han Zhang, Xingang Shi, Xia Yin, Yahui Li, Haijun Geng, Qianhong Wu, Jianwei Liu","year":"2019","venue":"IEEE TPDS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xin et al., 2013] Reynold S Xin, Josh Rosen, Matei Za- haria, Michael J Franklin, Scott Shenker, and Ion Stoica. Shark: Sql and rich analytics at scale. In ACM SIGMOD, pages 13-24, 2013.","title":"Shark: Sql and rich analytics at scale","authors":"Josh Reynold S Xin, Matei Rosen, Zaharia, J Michael, Scott Franklin, Ion Shenker, Stoica","year":"2013","venue":"ACM SIGMOD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu and Tang, 2015] Xiaoyong Xu and Maolin Tang. A new approach to the cloud-based heterogeneous mapre- duce placement problem. IEEE TSC, 9(6):862-871, 2015.","title":"A new approach to the cloud-based heterogeneous mapreduce placement problem","authors":"Xiaoyong Xu, Maolin Tang","year":"2015","venue":"IEEE TSC","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2016] Hong Zhang, Li Chen, Bairen Yi, Kai Chen, Mosharaf Chowdhury, and Yanhui Geng. Coda: To- ward automatically identifying and scheduling coflows in the dark. In ACM SIGCOMM, pages 160-173, 2016.","title":"Coda: Toward automatically identifying and scheduling coflows in the dark","authors":"Zhang","year":"2016","venue":"ACM SIGCOMM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhao et al., 2015] Yangming Zhao, Kai Chen, Wei Bai, Minlan Yu, Chen Tian, Yanhui Geng, Yiming Zhang, Dan Li, and Sheng Wang. Rapier: Integrating routing and scheduling for coflow-aware data center networks. In IEEE INFOCOM, pages 424-432, 2015.","title":"Rapier: Integrating routing and scheduling for coflow-aware data center networks","authors":"Yangming Zhao, Kai Chen, Wei Bai, Minlan Yu, Chen Tian, Yanhui Geng, Yiming Zhang, Dan Li, Sheng Wang","year":"2015","venue":"IEEE INFOCOM","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhou et al., 2018] Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. arXiv preprint arXiv:1812.08434, 2018.","title":"Graph neural networks: A review of methods and applications","authors":"Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}